I am doing embedded system programming.
our process is set as higher priority by default, however for some actions like invoking shell command, write file. I was thinking to lower its priority and then up it again. so it's kind of like a pair of function calling: "setdefaultpriority" and "improve priority".
And there are lots of shell command calling in our process. In one file, I may need to call tens of pair of "setdefault..." and "improve.."
My question, would so many priority operation in one process have any bad effect ?
setpriority in a non-root process can only go up (decrease priority), never down.
What you can do is decrease process priority in the child process, before it execs the shell command.
//errror checks ommited
#include <sys/resource.h>
#include <sys/time.h>
#include <stdio.h>
#include <unistd.h>
#include <assert.h>
#include <sys/wait.h>
int main()
{
pid_t pid;
pid=fork();
assert(pid>=0);
if (!pid){
execlp("nice", "nice", (char*)0);
_exit(1);
}
wait(0);
pid=fork();
if (!pid){
setpriority(PRIO_PROCESS, 0, 10);
execlp("nice", "nice", (char*)0);
_exit(1);
}
}
/* should print:
0
10
*/
The performance overhead of a system call as simple as setpriority should be negligible compared to the cost of fork and exec*.
Related
I want to run a particular MPI function under google benchmark. Something like:
#include <mpi.h>
#include <benchmark/benchmark.h>
template<class Real>
void MPIInitFinalize(benchmark::State& state)
{
auto mpi = []() {
MPI_Init(nullptr, nullptr);
foo();
MPI_Finalize();
};
for(auto _ : state) {
mpi();
}
}
BENCHMARK_TEMPLATE(MPIInitFinalize, double);
BENCHMARK_MAIN();
Of course, we know what will happen:
*** The MPI_Init() function was called after MPI_FINALIZE was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
I understand that MPI isn't cool with what I want to do. But google benchmark is simply too useful to not at least try to find a hack to make this work.
Is there anything that can be done? Can I fork a process and pass the lambda to it? Is there a threading pattern that will work? Even expensive things will be helpful, as I can just subtract the cost of doing whatever hack works without a call too foo() from the one which call foo().
If you don't need to include MPI_Init and MPI_Finalize in your time (which you probably don't want anyways) you can take alook at this gist: https://gist.github.com/mdavezac/eb16de7e8fc08e522ff0d420516094f5
It countains an example on how to benchmark MPI enabled code with google benchmark. The basic idea is to call google benchmark from your own main method (using ::benchmark::Initialize(&argc, argv) and ::benchmark::RunSpecifiedBenchmarks()), synchronize using MPI_Barrier, time your code using std::chrono::high_resolution_clock and using MPI_Allreduce to find the slowest process. You can then publish that time using state.SetIterationTime (but only on the main process).
I am running a script that does multiple subsequent mpirun calls through slurms squeue command. Each call to mpirun will write its output to an own directory, but there is a dependency between them in the way that a given run will use data from the former runs output directory.
The mpi program internally performs some iterative optimization algorithm, which will terminate if some convergence criteria are met. Every once in a while it happens, that the algorithm reaches a state in which those criteria are not quite met yet, but by plotting the output (which is continuosly written to disk) one can quite easily tell that the important things have converged and that further iterations would not change the nature of the final result anymore.
What I am therefore looking for is a way to manually terminate the run in a controlled way and have the outer script proceed to the next mpirun call. What is the best way to achieve this? I do not have direct access to the node on which the calculation is actually performed, but I have of course access to all of slurms commands and the working directories of the individual runs. I have access to the mpi programs full source code.
One solution that would work is the following: If one manually wants to terminate a run, one places a file with a special name like killme in the working directory, which could easily be done with touch killme. The mpi program would regulary check for the existence of this file and terminate in a controlled manner if it exists. The outer script or slurm would not be involved at all here and the script would just continue with the next mpirun call. What do you think of this solution? Can you think of anything better?
Here is a short code snippet for getting SIGUSR1 as a signal.
More detailed explanation can be found here.
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <string.h>
#include <unistd.h>
void sighandler(int signum, siginfo_t *info, void *ptr) {
fprintf(stderr, "Received signal %d\n", signum);
fprintf(stderr, "Signal originates from process %lu\n",
(unsigned long) info->si_pid);
fprintf(stderr, "Shutting down properly.\n");
exit(0);
}
int main(int argc, char** argv) {
struct sigaction act;
printf("pid %lu\n", (unsigned long) getpid());
memset(&act, 0, sizeof(act));
act.sa_sigaction = sighandler;
act.sa_flags = SA_SIGINFO;
sigaction(SIGUSR1, &act, NULL);
while (1) {
};
return 0;
}
nearly searching for hours I became more confused about the Multithreading concept and I need help understanding and implementing it. After searching I have come to the following implementation
main.cpp
#include <QtGui/QApplication>
#include "mainwindow.h"
#include <mythread1.h>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
mythread abc;
abc.start();
abc.wait();
return a.exec();
}
In above code mythread.h is my header file which I created for multithreading and below is my code of mythread1.h
mythread1.h
#ifndef MYTHREAD1_H
#define MYTHREAD1_H
#include <QtGui>
#include <windows.h>
class mythread : public QThread
{
public:
void run();
};
void mythread::run()
{
}
#endif // MYTHREAD_H
Now my questions are
I have defined only one function run() and when the thread is initialized how compiler determines it has to execute run() function and what it does when it has multiple functions, I got this question because in main.cpp I just typed abc.start(); and didn't specify what to execute?
Inside mythread1.h and in run function I made a infinite loop using while(0<1) and ran the program and to my surprise I was just presented with white screen with 100% CPU usage(not a surprise), but it should run concurrently without interfering with main thread right? then why this is happening?
I then used QFile::copy("pathtomytargetfile","targetpath"); inside the run function inside in mythread1.cpp but that didn't work :O and it didn't copy that file but when i connected it with Push button in main thread then it got successfully copied, why this happened?
Does anyone know a simpler way to implement multithreading?
Thank you
First of all: run method should be protected!
Secondly, what do you mean by "I have defined only one function run()". You defined method, not function. abc.start() means that abc instance of mythread will start and use it's entry point method void run(). You can't declare more than one method with same signature. And if you wonder how it calls your run() instead of QThread::run() then you need to read something about virtual methods in C++
Ad. 3: How can we tell why your file didn't copy? Check what errors QFile provides you. Change your code something like that:
QFile f("pathtomytargetfile");
if( ! f.copy("targetpath") )
{
qDebug() << f.errorString();
}
It will provide you some usefull info
My answer might confuse you even more, but the following links are worth reading in my opinion:
This is an article by the engineer who introduced the QThread class. He apologizes for suggesting that inheriting from QThread is the way to go.
This article shows you, how you should do it (strictly speaking -- inheriting from QThread will work as well, it just is not as nice design-wise).
On this page you can find an overview of multithreading techniques that Qt offers and some help deciding which one you should use for your specific problem.
HTH
I started writing some tests with Qt's unit testing system.
How do you usually organize the tests? It is one test class per one module class, or do you test the whole module with a single test class? Qt docs suggest to follow the former strategy.
I want to write tests for a module. The module provides only one class that is going to be used by the module user, but there is a lot of logic abstracted in other classes, which I would also like to test, besides testing the public class.
The problem is that Qt's proposed way to run tests involved the QTEST_MAIN macro:
QTEST_MAIN(TestClass)
#include "test_class.moc"
and eventually one test program is capable of testing just one test class. And it kinda sucks to create test projects for every single class in the module.
Of course, one could take a look at the QTEST_MAIN macro, rewrite it, and run other test classes. But is there something, that works out of the box?
So far I do it by hand:
#include "one.h"
#include "two.h"
int main(int argc, char *argv[])
{
QCoreApplication app(argc, argv);
TestOne one;
QTest::qExec(&one, argc, argv);
TestOne two;
QTest::qExec(&two, argc, argv);
}
Related to the answer posted by #cjhuitt
This is an example that removes the need of manually calling each test object
I TRY TO AVOID THINGS LIKE THIS:
MyTestClass1 t1; t1.run();
MyTestClass2 t2; t2.run();
//etc...
My solution is to let the test objects inherit from a base class that adds itself to a static list
The main program then executes all the test objects in that list. In that way, none of the supporting framework code needs to be changed. The only things that change are the test classes themselves.
Here is how i do it:
qtestsuite.h - base class for the test objects
#ifndef QTESTSUITE_H
#define QTESTSUITE_H
#include <QObject>
#include <vector>
class QTestSuite : public QObject
{
Q_OBJECT
public:
static std::vector<QObject*> m_suites;
public:
explicit QTestSuite();
};
#endif // QTESTSUITE_H
qtestsuite.cpp
#include "qtestsuite.h"
#include <iostream>
std::vector<QObject*> QTestSuite::m_suites;
QTestSuite::QTestSuite() : QObject()
{
m_suites.push_back(this);
}
testall.cpp - runs the tests
#include "qtestsuite.h"
#include <QtTest/QtTest>
#include <iostream>
int main(int, char**)
{
int failedSuitesCount = 0;
std::vector<QObject*>::iterator iSuite;
for (iSuite = QTestSuite::m_suites.begin(); iSuite != QTestSuite::m_suites.end(); iSuite++)
{
int result = QTest::qExec(*iSuite);
if (result != 0)
{
failedSuitesCount++;
}
}
return failedSuitesCount;
}
mytestsuite1.cpp - an example test object, create more of these
#include "qtestsuite.h"
#include <QtTest/QtTest>
class MyTestSuite1: public QTestSuite
{
Q_OBJECT
private slots:
void aTestFunction();
void anotherTestFunction();
};
void MyTestSuite1::aTestFunction()
{
QString str = "Hello";
QVERIFY(str.toUpper() == "this will fail");
}
void MyTestSuite1::anotherTestFunction()
{
QString str = "Goodbye";
QVERIFY(str.toUpper() == "GOODBYE");
}
static MyTestSuite1 instance; //This is where this particular test is instantiated, and thus added to the static list of test suites
#include "mytestsuite1.moc"
also, to create the .pro file
qmake -project "CONFIG += qtestlib"
In our setup with QTest, we did a few things to make it nicer.
Define a subclass of QObject that is used as a base class for any new unit-test class.
In the constructor for that class, we add the instance of the test to a static list of tests, and in the destructor we remove it.
We then have a static function that loops through the tests and runs them using QTest::qExec(). (We accumulate the values returned each time, and return that from our function.)
main() calls this function, and returns the result as the success/failure.
Finally, in the compilation unit of the specific test itself, we usually include a static instance of that class.
This setup means that the class will be instantiated before main() is run, so it will be added to the list of classes to test when main runs. The framework requires that you just need to inherit your class properly, and instantiate a static instance if you always want it run.
We also occasionally create other optional tests, that are added based on command line switches.
Yeah, QTest forces bit strange test structure and is generally inferior to Google Test/Mock Framework. For one project I'm forced to use QTest (client requirement), and here's how I use it:
I compile all test together as a subdir template project
To make creating new tests easier, I share a lot of project configuration by using common.pri file I include in every test .pro file
If possible I share the object files directory to speed up compilation
I run them all using a batch+awk+sed script.
Setting up this four points is very easy and makes usage of QTest almost pleasant. Do you have some problems with running multiple tests that are not solved by the config described above?
PS: running tests the way you do, i.e. calling multiple QTest::qExec causes problems with -o command line switch - you'll get only results for the last tested class.
Usually I organize tests with one test executable per class under test.
and eventually one test program is
capable of testing just one test
class.
This is a good thing. It isolates your tests from each other, preventing things like a crash in one test from blocking all your other tests. That crash could be caused by a common component in several classes under test. The pattern of the failures would then tip you off to the underlying origin of the problem. Basically, you have better diagnostic information for failures if your tests are independent of each other.
Make it easy to set up multiple executables and run each test separately. Use a test runner to spawn off all the test processes.
Update:
I've changed my mind on this somewhat. Once you have a big program with lots of tests, linking hundreds of test executables becomes very slow. My new preference is to put all the tests for a library into an executable and choosing which tests to invoke using command-line arguments passed to the test executable.
That cuts down the number of executables from hundreds to dozens, but retains the advantages of running tests separately.
I'm trying to fork 3 different child processes from a parent (and running this on a UNIX box), and I want to have this requirement :
The parent must wait till all the 3 children processes have finished executing.
I'm using wait for the same .. Here's the code snippet :
#include <unistd.h>
#include <sys/signal.h>
#include <sys/types.h>
#include <sys/wait.h>
int main()
{
int stat;
/* ... */
At last, in the parent, I do this :
wait (&stat);
/* ... */
return 0;
}
Question :
Do I need to call wait thrice or does a single call suffice?
I need to know how this works..
You have to issue three waits. Each wait blocks until a child exits or doesn't block if a child has already exited. See wait.
You have to wait three times.
Side note: If you don't want to block waiting for each to terminate in turn, you can instead install a signal handler for SIGCHLD and then call wait() to collect the return code once you know it is ready.