To wait for kernel termination on the host side I can do this:
error = clEnqueueNDRangeKernel(..., &event);
CHECK_ERROR(error);
clWaitForEvents(1, &event);
But is there a way to precise some maximal time of waiting? That is, if my kernel is not finished after, let's say, 10 seconds I'd like to continue, but if it takes only one second of execution I don't want to wait for the 9 remaining seconds.
There is no way to wait only for a given amount of time, however you can:
Check if the event has finished every N seconds via clGetEventInfo() and a loop.
Use clSetEventCallback() to define a function that will be called when the event completes. Sleep the current thread for 10 seconds, and make the event callback wake the thread if it finishes quicker than 10 seconds.
Related
I have a bit of confusion about the way BaseSensorOperator's parameters work: timeout & poke_interval.
Consider this usage of the sensor :
BaseSensorOperator(
soft_fail=True,
poke_interval = 4*60*60, # Poke every 4 hours
timeout = 12*60*60, # Timeout after 12 hours
)
The documentation mentions the timeout acts to set the task to 'fail' after it runs out. But I'm using a soft_fail=True, I don't think it retains the same behavior, because I've found the task failed instead of skipping after I've used both parameters soft_fail and timeout.
So what does happen here?
The sensor pokes every 4 hours, and at every poke, will wait for the duration of the timeout (12 hours)?
Or does it poke every 4 hours, for a total of 3 pokes, then times out?
Also, what happens with these parameters if I use the mode="reschedule"?
Here's the documentation of the BaseSensorOperator
class BaseSensorOperator(BaseOperator, SkipMixin):
"""
Sensor operators are derived from this class and inherit these attributes.
Sensor operators keep executing at a time interval and succeed when
a criteria is met and fail if and when they time out.
:param soft_fail: Set to true to mark the task as SKIPPED on failure
:type soft_fail: bool
:param poke_interval: Time in seconds that the job should wait in
between each tries
:type poke_interval: int
:param timeout: Time, in seconds before the task times out and fails.
:type timeout: int
:param mode: How the sensor operates.
Options are: ``{ poke | reschedule }``, default is ``poke``.
When set to ``poke`` the sensor is taking up a worker slot for its
whole execution time and sleeps between pokes. Use this mode if the
expected runtime of the sensor is short or if a short poke interval
is requried.
When set to ``reschedule`` the sensor task frees the worker slot when
the criteria is not yet met and it's rescheduled at a later time. Use
this mode if the expected time until the criteria is met is. The poke
inteval should be more than one minute to prevent too much load on
the scheduler.
:type mode: str
"""
Defining the terms
poke_interval: the duration b/w successive 'pokes' (evaluation the necessary condition that is being 'sensed')
timeout: Just poking indefinitely is inadmissible (if for e.g. your buggy code is poking on day to become 29 whenever month is 2, it will keep poking for upto 4 years). So we define a maximum period beyond which we stop poking and terminate (the sensor is marked either FAILED or SKIPPED)
soft_fail: Normally (when soft_fail=False), sensor is marked as FAILED after timeout. When soft_fail=True, sensor will instead be marked as SKIPPED after timeout
mode: This is a slightly complex
Any task (including sensor) when runs, eats up a slot in some pool (either default pool or explicitly specified pool); essentially meaning that it takes up some resources.
For sensors, this is
wasteful: as a slot is consumed even when we are just waiting (doing no actual work
dangerous: if your workflow has too many sensors that go into sensing around the same time, they can freeze a lot of resources for quite a bit. In fact too many having ExternalTaskSensors is notorious for putting entire workflows (DAGs) into deadlocks
To overcome this problem, Airflow v1.10.2 introduced modes in sensors
mode='poke' (default) means the existing behaviour that we discussed above
mode='reschedule' means after a poke attempt, rather than going to sleep, the sensor will behave as though it failed (in current attempt) and it's status will change from RUNNING to UP_FOR_RETRY. That ways, it will release it's slot, allowing other tasks to progress while it waits for another poke attempt
Citing the relevant snippet from code here
if self.reschedule:
reschedule_date = timezone.utcnow() + timedelta(
seconds=self._get_next_poke_interval(started_at, try_number))
raise AirflowRescheduleException(reschedule_date)
else:
sleep(self._get_next_poke_interval(started_at, try_number))
try_number += 1
For more info read Sensors Params section
And now answering your questions directly
Q1
The sensor pokes every 4 hours, and at every poke, will wait for the duration of the timeout (12 hours)?
Or does it poke every 4 hours, for a total of 3 pokes, then times out?
point 2. is correct
Q2
Also, what happens with these parameters if I use the
mode="reschedule"?
As explained earlier, each one of those params are independent and setting mode='reschedule' doesn't alter their behaviour in any way
BaseSensorOperator(
soft_fail=True,
poke_interval = 4*60*60, # Poke every 4 hours
timeout = 12*60*60, # Timeout of 12 hours
mode = "reschedule"
)
Let's say the criteria is not met at the first poke. So it will run again after 4 hours of interval. But the worker slot will be freed during the wait since we're using the mode="reschedule".
That is what I understood.
I have a QThread spawned off from my main that is doing a synchronous read on a device. The read has a 1000ms timeout. The read is wrapped in a forever loop, and the read is protected with a QMutex. The basic code is:
Thread 1 - Read forever on a device
for (;;){
readMutex.lock(); // Lock so that system cannot change device parameters in middle of read
read (&devicePtr, 1000);
readMutex.unlock; //Unlock so that thread 2 can grab lock if its waiting
}
I have another method that that runs under the main event loop that can set some parameters to the devicePointer. Its not safe to do that when the device is being read, so it tries to take the lock, then set the parameters, then unlock the mutex. This will allow the Thread 1 read loop to continue. The basic code is:
Thread 2 - Set device params
void setParm(){
readMutex.lock(); //Expects to take lock as soon as Thread 1 unlocks it
setParam (&devicePtr, PARAM);
readMutex.unlock(); //Unlock so thread 1 can resume its read forever loop
}
I have some qDebug in the code dumping the thread id when each of the thread take the lock. What I see is that Thread 2 calls the lock and blocks while thread 1 has the lock and is doing the read. Thread 1 completes the read and unlocks the Mutex. Thread 1 moves on to the next iteration of the loop, and takes the lock again. Thread 2 remains blocked on the readMutex.lock () call. This will have 5 or 6 times before Thread 2 is eventually allowed to take the lock and proceed.
I have assumed that the QMutex queue up the threads with round robin, but it doesn't seem so with Thread 1 able to take the lock back on the next iteration. I can sort of force Thread 2 to take the lock if I add a QThread::msleep(250) to the end of Thread 1's loop after the unlock. That sleep does the trick, and Thread 2 is able to take the lock right away and set the device parameters.
Am I doing something wrong, is this a priority thing? Any idea how I can make this round robin through the threads without using a msleep to put Thread 1 in the background?
There is no way to ensure the scheduler will attend one or the other thread. You could try changing the priority of the running threads to give a better chance to execute to one over the other, but it will no warranty anything and not all system supports priority change (ie:linux does not).
You should use a QWaitCondition instead of the sleep (you could pass the sleep value as a timeout for that condition) and make each thread wakes one pending thread waiting on that condition (QWaitCondition::wakeOne) before waiting on it (QWaitCondition::wait).
My requirement is to run Intel PIN tool for specified amount of time lets say around 1 minute and then terminate.
For example:
I want to run notepad.exe for 1 minute under PIN. After 1 minute do post processing and close the log files properly and terminate notepad.exe using PINTool.
Use PIN_ExitApplication() to achieve this. You can perform post processing in the Fini callback.
Let's say we start a QTimer with a 100ms interval at t0.
Let's say first timeout occurs at t0+100ms. Fine.
Let's say that, due to huge CPU load and/or lots of events having to be handled by the event loop, second timeout occurs at t0+230ms.
Let's say CPU is back to normal load. Is their any chance that third timeout could occur at t0+300ms (QTimer object realising it was late and trying to correct that by resynchronizing itself), or will it most likely timeout at t0+330ms?
Per QTimer documentation :
All timer types may time out later than expected if the system is busy or unable to provide the requested accuracy. In such a case of timeout overrun, Qt will emit activated() only once, even if multiple timeouts have expired, and then will resume the original interval.
I'm not sure I understand this correctly but, apparently, it won't resynchronize itself and third timeout will occur at t0+330ms.
Having a bit of trouble wrapping my head around Qtimer this morning.
Here's the basic idea:
I have a QTimer that is started by clicking a button.
Once it is active, every 5 minutes it calls a function (let's call it start() ).
Start() calls a seperate function (call it work() )using QtConcurrent. Thus, it returns quickly, even though processing is still occuring.
Here's my problem:
I want to call work() a number of times, once after the other. My issue is that currently, it will call the work() function multiple times before it has finished processing, which, since it interfaces to hardware, doesn't work.
How should I go about this properly?
EDIT:
Here is the basic flow of the program:
User clicks startTimer().
StartTime() calls timer->start()
when the timer emits a signal, it calls Start()
Start() does some light UI stuff, then makes does
future = QtConcurrent::run(...work()...) //work takes a long time
watcher->setFuture(*future)
somehow (this is my issue), when work() is done, I want to call it again (4 times, to be exact)
If I understand correctly your question you want to avoid running the work() function in multiple threads simultaneously.
Use a QMutexLocker at the top of your work() function and add the corresponding QMutex to your class definition.
THis way the execution of the work() function will be blocked until the previous execution has finished.
If your work() function had an anomaly where it took 20 minutes for some reason, would you want to then run the 4 next work() callbacks as soon as possible...or still want them to be spaced out by 5 minutes despite the delay?
If what you want is really 5 minute intervals, then you should use a single-shot timer that is re-queued each time you complete a work item. This way, you don't have the potential of winding up with a big blocked up queue of QTimer messages that can flood in and run all at once:
http://doc.qt.io/qt-4.8/qtimer.html#singleShot-prop