I have a question about when the deadline starts to count down? Does the deadline start to count down when arguments are marshaled and waiting to be sent, or after the arguments are sent out?
A deadline refers to a specific moment in time, usually in the future. It's like an alarm for a specific time, rather than a timer for a specific amount of time. So, for example, if you set a deadline of now + 1 second, it is by definition counting 1 second from when you performed that calculation. If you set the deadline to 4:30 PM today, it doesn't matter when you start counting, because that doesn't change when the deadline will be reached.
The client converts the deadline to a duration when sending it to the server, and then the server counts from when it receives it. Those calculations do not account for the transit time, so that timer will have a slight delay added. However, he deadline on the server is just for determining when it is no longer useful to continue processing a request, so it doesn't need to be precise.
Related
I have a bit of confusion about the way BaseSensorOperator's parameters work: timeout & poke_interval.
Consider this usage of the sensor :
BaseSensorOperator(
soft_fail=True,
poke_interval = 4*60*60, # Poke every 4 hours
timeout = 12*60*60, # Timeout after 12 hours
)
The documentation mentions the timeout acts to set the task to 'fail' after it runs out. But I'm using a soft_fail=True, I don't think it retains the same behavior, because I've found the task failed instead of skipping after I've used both parameters soft_fail and timeout.
So what does happen here?
The sensor pokes every 4 hours, and at every poke, will wait for the duration of the timeout (12 hours)?
Or does it poke every 4 hours, for a total of 3 pokes, then times out?
Also, what happens with these parameters if I use the mode="reschedule"?
Here's the documentation of the BaseSensorOperator
class BaseSensorOperator(BaseOperator, SkipMixin):
"""
Sensor operators are derived from this class and inherit these attributes.
Sensor operators keep executing at a time interval and succeed when
a criteria is met and fail if and when they time out.
:param soft_fail: Set to true to mark the task as SKIPPED on failure
:type soft_fail: bool
:param poke_interval: Time in seconds that the job should wait in
between each tries
:type poke_interval: int
:param timeout: Time, in seconds before the task times out and fails.
:type timeout: int
:param mode: How the sensor operates.
Options are: ``{ poke | reschedule }``, default is ``poke``.
When set to ``poke`` the sensor is taking up a worker slot for its
whole execution time and sleeps between pokes. Use this mode if the
expected runtime of the sensor is short or if a short poke interval
is requried.
When set to ``reschedule`` the sensor task frees the worker slot when
the criteria is not yet met and it's rescheduled at a later time. Use
this mode if the expected time until the criteria is met is. The poke
inteval should be more than one minute to prevent too much load on
the scheduler.
:type mode: str
"""
Defining the terms
poke_interval: the duration b/w successive 'pokes' (evaluation the necessary condition that is being 'sensed')
timeout: Just poking indefinitely is inadmissible (if for e.g. your buggy code is poking on day to become 29 whenever month is 2, it will keep poking for upto 4 years). So we define a maximum period beyond which we stop poking and terminate (the sensor is marked either FAILED or SKIPPED)
soft_fail: Normally (when soft_fail=False), sensor is marked as FAILED after timeout. When soft_fail=True, sensor will instead be marked as SKIPPED after timeout
mode: This is a slightly complex
Any task (including sensor) when runs, eats up a slot in some pool (either default pool or explicitly specified pool); essentially meaning that it takes up some resources.
For sensors, this is
wasteful: as a slot is consumed even when we are just waiting (doing no actual work
dangerous: if your workflow has too many sensors that go into sensing around the same time, they can freeze a lot of resources for quite a bit. In fact too many having ExternalTaskSensors is notorious for putting entire workflows (DAGs) into deadlocks
To overcome this problem, Airflow v1.10.2 introduced modes in sensors
mode='poke' (default) means the existing behaviour that we discussed above
mode='reschedule' means after a poke attempt, rather than going to sleep, the sensor will behave as though it failed (in current attempt) and it's status will change from RUNNING to UP_FOR_RETRY. That ways, it will release it's slot, allowing other tasks to progress while it waits for another poke attempt
Citing the relevant snippet from code here
if self.reschedule:
reschedule_date = timezone.utcnow() + timedelta(
seconds=self._get_next_poke_interval(started_at, try_number))
raise AirflowRescheduleException(reschedule_date)
else:
sleep(self._get_next_poke_interval(started_at, try_number))
try_number += 1
For more info read Sensors Params section
And now answering your questions directly
Q1
The sensor pokes every 4 hours, and at every poke, will wait for the duration of the timeout (12 hours)?
Or does it poke every 4 hours, for a total of 3 pokes, then times out?
point 2. is correct
Q2
Also, what happens with these parameters if I use the
mode="reschedule"?
As explained earlier, each one of those params are independent and setting mode='reschedule' doesn't alter their behaviour in any way
BaseSensorOperator(
soft_fail=True,
poke_interval = 4*60*60, # Poke every 4 hours
timeout = 12*60*60, # Timeout of 12 hours
mode = "reschedule"
)
Let's say the criteria is not met at the first poke. So it will run again after 4 hours of interval. But the worker slot will be freed during the wait since we're using the mode="reschedule".
That is what I understood.
Background: I am developing a small game and use the player's latency to do lag compensation. The game is open sourced, so at the moment it is a very easy task to reverse engineer the system and delay ones response time to artificially increment ones reported delay, resulting in possibly unfair advantages.
My current strategy for latency retrieval is:
Every fixed interval I send a message labeled as "ping" to a player. (This has nothing to do with ICMP)
This ping message consists of a special "ping" opcode and a payload with a sequence number
Once the client receives said message, he sends back one with a "pong" opcode and a payload with the same sequence number
When the server receives the message labeled as "pong", it calculates how much time passed in between sending and receiving. This is the round trip time
Our latency is the rtt / 2
In pseudo code
Server:
function now() {
return current UTC time in millis
}
i = 0
function nextSequence() {
return i++
}
sendingTimestamps = []
function onPingEvent() {
id = nextSequence()
sendingTimestamps[id] = now()
sendPingMessage(id)
}
function onPongReceived(id) {
received = now()
sent = sendingTimestamps[id]
rtt = received - sent
latency = rtt / 2
}
Client:
function onPingReceived(id) {
sendPongMessage(id)
}
As you can see, it's very easy for the client to just add a delay in his code to inflate his reported latency.
Is there a better way to get a clients latency in order to leave them less room for cheating?
Answer below is a summary of topics discussed in comments to have them all in one place.
Lag compensation should rely on precise time stamp of event rather than average packet delay
Transition time may drastically vary even for two successive packets. Suggested approach with measuring average latency and assuming, that each received packet was sent "latency" ms ago for lag compensation is way too inaccurate. The following scheme should be applied instead:
Server starts emulating world on its side and sends command START to all clients. Clients initiate emulating world and count ticks from its creation. Whenever any event occurs on client side, client sends it with timestamp to server. Like "user pressed fire at tick #183". Server's emulation of game is far ahead due to packet transition time, but server can "go back in time" to handle user's order and resolve consequences.
Time stamps and events still can be faked
AFAIU problem of verifying client input is generally unsolvable. Any algorithm implemented in client can be recreated to fake events/timestamps/packets. Closed code can be reversed, so it is not an answer. Even world wide spread games like Counter-Strike or OverWatch have cheaters, despite they are developed by large companies, which, I bet, have separate department focused solely on game security. Some companies develop antivirus like modules, which check game file integrity or hash of parts of RAM snapshot, but it still can be bypassed.
The question is amount of efforts required to fake algorithm. The more efforts needed the less fakers will be. Trivial timestamp verifycation is the following:
If you receive event#2 in TCP stream after event#1, but its time stamp is before event#1, then it's faked.
If time stamp is far behind server's time, then warn and kick player for enormously bad delay. If it's a real player, the game anyway is unplayable for him, otherwise you kicked hacker. CS servers do this if I'm not mistaken.
Let's say we start a QTimer with a 100ms interval at t0.
Let's say first timeout occurs at t0+100ms. Fine.
Let's say that, due to huge CPU load and/or lots of events having to be handled by the event loop, second timeout occurs at t0+230ms.
Let's say CPU is back to normal load. Is their any chance that third timeout could occur at t0+300ms (QTimer object realising it was late and trying to correct that by resynchronizing itself), or will it most likely timeout at t0+330ms?
Per QTimer documentation :
All timer types may time out later than expected if the system is busy or unable to provide the requested accuracy. In such a case of timeout overrun, Qt will emit activated() only once, even if multiple timeouts have expired, and then will resume the original interval.
I'm not sure I understand this correctly but, apparently, it won't resynchronize itself and third timeout will occur at t0+330ms.
When I have call request in AGI script, is it possible to get time since call was created? For example, if it was routed sequentially through several softswitches can I get whole time sinse first softswitch or since generation in remote softphone ?
SIP only is used.
I also use AMI, and in CDR event I see that (Duration - BillableSeconds) is sometimes longer (for 10-20 seconds) then time from AGI request to Up, so duration may be already includes some time before call passed to my Asterisk.
No, there are no way determine any history of call before it go your switch unless other switch inform you about by special headers.
No, CDR(duration) is counted from time when call arived to asterisk. CDR(billsec) always less or equal to duration.
When a workflow has a receive activity that occurs after another receive activity and the second receive activity is called first the workflow holds the caller by blocking for 1 minute before timing out.
I want the workflow to return immediately when there are no matching workflow instances.
I do not want to change the timeout on the client as some calls may take a while.
This is a known issue in WF4, at least I am not aware of it being fixed yet.