Would anyone know what is the implication of setting ackAfterHandle to false in Spring Kafka's DefaultErrorHandler?
Does it
Essentially causes the application to be "stuck" and unable to move on to processing the next message, or
Not commit the offset, but moves on to the offset after, which in essence skips the message that caused the error?
Intuitively, I would expect that if I set ackAfterHandle to false, it #1 is the expected behaviour, but my testing suggests it would #2 is a more accurate description of the behaviour.
If it is indeed #2, what is the point of ackAfterHandle given that regardless of whether it is true and false, in essence the error is discarded and subsequent messages processed?
Thanks!
#2; that is why it is true by default.
It doesn't make much sense to ever set it to false, but it is there for historical reasons (early versions of the framework never committed the offsets of "handled" error records).
To achieve #1 behavior use the CommonContainerStoppingErrorHandler.
Related
I'm using ReplyingKafkaTemplate to make synchronous call with reply. From what i found till now, every time i'm going to use template i call start() and after receiving response the stop() method. However, I came across a problem with the message commit, the offset of my consumer was not increasing. I assumed, that is because consumer did not have time to make a commit, because basic commit time (property "auto.commit.interval.ms") is set to 5 seconds in ConsumerConfig class and I'm stopping him immediatelly after receiving a message. So i changed this time to 1 ms, to commit immediatelly after receiving message. This way it's working, but i would like to understand it better
My question is : How start() and stop() methods should be used properly, is there a purpose to start it before every call and stop after ? And what is a right way to make sure that commit was made ?
Btw. I would be honored if Gary answered the question
You should not start and stop the container each time; just leave the reply container running all the time.
In any case, you should leave enable.auto.commit alone - while it's default is true, Spring will set it to false unless you explicitly set it to true.
The container will commit the offset in a more deterministic manner than the built-in auto commit mechanism.
According to https://learn.microsoft.com/en-us/windows/desktop/api/mfidl/nf-mfidl-imfmediasession-close , once the IMFMediaSession.Close is called, i am supposed to receive an event called MESessionClosed, which i am not getting always, but in most cases.
I got a few customers with growing native memory leaks, and i think that one of the reasons is either what i mentioned above, or MediaFoundation interaction with the GPU driver, since i have analyzed dumps where i saw thousands of threads open in atiumd64.dll, method OpenAdapter:
00 000000b0`cecff8f8 00007ff8`c1cf9252 ntdll!NtWaitForSingleObject+0x14
01 000000b0`cecff900 00007ff8`752d2ccd KERNELBASE!WaitForSingleObjectEx+0xa2
02 000000b0`cecff9a0 00007ff8`757bf247 atiumd64!OpenAdapter+0x63ced
03 000000b0`cecff9d0 00007ff8`757bf3ee
atiumd64!XdxInitXopAdapterServices+0x3d0a57
04 000000b0`cecffa00 00007ff8`c4293034
atiumd64!XdxInitXopAdapterServices+0x3d0bfe
05 000000b0`cecffa30 00007ff8`c4d91461 kernel32!BaseThreadInitThunk+0x14
06 000000b0`cecffa60 00000000`00000000 ntdll!RtlUserThreadStart+0x21
I had a total of 160000 topologies created over the span of 4 days, and some 100 did not raise the MESessionClosed at all, and i fear these are the ones which cause a leak.
In cases where no MESessionClosed is sent, i notice that they all have one error in common: -1072870850, which is MF_E_SAMPLEALLOCATOR_EMPTY.
I would love to know if anyone has had experience with MediaFoundation not raising MESessionClosed according to documentation.
MESessionClosed event is created as a result of completion of asynchronously executed IMFMediaSession::Close call. Your not getting indicates a closing problem, perhaps a problem with one of the primitives participating in the topology, such as for example, failure to end streaming because of outstanding or leaked reference on some object.
Given the description of the problem perhaps the best way to address the problem is to attach debugger to the process (live or creating a dump and reviewing it interactively) expecting to find a thread waiting for something to close or complete.
Your seeing MF_E_SAMPLEALLOCATOR_EMPTY earlier might suggest that a leaked pointer to one of the samples prevents from terminating a sample allocator inside one of the primitives, which in turn create a deadlock.
Other than this you might want to do mftrace on the process and compare output produced by closed session to the other one that fails.
One thing you are also interested in, including putting it as a part of the question, is understanding the topology and especially whether it has third party or optional segments you can temporarily exclude. Since you cannot do much of debugging of MF internals directly, your options to change the topology could help you narrow the scope of the issue to specific primitive which is giving you the trouble. If the topology has your own primitives, you are interested in reviewing their termination behavior.
This is the 1st time I meet this.
Normally when we received an inbound X12 file. A 999 will always be generated (by configuration in BizTalk) and in case of interchange level error occurs, a TA1 will be created.
But today I got a X12 file with some formatting errors, the error popup in BizTalk is :
Delimiters are not unique, field and component seperator are the same.
The sequence number of the suspended message is 1.
I am expecting to have a 999 or TA1 generated to reject the inbound file. but none of these 2 files created.
My question:
What file shall I expect to created for this kind of error? 999 or
TA1?
Is this a bug or normal behavior for BizTalk?
If this is normal, what is the best mechanism to catch this error and
response back to trading partner.
You should definitely not expect a 999 (which would be transaction set specific), because this error prevents BizTalk from parsing the transaction set at all - it doesn't have a reliable way to determine what kind of transaction it is.
A TA1 could be appropriate, but this seems like a grey area - might be worth contacting Microsoft support about. The documentation indicates that an invalid ISA should result in a negative TA1, but the error codes for TA1 don't list this particular scenario as one that's support (or at all).
A possible work around would be capturing this kind of message, generating a TA1 for it, and routing it back to the TP. However, having non-unique delimiters may make it impossible to determine the TP from the message itself, even though you might be able to determine it from context (but maybe not if multiple trading partners use the same ports/locations). My guess is that's why BizTalk isn't handling it properly out of the box. To be honest, unless this happens fairly frequently it'd probably be easier/more realible to deal with it on an exception basis with human intervention.
As far as capturing the message, I'm thinking you'd need a custom pipeline component - perhaps even subclassing the EdiDisassembler so you can catch this particular exception and deal with it.
Two processes X and Y need to access a critical section. Consider the following synchronization construct used by both the processes.
http://d18khu5s3lkxd9.cloudfront.net//wp-content/uploads/2015/02/Q20.png
In the link given above,
varP and varQ are shared variables and both are initialized to false. Which one of the following statements is true?
1.The proposed solution prevents deadlock but fails to guarantee mutual exclusion
2.The proposed solution guarantees mutual exclusion but fails to prevent deadlock
3.The proposed solution guarantees mutual exclusion and prevents deadlock
4.The proposed solution fails to prevent deadlock and fails to guarantee mutual exclusion
According to the question setter 4th answer is the correct answer.
I have figured that it fails to guarantee mutual exclusion but how does it fails to prevent deadlock?
I came up with this after studying the algo carefully.
Say process Y has used the Critical Section.Therefore,it must have set VarQ variable as false.
Now if Process X tries to enter Critical Section.It can never enter unless Process Y also tries to enter.Reason being the condition while(varQ == true) will remain false unless Process Y tries to enter Critical Section and in doing so sets VarQ to true which before leaving Critical Section(CS) it had set to false.
So as we can see if Process Y does not tries to enter CS,Process X is indefinitely blocked and also the Critical Section is lying unused.
But the question still remains that how is lack of starvation freedom leading to lack of deadlock freedom.In deadlock every process is blocked,but if Process Y indeed tried to enter CS again,Process X could have been successful in its attempt to enter CS.
I'd like to learn more about Timed Automata to verify real-time systems. Currently, I'm trying to get myself familiar with a tool called UPPAAL.
I drew some simple graphs and added different properties. The entire model is supposed to represent a production cell system where different mechanical units pass a block to each other.
I've modelled the block (BlockCycle), 2 mechanical units (Feeder, FeederBelt) and 2 sensors which sense the arrival of the block.
Even though I thought my system would make sense, it gets deadlocked:
The target states of this transition violate the invariants. This is not a problem, as long as you intended your model to behave like this.
(No I didn't. ;P)
I can't seem to find the reason for the deadlock, though. The tool points me to the BlockCycle model but I didn't specify any invariant there. In fact, all the transition requirements are fulfilled so the transition (from Pos7 to Pos8) should definitely be taken.
Below you'll see how the systems evolves and finally gets stuck (error message pops up).
Labels:
green : property check (e.g. FB_Running means FB_Running == true )
dark blue: property updates/assignments
dark red: locations (e.g. Pos7 or Pos8)
The BlockCycle model with the respective transition in question:
My Question: Why does the system deadlock even though there are still transaction which could be taken.
Edit1: When I remove the invariant property of Sensor7's location Active (called BlockAtPos[7]) the deadlock is resolved. So I guess, since there is no synchronization between Sensor7 and BlockCycle for the last transition before it deadlocks, that would cause to a contradiction (BlockAtPos[7] becoming false while the sensor has not yet the chance to take the InActive transition) and thus violating the invariant.
Note: You can find my UPPAAL code/file here: pcs.xml.