while upgrading our spring kafka to 2.7.8, we are getting error in the setAckOnError(false) method as it has been deprecated now. Is there any way now to set the acknowledgement for the errors to false? Any other methods that can help me to set it as false for errors acknowledgement?
P.S: I am new to Kafka, any help appreciated!
That property was found to have a (very small) timing hole in that a record could be ack'd before the error handler handles it; if the app dies at that time, the record could be "lost".
It was replaced by a new feature in the error handlers ackAfterHandle; which is true by default - i.e. the record's offset is only committed if the error handler "handles" the error.
Records are now never ack'd if the error handler (such as the SeekToCurrentErrorHandler) throws an exception (after it repositions the partitions).
There is no extra configuration needed any more.
See Spring Kafka AckOnError for more details.
Related
I am working on Spring Kafka implementation and my use case is consume messages from Kafka topic as batch (using batch listener). when I consumer the list of messages, will iterate and call the REST endpoint for message enrichment. In case REST API fails for any runtime exception, I have implemented retry logic using spring retry. I want to stop the container, after the number of retries fails. So planning to use KafkaContainerStoppingErrorHandler to achieve this. Does the KafkaContainerStoppingErrorHandler commits the previous success messages - say if we receive 10 messages, and for message 1,2,3,4, enrichment call is success and for message 5 enrichment API call fails. so when we restart the container, will I get all 10 again or will I receive messages 5- 10?
or is there a way we can achieve above use case? I looked into all types of error handles of Spring kafka and need input on how to achieve above requirement.
You will get them all again.
You can use the DefaultErrorHandler (with a custom recoverer) and throw a BatchListenerFailedException to indicate which record in the batch failed.
The error handler will commit the offsets up to that record and call the recoverer with the failed record; in your custom recoverer you can stop the container (use the same logic as the container stopping error handler).
In versions before 2.8, this same functionality is provided by the RecoveringBatchErrorHandler.
I am using spring data 1.5.8 and manual ack mode of kafka. There is only a simple ErrorHandler interface, the handle method has parameter exception and consumerRecord, but how can I set the offset to next if exception threw when json can not be deserialized? I need omit the message or else the consumer will stuck there
spring data 2.0 introduce ConsumerAwareErrorHandler, the only way is update spring version?
Yes. You need to upgrade. The current version is 2 4.0 or 2.3.4.
I am looking for a way to capture when a call is completed on the server and no errors were thrown.
I understand that SimpleForwardingServerCallListener exists, however,onComplete is called when an exception is thrown.
My use case is for transaction management.
Yes, it currently only triggers onComplete(). There is a bug filed for that. If it's fixed, you can probably get onCancel() instead.
For now, you can wrap the ServerCall with SimpleForwardingServerCall in your ServerInterceptor, and override close(). If the RPC ends successfully, OK will be passed to close().
I've updated HereMaps JavaScript from 2.2.4 to 2.5.3 and the map has stopped working on Chrome (version 31.0.1650.57 m). Firefox and IE work as before.
On the console I see the message many times
Coroutine 'nokia.maps.map.render.p2d.Engine#_renderCo' aborted abnormally with exception base.js:158
TypeError: Cannot read property 'body' of undefined
Does anyone have a hint of what might be wrong?
Thanks.
It is very difficult to answer your specific problem without seeing your code, but I am able to produce a similar error which may help you track down your issue.
If you are getting multiple error messages, then it is likely that the JavaScript library itself is calling something periodically. From the text of the error it likely that the library is running a Coroutine, which is usually used for threading within the application.
The error I can produce is in a callback - if I add an overlay to the map, and deliberately try to access a non-existent property within my code I can get:
Coroutine 'nokia.maps.map.render.p2d.Engine#_renderCo' aborted abnormally with exception base.js:158
TypeError: Cannot read property 'bar' of undefined
This was because I added a typo as follows:
I changed the reference from foo.something.bar in the callback to foo.does_not_exist.bar. So I guess somewhere in your callbacks you are referencing foo.something.body in your callback - it could be in a listener for example.
Another possibility, is that you are switching directly to Display.SATELLITE when you are creating the map - this is no longer allowed:
Taken from the News feed on developer.here.com:
Breaking changes with 2.5.3
The new rendering engine requires the map to be initialized
asynchronously. If you want to change the center, zoom level or the
base map type the API now requires the application to wait for the
"displayready" event to be fired. This event is only fired once.
Please refer to the developer's guide (section "Beyond the Basic Map Application
") for details.
I have many servlets in a web applicaton; for some stange reason, only and only one of them always fails in initialization with the following error trace:-
00000045 ServletWrappe E SRVE0100E: Did not realize init() exception thrown by servlet MyServletX: java.lang.NullPointerException
at com.ibm.ws.webcontainer.WebAppPmiListener.onServletStartInit(WebAppPmiListener.java:120)
at com.ibm.ws.webcontainer.webapp.FireOnServletStartInit.fireEvent(WebAppEventSource.java:237)
at com.ibm.ws.webcontainer.util.EventListeners.fireEvent(EventListeners.java:48)
at com.ibm.ws.webcontainer.webapp.WebAppEventSource.onServletStartInit(WebAppEventSource.java:105)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:261)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:444)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:2841)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:220)
at com.ibm.ws.webcontainer.VirtualHost.handleRequest(VirtualHost.java:204)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1681)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:77)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:421)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:367)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:94)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java:548)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java:601)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java:934)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java:1021)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1332)
I could not figure out if there is anything extra ordinary with this servlet. There is no init() method in this servlet and it extends HTTPServlet. Any idea what could be reason? I am using websphere server 6.0.x. How to get more debugging information in this case?
Well I don't know still cause of above error, but this is how it started working strangely:- i) Re-applied recommended fixes by IBM for my WAS version (especially there are IBM JDK upgrade related fix patches) ii) created a new profile of server iii) Install web application to new profile and it started working.
I don't think this is a product issue.
To debug this problem what i would suggest is to place a simple servlet (kind of Hello World) and deploy it to the server and see what happens.
initialization does not necessarily mean init() method alone.
If you have a static block in your servlet, if you have any variables that are initialized they would all be part of the initialization activity.
Look at the FFDC logs that were generated when this error occurred and that should provide you with clues.
As bkail mentioned, also ensure that yo have the latest fixpacks just to eliminate known problems with the product.
if the hello world servlet works, suggest you place hte servlet code here along with the SystemOut and System Err logs that correspond to this issue along with the relevant FFDC logs and i am sure most of us will be able to help you out with this
HTH
Manglu