We have an application that makes use of Spring Kafka's non blocking retries via the RetryableTopic annotation.
We are in the middle of upgrading spring-kafka from 2.8.4 to 2.9.0.
We have several SpringBootTests that makes use of EmbeddedKafka. Each of these tests are marked with DirtyContext and AutoConfigureMockMvc
After upgrading, the first test would run fine, but the later test would fail to start the application with
Constructor threw exception; nested exception is java.lang.IllegalStateException: Only one 'RetryTopicConfigurationSupport' is allowed
I understand that the RetryTopicConfigurationSupport tries to ensures only one instance of itself is ever instantiated. So in the use case of running multiple unit tests, which includes multiple different SpringBootTests, how do we avoid hitting this problem?
I have tried marking the context dirty, but of course that didn't solve the problem as RetryTopicConfigurationSupport is using a static variable to track whether it has been instantiated before or not.
I have tried NOT marking the ContextDirty, but the later tests would fail to start the application because the Port is already in use.
Appreciate any advise!
Looks like this problem has been addressed in spring-kafka 3.0 and backported to 2.9.3
https://github.com/spring-projects/spring-kafka/issues/2477
Related
I'm using MassTransit in a project with AmazonSQS and since I updated the packages to the latest version 7.3 I'm getting this exception
---> Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceException: Rate exceeded
---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
Sometimes the exception is coming from SQS, the thing is when I was working with the version 6 I didn't have those exceptions.
This solution has three projects:
Two web applications (which produce the messages)
BackgroundService (which receive and process the messages)
I designed this system using CQRS pattern with several commands and for that reason it's creating 100 topics and I don't know if I need to consider some limits either from AWS or MassTransit
Someone can help me? Thanks
In Apache Airflow (2.x), each Operator Instance has a state as defined here (airflow source repo).
I have two use cases that don't seem to clearly fall into the pre-defined states:
Warn, but don't fail - This seems like it should be a very standard use case and I am surprised to not see it in the out-of-the-box airflow source code. Basically, I'd like to color-code a node with something eye-catching - say orange - corresponding to a non-fatal warning, but continue execution as normal otherwise. Obviously you can print warnings to the log, but finding them takes more work than just looking at the colorful circles on the DAGs page.
"Sensor N/A" or "Data not ready" - This would be a status that gets assigned when a sensor notices that data in the source system is not yet ready, and that downstream operators can be skipped until the next execution of the DAG, but that nothing in the data pipeline is broken. Basically an expected end-of-branch.
Is there a good way of achieving either of these use cases with the out-of-the-box Airflow node states? If not, is there a way to defining custom operator states? Since I am running airflow on a managed service (MWAA), I don't think changing the source code of our deployment is an option.
Thanks,
The task states are tightly integrated with Airflow. There's no way to configure which logging levels lead to which state. I'd say the easiest way is to grep log files for "WARNING" or set up a log aggregation service e.g. Elasticsearch to make log files searchable.
For #2, sensors have no knowledge about why a sensor timed out. After timeout or execution_timeout is reached, they simply raise an Exception. You can deal with exceptions using trigger_rules, but these still don't take the reason for an exception into account.
If you want more control over this, I would implement your own Sensor which takes an argument e.g. data_not_ready_timeout (which is smaller than timeout and execution_timeout). In the poke() method, check if data_not_ready_timeout has been reached, and raise an AirflowSkipException if so. This will skip downstream tasks. Once timeout or execution_timeout are reached, the task is failed. Look at BaseSensorOperator.execute() for some inspiration to get the initial starting date of a sensor.
I've written a .NET Core Rest API which does migrate/ update the database (using Entity Framework Core) in Startup.cs. Currently, only one instance is running in the production environment. It seems to be recommended to run 2 instances in production.
What happens while executing the cf push command? Are both instances stopped automatically or do I need to execute cf stop?
In addition, how do I prevent both instances from updating the database?
I've read about the CF_INSTANCE_INDEX environment variable. Is it OK to only start the database migration when CF_INSTANCE_INDEX is 0? Or does CloudFoundry provide the next mechanism: start the first instance and when this one is up-and-running, the second instance will be started?
What happens while executing the cf push command? Are both instances stopped automatically or do I need to execute cf stop?
Yes, your app will stop. The new code will stage (i.e. buildpacks run) and produce a droplet. Then the system will bring up all the requested instances using the new droplet.
In addition, how do I prevent both instances from updating the database? I've read about the CF_INSTANCE_INDEX environment variable. Is it OK to only start the database migration when CF_INSTANCE_INDEX is 0?
You can certainly do it that way. The instance number is guaranteed to be unique and the zeroth instance will always exist, so if you limit to the zeroth instance then it's guaranteed to only run once.
Another option is to run your migration as a task (i.e. cf run-task). This runs in its own container, so it would only run once regardless of the number of instances you have. This SO post has some tips about running a migration as a task.
Or does CloudFoundry provide the next mechanism: start the first instance and when this one is up-and-running, the second instance will be started?
It does, it's the --strategy=rolling flag for cf push.
See https://docs.cloudfoundry.org/devguide/deploy-apps/rolling-deploy.html
I'm not sure that this feature would work for ensuring your migration runs only once. According to the docs (See "How it works" section at the link above), your new and old containers could overlap for a short period of time. If that's the case, running the migration could potentially break the old instances. It'll be a short period of time, just until they get replaced with new instances, but maybe something to consider.
I want to use mbeans on startup of j2ee application to check if all the MDBs are running and jms specification has been activated.
Any pointers will be very helpful
The only way I know of to do this would be to use the ServerEndpointControl MBean. This is a Liberty specific MBean for controlling the input sources for work into the runtime. This can also be used to get status on http listeners.
The best place to find the Javadoc for the MBean is here. To find out if an MBean is running you call the isPaused method providing the MDB name which is defined as:
ApplicationName#ModuleName#BeanName
if the MDB is running it'll return false.
I have many servlets in a web applicaton; for some stange reason, only and only one of them always fails in initialization with the following error trace:-
00000045 ServletWrappe E SRVE0100E: Did not realize init() exception thrown by servlet MyServletX: java.lang.NullPointerException
at com.ibm.ws.webcontainer.WebAppPmiListener.onServletStartInit(WebAppPmiListener.java:120)
at com.ibm.ws.webcontainer.webapp.FireOnServletStartInit.fireEvent(WebAppEventSource.java:237)
at com.ibm.ws.webcontainer.util.EventListeners.fireEvent(EventListeners.java:48)
at com.ibm.ws.webcontainer.webapp.WebAppEventSource.onServletStartInit(WebAppEventSource.java:105)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:261)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:444)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:2841)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:220)
at com.ibm.ws.webcontainer.VirtualHost.handleRequest(VirtualHost.java:204)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1681)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:77)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:421)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:367)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:94)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java:548)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java:601)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java:934)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java:1021)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1332)
I could not figure out if there is anything extra ordinary with this servlet. There is no init() method in this servlet and it extends HTTPServlet. Any idea what could be reason? I am using websphere server 6.0.x. How to get more debugging information in this case?
Well I don't know still cause of above error, but this is how it started working strangely:- i) Re-applied recommended fixes by IBM for my WAS version (especially there are IBM JDK upgrade related fix patches) ii) created a new profile of server iii) Install web application to new profile and it started working.
I don't think this is a product issue.
To debug this problem what i would suggest is to place a simple servlet (kind of Hello World) and deploy it to the server and see what happens.
initialization does not necessarily mean init() method alone.
If you have a static block in your servlet, if you have any variables that are initialized they would all be part of the initialization activity.
Look at the FFDC logs that were generated when this error occurred and that should provide you with clues.
As bkail mentioned, also ensure that yo have the latest fixpacks just to eliminate known problems with the product.
if the hello world servlet works, suggest you place hte servlet code here along with the SystemOut and System Err logs that correspond to this issue along with the relevant FFDC logs and i am sure most of us will be able to help you out with this
HTH
Manglu