Am I missing something when using MassTransit and AmazonSQS in a large project? - .net-core

I'm using MassTransit in a project with AmazonSQS and since I updated the packages to the latest version 7.3 I'm getting this exception
---> Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceException: Rate exceeded
---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
Sometimes the exception is coming from SQS, the thing is when I was working with the version 6 I didn't have those exceptions.
This solution has three projects:
Two web applications (which produce the messages)
BackgroundService (which receive and process the messages)
I designed this system using CQRS pattern with several commands and for that reason it's creating 100 topics and I don't know if I need to consider some limits either from AWS or MassTransit
Someone can help me? Thanks

Related

Multiple SpringBootTests using EmbeddedKafka fails with "only one 'RetryTopicConfigurationSupport'"

We have an application that makes use of Spring Kafka's non blocking retries via the RetryableTopic annotation.
We are in the middle of upgrading spring-kafka from 2.8.4 to 2.9.0.
We have several SpringBootTests that makes use of EmbeddedKafka. Each of these tests are marked with DirtyContext and AutoConfigureMockMvc
After upgrading, the first test would run fine, but the later test would fail to start the application with
Constructor threw exception; nested exception is java.lang.IllegalStateException: Only one 'RetryTopicConfigurationSupport' is allowed
I understand that the RetryTopicConfigurationSupport tries to ensures only one instance of itself is ever instantiated. So in the use case of running multiple unit tests, which includes multiple different SpringBootTests, how do we avoid hitting this problem?
I have tried marking the context dirty, but of course that didn't solve the problem as RetryTopicConfigurationSupport is using a static variable to track whether it has been instantiated before or not.
I have tried NOT marking the ContextDirty, but the later tests would fail to start the application because the Port is already in use.
Appreciate any advise!
Looks like this problem has been addressed in spring-kafka 3.0 and backported to 2.9.3
https://github.com/spring-projects/spring-kafka/issues/2477

Could not create internal topics - Stream-thread exception

I am trying to execute a simple Wordcount stream application but I face the error "Could not create internal topics - Stream-thread exception"
I have seen a similar thread but that seems to be more of a network issue.
Here is no security enabled on the kafka broker.
Only one broker is configured and still this issue.
Can someone let me know how to fix this?
Clean your temporary kafka queues.
Run --list command on kafka to see all the queues starting with your names and ending with -changelog & -repartition and manually run delete on them.
This one worked for me.
Also, check your settings on delete.topic.enable for actual deletion happening. It was not the default setting until 1.0.0 - see https://issues.apache.org/jira/browse/KAFKA-5384
i have connected to kafka using kafka tool and delete them manually

Unity Hololens WorldAnchorTransferBatch ExportAsync fails most of the time

I'm trying to create a hologram that is shared among multiple Hololenses by following Microsoft's Holograms 240 tutorial at https://developer.microsoft.com/en-us/windows/holographic/holograms_240
The network connection is working fine, and I have all the correct permissions in my project settings. The issue is that aside from a rare occurence, the call to WorldAnchorTransferBatch.ExportAsync() will fail with a SerializationCompletionReason.UnknownError and print out "Failed to world lock serialization failed"
I haven't been able to find any information about this error in my searching. Does anyone know what the cause of this could be?

AMQP consumer loads CPU to 100%

I have Symfony2 application with RabbitMQBundle installed. I've setup consumers and producers as it's described in the bundle documentations and everything works correct. But my consumers started with ./app/console rabbitmq:consumer take all available CPU time. Basically consumer does nothing but waiting for a message and output it. If I start demo consumer from php-amqplib CPU consumption is almost zero. I tried different virsions of Symfony (2.6 and 2.3) but this does not affect CPU load. My server configuration:
Debian 7
PHP 5.6.4 (also tried 5.4)
no database used
RabbitMq 3.4.2
Is there any way to reduce CPU consumption? Thanks
Just ran into a very similar issue and after some debugging realized that I was using an old way of instantiating the connection to rabbitmq.
The new signature of the method is described here: https://github.com/videlalvaro/php-amqplib/blob/master/PhpAmqpLib/Connection/AbstractConnection.php#L136
I was sending in something that looked more like
$this->connection = new Connection\AMQPConnection(
$server->host,
$server->port,
$server->user,
$server->password,
$server->vhost,
$server->insist,
$server->login_method,
$server->locale,
$server->connection_timeout,
$server->read_write_timeout,
$server->context,
$server->keepalive,
$server->heartbeat
);
As per a very old definition around somewhere in version 2. https://github.com/videlalvaro/php-amqplib/blob/v2.0.0/PhpAmqpLib/Connection/AMQPConnection.php#L31
So your plugin seems to use a new version of the library but not the new way to initiate a connection.

servlet initialization failure in websphere 6.0

I have many servlets in a web applicaton; for some stange reason, only and only one of them always fails in initialization with the following error trace:-
00000045 ServletWrappe E SRVE0100E: Did not realize init() exception thrown by servlet MyServletX: java.lang.NullPointerException
at com.ibm.ws.webcontainer.WebAppPmiListener.onServletStartInit(WebAppPmiListener.java:120)
at com.ibm.ws.webcontainer.webapp.FireOnServletStartInit.fireEvent(WebAppEventSource.java:237)
at com.ibm.ws.webcontainer.util.EventListeners.fireEvent(EventListeners.java:48)
at com.ibm.ws.webcontainer.webapp.WebAppEventSource.onServletStartInit(WebAppEventSource.java:105)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:261)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:444)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:2841)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:220)
at com.ibm.ws.webcontainer.VirtualHost.handleRequest(VirtualHost.java:204)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1681)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:77)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:421)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:367)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:94)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java:548)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java:601)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java:934)
at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java:1021)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1332)
I could not figure out if there is anything extra ordinary with this servlet. There is no init() method in this servlet and it extends HTTPServlet. Any idea what could be reason? I am using websphere server 6.0.x. How to get more debugging information in this case?
Well I don't know still cause of above error, but this is how it started working strangely:- i) Re-applied recommended fixes by IBM for my WAS version (especially there are IBM JDK upgrade related fix patches) ii) created a new profile of server iii) Install web application to new profile and it started working.
I don't think this is a product issue.
To debug this problem what i would suggest is to place a simple servlet (kind of Hello World) and deploy it to the server and see what happens.
initialization does not necessarily mean init() method alone.
If you have a static block in your servlet, if you have any variables that are initialized they would all be part of the initialization activity.
Look at the FFDC logs that were generated when this error occurred and that should provide you with clues.
As bkail mentioned, also ensure that yo have the latest fixpacks just to eliminate known problems with the product.
if the hello world servlet works, suggest you place hte servlet code here along with the SystemOut and System Err logs that correspond to this issue along with the relevant FFDC logs and i am sure most of us will be able to help you out with this
HTH
Manglu

Resources