I'm using an event handler method with #KafkaListener based on default spring-kafka library, together with the embedded Datadog agent on the container running the application.
When Rest endpoints calls fails, it automatically inject the error messages on spans and I can see them on Datadog.
Although for Kafka errors, I can only see spans with "error: true".
I'm not using anything special other than that. Is there something needed in order to track event errors on Datadog?
Related
I am using axon distributed command bus autoconfiguration using spring cloud and eureka
I observed all the required beans are getting created./member-capabilities is also exchanging the command information with other services and it is I can see this mapping exists in request mappings as well. Now when the command is being sent from another service by internally making a post-call to /spring-command-bus-connector/command endpoint
I am getting 404.
I checked that SpringHttpCommandBusConnector's bean is loaded in application context
I tried adding a component scan as well
Also tried registering this mapping manually with dispatcher servlet still no luck
I'm using spring boot 2.1.7.RELEASE and spring-kafka 2.2.8.RELEASE.And I'm using #KafkaListener annotation to create a consumer and I'm using all default settings for the consumer.
Now, In my consumer, the processing logic includes a DB call and I'm sending the record to DLT if there is an error/exception during processing.
With this setup, If the DB is down for few mins because of some reason, I want to pause/stop my consumer from consuming more records otherwise it keeps on consuming the messages and will get the DB exception and eventually fill up my DLT which I don't want to do unless the DB is back (based on some health check).
Now I've few questions here.
Does spring-kafka provide an option to trigger infinite retry based on the exception type (in this case a DB exception but I want to add few more types of exception based on my consumer logic)
Does spring-kafka provide an option to trigger the message consumption based on a condition?
There is a ContainerStoppingErrorHandler but it will stop the container for all exceptions.
You would need to create a custom error handler that stops (or pauses) the container after a specific failure as well as some mechanism to restart (or resume) the container.
I'm using Axon version (3.3) which seamlessly supports Kafka with annotation in the SpringBoot Main class using
#SpringBootApplication(exclude = KafkaAutoConfiguration.class)
In our use case, the command side microservice need to pick message from kafka topic rather than we expose it as Rest api. It will store the event in event store and then move it to another kafka topic for query side microservice to consume.
Since KafkaAutoCOnfiguration is disabled, I cannot use spring-kafka configuration to write a consumer. How can I consume a normal message in Axon?
I tried writing a normal Kafka spring Consumer but since Kafka Auto COnfiguration is disabled, initial trigger for the command is not picked up from the Kafka topic
I think I can help you out with this.
The Axon Kafka Extension is solely meant for Events.
Thus, it is not intended to dispatch Commands or Queries from one node to another.
This is very intentionally, as Event messages have different routing needs apposed to Command and Query messages.
Axon views Kafka a fine fit as an Event Bus and as such this is supported through the framework.
It is however not ideal for Command messages (should be routed to a single handler, always) or Query messages (can be routed to a single handler, several handlers or have a subscription model).
Thus, I you'd want to "abuse" Kafka for different types of messages in conjunction with Axon, you will have to write your own component/service for it.
I would however stick to the messaging paradigm and separate these concerns.
For far increasing simplicity when routing messages between Axon applications, I'd highly recommend trying out Axon Server.
Additionally, here you can hear/see Allard Buijze point out the different routing needs per message type (thus the reason why Axon's Kafka Extension only deals with Event messages).
Is there any error logging option in APIGee?
I am enabling a proxy endpoint in APIGee for a customer. If there is any error in the flow, how can I log it to a persistent store?
Specifically, I am using javascript policies and parsing some of the returned by back-end service, format to a different format. If there is any parsing error, where and how can I log it?
I am able to catch the error with try catch block.
Can I send an email in the catch block to a specific email address?
Thanks,
Deepak
You can catch the error in your catch block and then set the appropriate variable to contain the detailed error message you might have ran into.You can use syslog/messagelogging policy to send all the request details along with any parsing exception that you might have. You need some kind of a logging server at your end to be sent the logs to or you could also use public log management services, such as loggly. Refer to this section for more details - http://apigee.com/docs/api-services/content/log-messages-using-messagelogging
Vineet I think you are looking to solve the following problems here:
During the coding process you want to debug and understand what is going on? Something that lets you log.debug equivalent.
You want to also trigger exception flows or external processes when things go wrong in your proxy flow.
For #1 you can assign variables and trace using apigee's debug view. Any flow variable assignment in Apigee policies is printed in the debug view, if the policy is executed. So that provides you with log.debug mechanism, whenever you trace.
For #2 you can take a variety of approaches based on rest of your systems and processes. The previous answer by #Mike Dunker is a good approach. I can suggest a few more alternates
As an alternate you can also use Apigee's analytics views to monitor errors, albeit after the fact.
You can set up external monitoring scripts on your backend or on the apigee endpoint based on some synthetic transaction that touches upon the entire logic of your flow. When the services return error you can measure metrics and/or raise alert from your monitoring system.
If you have an on-premise installation of Apigee, you may want to consume apigee log files on the servers and raise appropriate actions on errors.
You can raise a desired error message to a JMS queue when error happens. Refer to Apigee's JMS support here
Can ASP.NET's System.web.Management be used for logging events like user logins, password changes and access to certain resources? Should system.Web.Management be used for logging of errors and health monitoring instead?
I want to log events without re-inventing the whole thing. I know ELMAH is used for errors; can it be used for logging events too?
From the article ELMAH - Error Logging Modules And Handlers on the now defunct DoNetSlackers:
Error signaling is exposed via the ErrorSignal class, which provides a single overloaded method called Raise. Simply put, exceptions raised via the ErrorSignal class are not thrown, therefore they don't bubble up, but instead are only sent out to ELMAH, as well as to whomever subscribes to the Raise event of the ErrorSignal class.
The code snippet below shows how to obtain an instance of the ErrorSignal class, which is unique per application and can be retrieved simply with the static FromCurrentContext method, and then use it to signal an exception.
ErrorSignal.FromCurrentContext().Raise(new NotSupportedException());
The difference between signaling and throwing an exception is that in the first case no one will ever know about the error except ELMAH.
Further Reading:
How to get ELMAH to work with ASP.NET MVC [HandleError] attribute?