I am trying to configure a Kafka client (it happens to be a DataPower appliance running V10) to my IBM Event Streams SaaS instance in IBM Cloud. But the Kafka client keeps throwing an error:
Broker transport failure Initialization failed
Where can I go, in IBM Event Streams SaaS, to determine if I can see my Kafka client trying to make connection ? And, ideally, see some useful error messages like "pwd is wrong" !
Related
I have a doubt regarding spring kafka broker failover mechanism. I was checking by bringing the brokers down, voluntarily and I've been getting these "Connection to node -1 could not be established. Brokers may not be available" warnings continuously as soon as the brokers went down. I understand that it's because of the broker unavailability. I want a support document to know that if it happens automatically by the API itself?
This is all handled by the underlying kafka-clients code; it is outside of Spring's responsibility; Spring is not informed of the situation.
The client will reconnect when the brokers come back up.
On my dev instance of IBM MQ Server have been created two channels "DEV.APP.SVRCONN" and "CLIENT.DEF.CLNCONN". First one is "Server-connection" type and second "Client-Connection". I use .net core and assembly "amqmdnetstd.dll" to get the messages from the queue. Via server-connection channel I can do that. But use via client-connection channel, I can't event get connected just receive an error like below:
MQException caught: 2059 - MQRC_Q_MGR_NOT_AVAILABLE
at IBM.WMQ.MQQueueManager.Connect(String queueManagerName)
Please, help me to identify the reason of error. Thanks.
I have a Single Page Azure Web App that uses Signalr(Microsoft.AspNetCore.SignalR" Version="1.0.0-alpha1-final) to broadcast events(login, logout, department creation etc) to connected clients.
I also scale my application to several instances at peak times and i use Redis Cache Backplane (Microsoft.AspNetCore.SignalR.Redis" Version="1.0.0-alpha2-final) to distribute event broadcast messages to all connected clients
i use angular front end (#aspnet/signalr-client": "^1.0.0-alpha2-final)
on azure, i enabled diagnostic log to log information and error messages.
the above works fine but when i scale up the application, it is difficult to trace information or error messages as i have to look through up to 10 instance application logs to find information or error;
my question: How do i ensure redis cache logs error messages or information on all available instances rather than on instances where client is connected; how do i know if a client has missed out on event broadcast message? how do i ensure signalr sever/hub logs all messages on all application log instances?
thank you in advance
The best way to do this might be to use the redis-cli, run the MONITOR command and then pipe that to a file (or somewhere that can store the logs).
I have a task to implement a message broker at choice in a distributed system. Is Firebase Cloud Messaging considered one?
No. At the very least not identical in a sense, however both are related to interchanging of messages.
Firebase Cloud Messaging (FCM) is a messaging service commonly (if not always) used for Push Notifications:
A push notification is a message that is "pushed" from backend server or application to user interface, e.g. (But not limited to) mobile applications and desktop applications. It is more user experience specific which is different from Push technology, which pushes the requests between components such as server to server communication. A common scenario of push notification is the client application pops up a message in front of application's user information, along with the alert sounds. The notification could also coupled with images and hyper text link in some cases. Via interacting with the push notification it usually brings up the client applications to the front.
The service could be described as a middleware that handles the sending/delivery of the message between the App Server (usually the sender) and the client (the receiver). But for them to communicate accordingly, both the Sender and Receiver must be configured to receive the message itself (i.e. they are the ones that have to adjust to the message).
While a Message Broker is described as:
In computer programming, a message broker is an intermediary program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally-defined messages. Message brokers are a building block of Message oriented middleware.
From the description itself, the message broker could also be considered as a middleware, but it's task is more on transforming/translating/adjusting the message so that it is would be smoothly received by the receiver.
There is also an available list of Message Broker softwares from the Wikipedia page, containing:
Apache ActiveMQ
Apache Kafka
Apache Qpid
Celery
Cloverleaf (E-Novation Lifeline)
Comverse Message Broker (Comverse Technology)
Enduro/X Transactional Message Queue (TMQ)
Financial Fusion Message Broker (Sybase - acquired by SAP in 2010)
JBoss A-MQ (aka. Fuse Message Broker - enterprise ActiveMQ - acquired by RedHat in 2012)
Gearman
HornetQ (Red Hat) (donated to Apache ActiveMQ community)
IBM Integration Bus
IBM Message Queues
JBoss Messaging (JBoss - moved to HornetQ and now it's in bug-fix mode)
JORAM
Azure Service Bus (Microsoft)
BizTalk Server (Microsoft)
NATS (MIT Open Source License, written in Go)
Open Message Queue
Oracle Message Broker (Oracle Corporation)
QDB (Apache License 2.0, supports message replay by timestamp)
RabbitMQ (Mozilla Public License, written in Erlang)
Redis An open source, in-memory data structure store, used as a database, cache and message broker.
SAP PI (SAP AG)
Solace Systems Message Router
Spread Toolkit
Tarantool, a NoSQL database, with a set of stored procedures for message queues
WSO2 Message Broker
I'm trying to make a Flex-based desktop application consume messages from an ActiveMQ topic with a durable subscription, using the JMS bridge of BlazeDS. The basic scenario is as follows:
Messages are produced by other producers in the topic to which the Flex client is subscribed.
The Flex client may go offline from time to time, but it must receive all the messages it has missed while being offline when it connects to BlazeDS again. (Of course the Flex client connects with the same client ID every time).
It can not be guaranteed that the Flex client is shut down gracefully.
Everything works fine if I explicitly disconnect my consumer on the Flex side by calling disconnect() - I do it in the exit handler of the application. However, due to #3 above, it is not guaranteed that disconnect() is called all the time. When the Flex client shuts down without calling disconnect(), it seems that the subscription of the "proxy JMS client" that BlazeDS creates and associates to the Flex client stays active towards ActiveMQ, so ActiveMQ still thinks that the client is logged in. When the Flex app starts up the next time, it is unable to log in to BlazeDS because ActiveMQ refuses its subscription, claiming that the client ID is already taken. Why is it so and what can I do here to ensure that BlazeDS makes the "proxy JMS client" offline in ActiveMQ when its real Flex counterpart terminates unexpectedly?
More detailed information: some debugging revealed that:
BlazeDS becomes aware of the termination of the Flex client because it prints a few exceptions to the console when in debug mode. The messages are as follows:
[BlazeDS]23:18:13.688 [WARN] Endpoint with id 'my-streaming-amf' is closing the streaming connection to FlexClient with id '71E6466F-D91F-201C-F60A-A6CB52F95D9F' because endpoint encountered a socket write error, possibly due to an unresponsive FlexClient.
ClientAbortException: java.net.SocketException: Broken pipe
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:319)
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
at org.apache.catalina.connector.Response.flushBuffer(Response.java:542)
at org.apache.catalina.connector.ResponseFacade.flushBuffer(ResponseFacade.java:279)
at flex.messaging.endpoints.BaseStreamingHTTPEndpoint.handleFlexClientStreamingOpenRequest(BaseStreamingHTTPEndpoint.java:818)
at flex.messaging.endpoints.BaseStreamingHTTPEndpoint.serviceStreamingRequest(BaseStreamingHTTPEndpoint.java:1055)
at flex.messaging.endpoints.BaseStreamingHTTPEndpoint.service(BaseStreamingHTTPEndpoint.java:460)
at flex.messaging.MessageBrokerServlet.service(MessageBrokerServlet.java:353)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:737)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:434)
at org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:299)
at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:963)
at org.apache.coyote.Response.action(Response.java:183)
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:314)
... 20 more
[BlazeDS]23:18:13.689 [DEBUG] Streaming thread 'http-8400-1' for endpoint with id 'my-streaming-amf' is releasing connection and returning to the request handler pool.
[BlazeDS]23:18:13.689 [INFO] Number of streaming clients for FlexSession with id '5BC5E8D604A361BCA673B05AC624CCC1' is 0.
[BlazeDS]23:18:13.689 [DEBUG] Number of streaming clients for endpoint with id 'my-streaming-amf' is 0.
At this stage, the subscriptions are still shown on the ActiveMQ web admin interface as being active.
Killing BlazeDS (more precisely, the Tomcat server that hosts it) with kill -9 from the console makes ActiveMQ realize immediately that the "proxy JMS client" is gone and it becomes offline on the ActiveMQ web admin interface. This made me conclude that BlazeDS is keeping the proxy JMS client alive explicitly since kill -9 gives no chance to BlazeDS to unsubscribe the client but it still becomes offline in ActiveMQ.
So, the question once again: What can I do here to ensure that BlazeDS makes the "proxy JMS client" offline in ActiveMQ when its real Flex counterpart terminates unexpectedly? Is this a bug in BlazeDS or am I just missing some hidden configuration setting that would make it work?
Version information: BlazeDS 4.0, ActiveMQ 5.5.0, both freshly downloaded today. I'm using the Tomcat server in the BlazeDS turnkey but ActiveMQ is installed separately because the BlazeDS turnkey ships with ActiveMQ 4.1.1 only. By the way, that version of ActiveMQ has the same issue.
The problem is that there is no way for BlazeDS to detect that your Flex client was shutdown, you will have to implement your own mechanism - my suggestion is to use a heart beat implemented with messaging. If no message is received from the client after a time interval you can assume that the Flex client is gone and do the disconnect (or you can use the session timeout mechanism on the server, and do the disconnect on session expire).
What you have seen (the exception caught when the streaming channel is closed) is not enough to say 100% sure that the Flex client is gone. The streaming is implemented using an HTTP connection kept open forever (used to send server messages) and periodic HTTP post calls (initiated by the client to send messages). In some networks the firewall can decide to kill the HTTP connection after a couple of seconds and you will receive the same error like the one you posted. However, it does not mean that the Flex client is killed - the Flex client can use a fallback strategy and switch to short/long polling in this case. Actually it would be a bug if BlazeDS will automatically do the JMS disconnect in this case.