To allow the ActiveMQ client and broker to tunnel over HTTP, I'm following the reference documentation http-and-https-transports-reference and I add in activemq.xml file:
<transportConnector name="http" uri="http://0.0.0.0:8081"/>
Starting activemq broker, and when consuming message, the error log is as follows:
Caused by: java.io.IOException: Failed to perform GET on: http://10.128.64.2:8081 Reason: Read timed out
at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:34) ~[activemq-client-5.14.5.jar!/:5.14.5]
... 2 more
Caused by: java.net.SocketTimeoutException: Read timed out
Any help
Thanks
In the meantime I found the cause for the problem. A default socket timeout of 30 secods for the HTTP client was introduced with the following ticket: https://issues.apache.org/jira/browse/AMQ-6397
This timeout somewhow seems to conflict with the keep alive requests that stays open for arround 30 seconds. I increased the default timeout to 90 seconds and this seems to resolve the problem (see attached version of HttpClientTransport.java).
source: https://jira.apache.org/jira/browse/AMQ-6949
Related
Problem:
I testing my asp.net webapi application in my server (use IIS) and Concurrency number is set to 2000,loop count is forever,and alter several second i get Connection timed out: connect error
what i have tried:
set http connect timeout and response timeout as 200000ms in jmeter gui.
set requestQueueLimit to 65535 and min process to 15 in IIS manager.
set minWorkerThread and minIoThread to 200 and timeout to 20 miniutes in web.config file and restart my application in IIS
None of the above worked,and i found the server's cpu usage has been low ,here is the screenshot when using jmeter to test:
cpu usage
jmeter screen shot
here is the error log:
org.apache.http.conn.HttpHostConnectException: Connect to XXX.XXX.com:80 [XXX.XXXX.com/XXXX] failed: Connection timed out: connect
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$JMeterDefaultHttpClientConnectionOperator.connect(HTTPHC4Impl.java:408)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:939)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:650)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:66)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1301)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1290)
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:651)
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:570)
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
Check Maximum Concurrent Connections and other limits in your web site configuration: Advanced Settings->Limits->Maximum Concurrent Connections
It may be not connected with IIS at all and the timeout can happen at your website level due to incorrect database configuration or inefficient algorithms used. Consider re-running your test with profiler tool telemetry like YourKit or dotTrace - it will give you full information regarding what's going on under the hood
Don't run load tests using JMeter GUI, it's only for tests development and debugging, when it comes to execution you should be running your JMeter tests in command-line non-GUI mode
Remove all the listeners, they don't add any value and just consume valuable resources
I have a grpc server streaming RPC that communicates with the client. The client throws an error when it does not receive communication from the server. The error is:
Error: 13 INTERNAL: Received RST_STREAM with code 2
at Object.callErrorFromStatus (/app/node_modules/#grpc/grpc-js/build/src/call.js:30:26)
at Object.onReceiveStatus (/app/node_modules/#grpc/grpc-js/build/src/client.js:328:49)
at Object.onReceiveStatus (/app/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:304:181)
at Http2CallStream.outputStatus (/app/node_modules/#grpc/grpc-js/build/src/call-stream.js:116:74)
at Http2CallStream.maybeOutputStatus (/app/node_modules/#grpc/grpc-js/build/src/call-stream.js:155:22)
at Http2CallStream.endCall (/app/node_modules/#grpc/grpc-js/build/src/call-stream.js:141:18)
at ClientHttp2Stream.stream.on (/app/node_modules/#grpc/grpc-js/build/src/call-stream.js:410:22)
at ClientHttp2Stream.emit (events.js:198:13)
at emitCloseNT (internal/streams/destroy.js:68:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
Emitted 'error' event at:
at Object.onReceiveStatus (/app/node_modules/#grpc/grpc-js/build/src/client.js:328:28)
at Object.onReceiveStatus (/app/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:304:181)
[... lines matching original stack trace ...]
at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
I have tried searching for solutions and the only "solution" that kept the connection open was to manually ping the client every 10 seconds using setInterval. I read that grpc is supposed to keep the connections open for you and even reconnect if it is lost in this article.
As mentioned above, KeepAlive provides a valuable benefit: periodically checking the health of the connection by sending an HTTP/2 PING to determine whether the connection is still alive.
The way I set up the client is below
var grpc = require('#grpc/grpc-js');
require('dotenv').config({path:'/app/.env'});
var responderProto = grpc.loadPackageDefinition(packageDefinition).responder;
var client = new responderProto.ResponderService(process.env.GRPC_HOST_AND_PORT,
grpc.credentials.createInsecure(),
{
"grpc.http2.max_pings_without_data" : 0,
"grpc.keepalive_time_ms": 10000,
"grpc.keepalive_permit_without_calls" : 1
});
My understanding was that the "grpc.keepalive_time_ms" is supposed to ping the server every x ms
Am I doing something wrong or missing or misunderstanding something essential?
chan_sip locks causing registration issues with asterisk realtime mysql when displaying the following errors:
chan_sip.c:3821 retrans_pkt: Retransmission timeout reached on transmission for seqno 1 (Critical Response)
chan_sip.c:3821 retrans_pkt: Retransmission timeout reached on transmission for seqno 106
my registrations to my sip providers then timeout with the following errors:
chan_sip.c:13661 sip_reg_timeout: -- Registration for xxx timed out, trying again
this is not a connectivity issue as restarting asterisk will re-register to the sip providers immediately
the connections eventually re-establish, registrations start registering until the above happens again at a random interval. Server is a dedicated non natted server.
What could be the issue?
if you have debian or ubuntu, turn off res_timing_pthread.so and use res_timing_dahdi.so
That fix such issue.
We are running a CXF 2.7.11 application on WAS 8.5.5.2 server. Application has classloading parent last property also we disabled IBM JaxWS engine as instructed on CXF documentation.
Application is running fine a couple of days, after that we get below exceptions and TCP channel seems to be full.
From the stack trace that have ws classes I suspect CXF for this problem but that may be a result of another problem
The application is also a Spring MVC application that exposes REST resources..
[10.11.2014 05:00:20:887 EET] 00000049 TCPChannel W TCPC0004W: TCP Channel TCP_2 has exceeded the maximum number of open connections 20000.
[10.11.2014 05:02:16:343 EET] 0000023f SSLHandshakeE E SSLC0008E: Unable to initialize SSL connection. Unauthorized access was denied or security settings have expired. Exception is javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at com.ibm.jsse2.b.a(b.java:56)
at com.ibm.jsse2.nc.a(nc.java:90)
at com.ibm.jsse2.nc.unwrap(nc.java:292)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:26)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.readyInbound(SSLConnectionLink.java:535)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.ready(SSLConnectionLink.java:295)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1864)
So this is a bit tricky. You can simply increase the number of connections which can be done from the console:
Servers > WebSphere Application Servers > SERVER_NAME > web container > web container transport chains > TCP CHANNEL
The reason I said this is tricky because there could be a larger underlying issue, for example, a connection leak. To get to the point where you are using up 20K connections is quite a lot, however, I don't know how much load you're expecting on this server. If this is simply a test environment then you need to start looking into a possible connection leak.
Unable to initialize SSL connection. Unauthorized access was denied or security settings have expired. Exception is javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
This portion of the error message means that plain-text non-SSL connections are being made to an SSL port. You might also want to take a look at that and see who's making these calls because it's an overhead.
Using 20000 connections is extremely high. You probably have bugs in your client code which is leaking connections. If you are using CXF in client you may take a look at this https://issues.apache.org/jira/browse/CXF-5144.
Increasing connections number will not solve your issue, it will just delay it.
I'm trying to send a message and we sometime get this error :
Service not available, closing transmission channel. The server response was: 4.4.2 Timeout while waiting for command.
Anyone know what to do with this? Because it only happens "sometime" and apperently, for no specific reason.
I saw many article saying :
442 The server started to deliver the message but then the connection was broke (Source : http://www.sorkincomputer.net/SMTP%20errors.htm)
This is typically a server side (the SMTP server you're delivering to) error or a network connectivity error. There isn't anything you can do about it via your code, you would need to get the related IT staff involved to figure out why your connection is getting closed or interrupted.