We are using Apache Camel as an orchestration engine. Typically, the following scenario:
client sends HTTP request <-> CAMEL code <-> external server(s)
The ball starts to roll when our client sends a HTTP request to our CAMEL code.
The Camel code will trigger external servers via REST HTTP calls.
Eventually, the Camel code will send a reply back to the client.
The last action before sending the response back to the client, the Camel code sends a HTTP GET towards an external server. So a TCP connection is setup first, then the data sent. After some time (this might take up 5 to 10 seconds), the external server replies with a 200 OK.
Problem: Camel does not send a TCP FIN to the external server after receiving the 200 OK. As a result, the TCP connection remains open ... (the external server then closes the TCP connection itself after a timeout of 200 seconds, but this means a TCP resource lost during 200 seconds).
So, at TCP level, it goes like this:
Camel <----------> external server
TCP SYN -->
<-- TCP SYN,ACK
TCP ACK -->
HTTP GET -->
<-- 200 OK
TCP ACK -->
<200 seconds later>
<-- TCP FIN,ACK
TCP ACK -->
Any idea how I can have Camel close the TCP connection after it has received the 200 OK ?
Note: I tried adding the "Connection: close" header, but Camel did not add the header ?! It seemed to ignore it ...
This was the code to add the header:
exchange.getOut().setHeader("Connection","Close");
I am using Camel 2.9.1 in a Spring framework with Eclipse IDE.
Unfortunately, I did not see another solution than create a custom HttpHeaderFilterStrategy class which does not filter out the Connection header.
Then before sending out my request to the external server, I am setting the header "Connection: close". As soon as this request gets replied, the Camel code then sends a TCP FIN, ACK in order to close the TCP connection.
More details:
1) create a custom HttpHeaderFilterStrategy class, eg: CustomHttpHeaderFilterStrategy
2) adapt the applicationContext.xml so it points to that class, eg:
<bean id="http" class="org.apache.camel.component.http.HttpComponent">
<property name="camelContext" ref="camel"/>
<property name="headerFilterStrategy" ref="myHeaderFilterStrategy"/>
</bean>
<bean id="myHeaderFilterStrategy" class="com.alu.iptc.com.CustomHttpHeaderFilterStrategy">
</bean>
3) adapt your code, so that the Connection: close header is set, eg:
exchange.getOut().setHeader("Connection","close");
HTTP1.1 connections are to be considered to be kept alive after the first message for a while to allow multiple files to be delivered in one TCP session for performance reasons. Normlly, a http server might cut connections after a few seconds to save threads while allow multiple files to be downloaded. The Camel http component will probably behave the same way.
http://en.wikipedia.org/wiki/HTTP_persistent_connection
The official HTTP client which Camel relies on can be configured to use or not use persistent connections, but default is true:
http://docs.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html
Although I have not tried it, it should be possible to set a system property to configure this
http.keepAlive=<boolean>
You should be able to set it on the camel context if you want
<camelContext>
<properties>
<property key="http.keepAlive" value="false"/>
</properties>
</camelContext>
Note that I have not tried it. If you make it work, it would be nice to hear the results!
Related
Below is the configured thread group
Thread Group
|
|--> HTTP SAMPLE
I am opening 5 threads in 5 seconds with loop count is 5. It means in each thread 5 HHTP request will be sent, so total 25 request.
Thread 1 : TCP CONNECTION OPENED --> HTTP REQUEST SENT ---> HTTP RESPONSE RECEIVED.
Question: After receiving response whether TCP connection will be closed and again TCP connection will be opened and then HTTP request will be sent?
Or once response is received in same TCP connection next HTTP request will be sent?
It depends on what do you set in the HTTP Request sampler, if you tick Use KeepAlive box - then JMeter will send Connection header with the value of keep-alive
it means that the underlying TCP connection will remain open and the next HTTP Request sampler(s) will be reusing this connection.
If you untick the box JMeter will send Connection header with the value of close and next HTTP Request sampler will re-establish the connection.
You need to check what's going on in reality using your browser developer tools or an external sniffer tool like Wireshark and configure JMeter to behave exactly like the real browser does
While making HTTP call to server using Jmeter we see SYN flood.
This is causing most probably since Jmeter doesn't sends back the ack as part of 3 way handshake.
Is there any way we can force Jmeter to send the ack back to Server for TCP Connection ?
Given connection succeeded JMeter should normally send ACK, there is no need to "force" it by any means (at least this is true for JMeter 5.2 and HttpClient4 implementation):
I would recommend double-checking incoming and outgoing packets using a lower level sniffer tool like Wireshark
You can also get extra information regarding what's going on under the hood by adding the next line to log4j2.xml file (lives in "bin" folder of your JMeter installation) to enable debug logging for the HTTP protocol
<Logger name="org.apache.http" level="debug" />
JMeter restart will be required to pick up the change.
I understand the rule that, if client and server both support persistent connection, they can use it through a Connection:keep-alive header in the first request. After that, both client and server will still keep the underlying TCP connection open when they are done with the first request/response, and then use the same connection in the following requests/responses.
What I'm not clear about is the programming model. Consider the following client code in go:
resp, _ := client.Get("http://www.stackoverflow.com")
// do some other things
resp, _ = client.Get("http://www.stackoverflow.com/questions")
As far as I know, keep-alive is the default strategy in HTTP/1.1.
Q1: Do these two requests use the same TCP connection?
On the server side:
When one request comes, the Go HTTP framework dispatches it to a handler, then due to keep-alive, the framework should prepare for the next request on the same TCP connection. However I don't see any block-mode read code in a handler. So,
Q2: Does the Go HTTP framework use some kind of non-block mode to deal with keep-alive?
I mean, a handler will not block for reading, but just return when done with a request, then the framework will poll each non-block TCP connection, if one of them has data arrive, it dispatches it to an associated handler, and so on. Only when the request has a header of Connection:Close, the framework will close the TCP connection when the handler returns.
Q1: Do these two requests use the same TCP connection?
Yes, if the connection wasn't closed by the server, the http.Transport can re-use the connection in the default configuration.
Q2: Does the go http framework use some kind of non-block mode to deal with keep-alive?
The go HTTP Server handles keepalive connections be default. There's no such thing as a HTTP "non-block mode". It handles requests and responses on a single connection in a serial manner as described by the HTTP specification.
I need to operate with a SOAP service from Erlang. SOAP implementation is not a subject, I have a problem with HTTP requests at a client side.
I use IBrowse as a HTTP client. This SOAP service uses a specific authorization mechanism, which relates an opened session to a client connection (socket). So, the client should use only one persistent connection to server (socket), and if it try to send a request via another socket (e.g., connection from pool) - authorization will fail.
I use IBrowse in this way:
Spawn connection process to server (ibrowse:spawn_worker_process/1)
Send request to server via spawned process with {max_sessions, 1} and {max_pipeline_size, 0}.
If I understand the docs right, this should use one socket for server connection with disabled pipelining, also, I use Connection: Keep-Alive header and HTTP version explicitly set to 1.0. But my connection is always closed after the response is received.
How can I use IBrowse (or another http-client) the way I described above?
I think you could that with hackney by reusing a connection.
Also gun is quite nice http client, easy to use, keeping connection, but with little less connection control.
I'm using RESTEasy asynchronous (Comet) IO support on Tomcat 6 via the NIO Connector. Currently, TCP connections are getting dropped by the server after each response is sent back to the client.
All documentation I've read on HTTP Connector configuration for Tomcat suggests that it should keep connections alive by default, so I'm puzzled as to what the problem is.
Here's my connector config:
<Connector connectionTimeout="20000" port="6080"
emptySessionPath="true" enableLookups="false"
protocol="org.apache.coyote.http11.Http11NioProtocol"
acceptorThreadCount="4" pollerThreadCount="12"/>
Thanks for any suggestions!
It turns out the root of the problem is elsewhere (still investigating and will post a separate question directly on that to avoid confusion!).
Tomcat is releasing the connections after a period of a few seconds rather than immediately on responding to the HTTP request. The client in this case is at fault for creating new TCP connections for each request rather than re-using connections already established.