https server python based is closing connection after serving the response - http

I had created simple server in terminal
#!/usr/bin/env python3
import sys, os, socket, ssl
import requests
import string
import time
from socketserver import ThreadingMixIn
from http.server import HTTPServer,BaseHTTPRequestHandler
from io import BytesIO
import json
import cgi
class ThreadingServer(ThreadingMixIn, HTTPServer):
pass
class RequestHandler(BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['Content-Length'])
body = self.rfile.read(content_length)
#self.send_header('Content-type', 'Application/json')
self.send_response(200)
self.end_headers()
response = BytesIO()
self.allow_reuse_address = True
self.wfile.write(b"""{"signingResponse": {"compactidentity": "..SdOwnT70ZZDAjgSmQVP-_0keB_pu4FjkBg5DZDyFf_V5k0EUAY0KCHr2g2a6wOSs-JhsehdYUnrYCfkYItzxLg;info=<http://52.23.250.93:8080/certs/shaken.crt>;alg=ES256;ppt=shaken\n", "TEST": "Nitish","identity": "eyJhbGciOiJFUzI1NiIsInBwdCI6InNoYWtlbiIsInR5cCI6InBhc3Nwb3J0IiwieDV1IjoiaHR0cDovLzUyLjIzLjI1MC45Mzo4MDgwL2NlcnRzL3NoYWtlbi5jcnQifQ.eyJhdHRlc3QiOiJBIiwiZGVzdCI6eyJ0biI6WyIxMjM1NTU1MTIxMiJdfSwiaWF0IjoxNDgzMjI4ODAwLCJvcmlnIjp7InRuIjoiMTIzNTU1NTEyMTIifSwib3JpZ2lkIjoiOGE4ZWM2MTgtYzZiOS0zMGFlLWI0MjctYWY0MTA0YjFjMDJjIn0.SdOwnT70ZZDAjgSmQVP-_0keB_pu4FjkBg5DZDyFf_V5k0EUAY0KCHr2g2a6wOSs-JhsehdYUnrYCfkYItzxLg;info=<http://52.23.250.93:8080/certs/shaken.crt>;alg=ES256;ppt=shaken\n", "requestid": "0"}} """)
httpd = ThreadingServer(('192.168.1.2', 8003), RequestHandler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='/home/nakumar/key.pem', certfile='/home/nakumar/certificate.pem', server_side=True)
httpd.serve_forever()
Using above code i am trying to simulated the server
now when server receives request from client , it send back the responses and closed the connection , as shown below
Request
> POST /stir/v1/signing HTTP/1.1
Host: 192.168.1.2:8003
Accept: application/json
Content-Type: application/json
Content-Length: 331
Reponse
upload completely sent off: 331 out of 331 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: BaseHTTP/0.6 Python/3.5.2
< Date: Tue, 09 Oct 2018 12:43:21 GMT
<
* Closing connection 0
So we can see the connection close is coming from server after response is served,
is there way possible ,not to close the connection after response is served .

Curl Was closing the connection since content length was not present in the response body
after adding the same it started working
> POST /stir/v1/signing HTTP/1.1
Host: [FD00:10:6B50:4510:0:0:0:53]:8101
Accept: application/json
Content-Type: application/json
Content-Length: 325
* upload completely sent off: 325 out of 325 bytes
< HTTP/1.1 200 OK
< Server: HTTP/1.1 Python/3.5.2
< Date: Thu, 18 Oct 2018 09:13:11 GMT
< Content-type: Application/json
< Content-length: 150
<
* Connection #1 to host FD00:10:6B50:4510:0:0:0:53 left intact

Related

OkHttp3: Getting an 'unexpected end of stream' exception while reading a large HTTP response

I have a Java client, that is making a POST call to the v1/graphql endpoint of a Hasura server (v1.3.3)
I'm making the HTTP call using the Square okhttp3 library (v4.9.1). The data transfer is happening over HTTP1.1, using chunked transfer-encoding.
The client is failing with the following error:
Caused by: java.net.ProtocolException: unexpected end of stream
at okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.kt:415) ~[okhttp-4.9.1.jar:?]
at okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.kt:276) ~[okhttp-4.9.1.jar:?]
at okio.RealBufferedSource.read(RealBufferedSource.kt:189) ~[okio-jvm-2.8.0.jar:?]
at okio.RealBufferedSource.exhausted(RealBufferedSource.kt:197) ~[okio-jvm-2.8.0.jar:?]
at okio.InflaterSource.refill(InflaterSource.kt:112) ~[okio-jvm-2.8.0.jar:?]
at okio.InflaterSource.readOrInflate(InflaterSource.kt:76) ~[okio-jvm-2.8.0.jar:?]
at okio.InflaterSource.read(InflaterSource.kt:49) ~[okio-jvm-2.8.0.jar:?]
at okio.GzipSource.read(GzipSource.kt:69) ~[okio-jvm-2.8.0.jar:?]
at okio.Buffer.writeAll(Buffer.kt:1642) ~[okio-jvm-2.8.0.jar:?]
at okio.RealBufferedSource.readString(RealBufferedSource.kt:95) ~[okio-jvm-2.8.0.jar:?]
at okhttp3.ResponseBody.string(ResponseBody.kt:187) ~[okhttp-4.9.1.jar:?]
Request Headers:
INFO: Content-Type: application/json; charset=utf-8
INFO: Content-Length: 1928
INFO: Host: localhost:10191
INFO: Connection: Keep-Alive
INFO: Accept-Encoding: gzip
INFO: User-Agent: okhttp/4.9.1
Response headers:
INFO: Transfer-Encoding: chunked
INFO: Date: Tue, 27 Apr 2021 12:06:39 GMT
INFO: Server: Warp/3.3.10
INFO: x-request-id: d019408e-e2e3-4583-bcd6-050d4a496b11
INFO: Content-Type: application/json; charset=utf-8
INFO: Content-Encoding: gzip
This is the client code used for the making the POST call:
private static final MediaType MEDIA_TYPE_JSON = MediaType.parse("application/json; charset=utf-8");
private static OkHttpClient okHttpClient = new OkHttpClient.Builder()
.connectTimeout(30, TimeUnit.SECONDS)
.writeTimeout(5, TimeUnit.MINUTES)
.readTimeout(5, TimeUnit.MINUTES)
.addNetworkInterceptor(loggingInterceptor)
.build();
public GenericHttpResponse httpPost(String url, String textBody, GenericHttpMediaType genericMediaType) throws HttpClientException {
RequestBody body = RequestBody.create(okHttpMediaType, textBody);
Request postRequest = new Request.Builder().url(url).post(body).build();
Call postCall = okHttpClient.newCall(okHttpRequest);
Response postResponse = postCall.execute();
return GenericHttpResponse
.builder()
.body(okHttpResponse.body().string())
.headers(okHttpResponse.headers().toMultimap())
.code(okHttpResponse.code())
.build();
}
This failure is only happening for large response sizes. As per the server logs, the response size (after gzip encoding) is around 52MB, but the call is still failing. This same code has been working fine for response sizes around 10-15MB.
I tried replicating the same issue through a simple cURL call, but that ran successfully:
curl -v -s --request POST 'http://<hasura_endpoint>/v1/graphql' \
--header 'Content-Type: application/json' \
--header 'Accept-Encoding: gzip, deflate, br' \
--data-raw '...'
* Trying ::1...
* TCP_NODELAY set
* Connected to <host> (::1) port <port> (#0)
> POST /v1/graphql HTTP/1.1
> Host: <host>:<port>
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Accept-Encoding: gzip, deflate, br
> Content-Length: 1840
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
} [1840 bytes data]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Date: Tue, 27 Apr 2021 11:59:24 GMT
< Server: Warp/3.3.10
< x-request-id: 27e3ff3f-8b95-4328-a1bc-a5492e68f995
< Content-Type: application/json; charset=utf-8
< Content-Encoding: gzip
<
{ [6 bytes data]
* Connection #0 to host <host> left intact
* Closing connection 0
So I'm assuming that this error is specific to the Java client.
Based on suggestions provided in similar posts, I tried the following other approaches:
Adding a Connection: close header to the request
Sending Transfer-Encoding: gzip header in the request
Setting the retryOnConnectionFailure for the OkHttp client to true
But none of these approaches were able to resolve the issue.
So, my questions are:
What could be the underlying cause for this issue? Since I'm using chunked transfer encoding here, I suppose it's not due to an incorrect content-length header passed in the response.
What are the approaches I can try for debugging this further?
Would really appreciate any insights on this. Thank you.

TikaJAXRS PUT from Python client

Apache Tika should be accessible from Python program via HTTP, but I can't get it to work.
I am using this command to run the server (with and without the two options at the end):
java -jar tika-server-1.17.jar --port 5677 -enableUnsecureFeatures -enableFileUrl
And it works fine with curl:
curl -v -T /tmp/tmpsojwBN http://localhost:5677/tika
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5677 (#0)
> PUT /tika HTTP/1.1
> Host: localhost:5677
> User-Agent: curl/7.47.0
> Accept: */*
> Accept-Encoding: gzip, deflate
> Content-Length: 418074
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Date: Sat, 07 Apr 2018 12:28:41 GMT
< Transfer-Encoding: chunked
< Server: Jetty(8.y.z-SNAPSHOT)
But when I try something like (tried different combinations for headers, here I recreated same headers as python-tika client uses):
with tempfile.NamedTemporaryFile() as tmp_file:
download_file(url, tmp_file)
payload = open(tmp_file.name, 'rb')
headers = {
'Accept': 'application/json',
'Content-Disposition': 'attachment; filename={}'.format(
os.path.basename(tmp_file.name))}
response = requests.put(TIKA_ENDPOINT_URL + '/tika', payload,
headers=headers,
verify=False)
I've tried to use payload as well as fileUrl - with the same result of WARN javax.ws.rs.ClientErrorException: HTTP 406 Not Acceptable and java stack trace on the server. Full trace:
WARN javax.ws.rs.ClientErrorException: HTTP 406 Not Acceptable
at org.apache.cxf.jaxrs.utils.SpecExceptions.toHttpException(SpecExceptions.java:117)
at org.apache.cxf.jaxrs.utils.ExceptionUtils.toHttpException(ExceptionUtils.java:173)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.findTargetMethod(JAXRSUtils.java:542)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:177)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:77)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:274)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:261)
at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:76)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1088)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1024)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:748)
I've also tried to compare ( with nc -l localhost 5677 | less) what is so different with two requests (payload abbreviated):
From curl:
PUT /tika HTTP/1.1
Host: localhost:5677
User-Agent: curl/7.47.0
Accept: */*
Content-Length: 418074
Expect: 100-continue
%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)
From Python requests library:
PUT /tika HTTP/1.1
Host: localhost:5677
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-requests/2.13.0
Content-type: application/pdf
Content-Length: 246176
%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)
The question is, what is the correct way to call Tika server from Python?
I've also tried python tika library in client-only mode and using tika-app via jnius. With tika client, as well as using tika-app.jar with pyjnius, I only freezes (call never returns) when I use them in a celery worker. At the same, pyjnius / tika-app and tika-python script both work nicely in a script: I have not figured out what is wrong inside celery worker. I guess, something to do with threading and/or initialization in wrong place. But that is a topic for another question.
And here is what tika-python requests:
PUT /tika HTTP/1.1
Host: localhost:5677
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-requests/2.13.0
Content-Disposition: attachment; filename=tmpb3YkTq
Content-Length: 183234
%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)
And now it seems like this is some kind of a problem with tika server:
$ tika-python --verbose --server 'localhost' --port 5677 parse all /tmp/tmpb3YkTq
2018-04-08 09:44:11,555 [MainThread ] [INFO ] Writing ./tmpb3YkTq_meta.json
(<open file '<stderr>', mode 'w' at 0x7f0b688eb1e0>, 'Request headers: ', {'Accept': 'application/json', 'Content-Disposition': 'attachment; filename=tmpb3YkTq'})
(<open file '<stderr>', mode 'w' at 0x7f0b688eb1e0>, 'Response headers: ', {'Date': 'Sun, 08 Apr 2018 06:44:13 GMT', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/json', 'Server': 'Jetty(8.y.z-SNAPSHOT)'})
['./tmpb3YkTq_meta.json']
Cf:
$ tika-python --verbose --server 'localhost' --port 5677 parse text /tmp/tmpb3YkTq
2018-04-08 09:43:38,326 [MainThread ] [INFO ] Writing ./tmpb3YkTq_meta.json
(<open file '<stderr>', mode 'w' at 0x7fc3eee4a1e0>, 'Request headers: ', {'Accept': 'application/json', 'Content-Disposition': 'attachment; filename=tmpb3YkTq'})
(<open file '<stderr>', mode 'w' at 0x7fc3eee4a1e0>, 'Response headers: ', {'Date': 'Sun, 08 Apr 2018 06:43:38 GMT', 'Content-Length': '0', 'Server': 'Jetty(8.y.z-SNAPSHOT)'})
2018-04-08 09:43:38,409 [MainThread ] [WARNI] Tika server returned status: 406
['./tmpb3YkTq_meta.json']

Request Issue Failed with INVALID_ARGUMENT on Google Cloud Vision API

when executing, with a key that has worked in the past but that I haven't used for a few weeks, the following cURL
curl -v -k -s -H "Content-Type: application/json" https://vision.googleapis.com/v1/images:annotate?key=MyKey --data-binary #a.json
where a.json is
{"requests": [{"image": {"content": "SUkqADwmAAD////8gYEoGct1VHdGU..."}, "features": [{"type": "TEXT_DETECTION", "maxResults": 1}]}]}
returns
* Trying 173.194.205.239...
* Connected to vision.googleapis.com (173.194.205.239) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.googleapis.com
* Server certificate: Google Internet Authority G2
* Server certificate: GeoTrust Global CA
> POST /v1/images:annotate?key=MyKey HTTP/1.1
> Host: vision.googleapis.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 13454
> Expect: 100-continue
>
* Done waiting for 100-continue
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< Vary: X-Origin
< Vary: Referer
< Content-Type: application/json; charset=UTF-8
< Date: Wed, 10 Feb 2016 18:02:12 GMT
< Server: ESF
< Cache-Control: private
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Alternate-Protocol: 443:quic,p=1
< Alt-Svc: quic=":443"; ma=604800; v="30,29,28,27,26,25"
< Accept-Ranges: none
< Vary: Origin,Accept-Encoding
< Transfer-Encoding: chunked
<
{
"error": {
"code": 400,
"message": "Request Issue Failed.",
"status": "INVALID_ARGUMENT"
}
}
* Connection #0 to host vision.googleapis.com left intact
Version 1 of the Google Cloud Vision API (beta) does not permit TIFF [1]. Here is a list of the currently supported formats:
JPEG
PNG8
PNG24
GIF
Animated GIF (first frame only)
BMP
WEBP
RAW
ICO
[1] https://cloud.google.com/vision/docs/image-best-practices#image_types
Turns out this is because I was sending base64 encoded tif images. Works fine for PNGs. Pretty sure I was told this in the docs.

how to change standard header value in restlets?

I have the following get:
#Get
public String represent(Variant variant) throws ResourceException
{
String text = "returntext";
text+="\r\n";
return text;
}
The response from invoking this service:
CFG - HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Type: text/plain;charset=UTF-8
Date: Mon, 29 Jul 2013 19:59:37 GMT
Server: Restlet-Framework/2.0.9
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Length: 118
Connection: keep-alive
How do I change the connection header value to close ?
I think this maybe a restlet bug.
Whether the server closes the connection or not, depends on whether the client request asks for the connection to close or not.
Here is a sample Server code:
import org.restlet.data.Form;
import org.restlet.data.MediaType;
import org.restlet.data.Parameter;
import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;
import org.restlet.util.Series;
public class TestRestlet extends ServerResource {
#Get
public String getImpl(){
return "Sample Response Text\r\n";
}
}
And here is what i got on linux commmand line (using only telnet):
[Please note that the last line in request-header in each request is followed by 2 newlines] [To avoid any confusion, some of the requests do not contain request-body.]
[root#mylinuxserver]# telnet 172.16.101.34 6060
Trying 172.16.101.34...
Connected to win7comp01 (172.16.101.34).
Escape character is '^]'.
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=C2E77F4D0437E525A0FC66498EF09E8B; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:25:44 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Sample Response Text
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
Connection: Keep-Alive
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=1873DE26443F5DF62379B895AEA0F004; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:25:48 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Sample Response Text
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
Connection: close
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=43EC7C9AACC6C0CEF6FAC8F608B1D79C; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:25:57 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Connection: close
Sample Response Text
Connection closed by foreign host.
[root#mylinuxserver]# telnet 172.16.101.34 6060
Trying 172.16.101.34...
Connected to win7comp01 (172.16.101.34).
Escape character is '^]'.
GET /TestRestlet HTTP/1.0
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=2C879A91F2501DD9D3B39EF50C3F46CA; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:26:09 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Connection: close
Sample Response Text
Connection closed by foreign host.
[root#mylinuxserver]# telnet 172.16.101.34 6060
Trying 172.16.101.34...
Connected to win7comp01 (172.16.101.34).
Escape character is '^]'.
GET /TestRestlet
Sample Response Text
Connection closed by foreign host.
[root#mylinuxserver]#
In the above examples, several types of HTTP connections are made.
The response to the 1st request:
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
[Note: the line Host: 172.16.101.34:6060 is followed by 2 \r\n: \r\n\r\n]
is:
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=C2E77F4D0437E525A0FC66498EF09E8B; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:25:44 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Sample Response Text
The connection is not closed yet, and we send another request:
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
Connection: Keep-Alive
That gets the response:
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=1873DE26443F5DF62379B895AEA0F004; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:25:48 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Still the connection is not closed.
However on the 3rd request:
GET /TestRestlet HTTP/1.1
Host: 172.16.101.34:6060
Connection: close
The connection is closed, because the request contains Connection: close header.
You can see the telnet exits after displaying a message: Connection closed by foreign host.
There are 2 more sample request-response in the above given examples:
1.An HTTP 1.0 request:
GET /TestRestlet HTTP/1.0
With response:
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=2C879A91F2501DD9D3B39EF50C3F46CA; Path=/hotelSoft
Date: Wed, 31 Jul 2013 08:26:09 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Type: application/json;charset=UTF-8
Content-Length: 22
Connection: close
Sample Response Text
And the telnet exits after displaying: Connection closed by foreign host.
2.A request without HTTP version mentioned:
GET /TestRestlet
And response is: (Without headers)
Sample Response Text
And the telnet exits with a message: Connection closed by foreign host.
Conclusion:
Whatever is your client / client-program , make it send an HTTP-1.0 request , or a HTTP-1.1 request with Connection: close header.
In Java, you achieve this by:
import java.net.HttpURLConnection;
import java.net.URL;
.
.
.
HttpURLConnection httpURLConnection = (HttpURLConnection) new URL("http://....").openConnection();
httpURLConnection.setRequestProperty("Connection","close");
// rest of the code here....
Also check if a statement like this:
httpURLConnection.disconnect();
can help you disconnect the connection.

urlopen debug doesn't show header values

In python3 ,I was trying to see the header values sent in the request
>>from urllib.request import urlopen
>> url1='http://diveintopython3.org/examples/feed.xml'
>>from http.client import HTTPConnection as httpcon
>>httpcon.debuglevel = 1
>>resp1 = urlopen(url1)
this produced
send: b'GET /examples/feed.xml HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: diveintopython3.org\r\nUser-Agent: Python-urllib/3.3\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control header: Pragma header: Content-Type header: Expires header: Server header: X-AspNet-Version header: X-Powered-By header: Date header: Content-Length header: Age header: Connection
whereas curl gives me the header values
$curl -I http://diveintopython3.org/examples/feed.xml
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 783
Content-Type: text/html; charset=utf-8
Expires: -1
Server: ATS/3.2.4
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Fri, 31 May 2013 02:48:12 GMT
Age: 0
Connection: keep-alive
what should I do to get the header values listed (as debug info) in python3 ?
The object returned by urllib.request.urlopen being an http.client.HTTPResponse object, you can use all of the methods it defines. There is one called getheaders which does what you want:
>>> from urllib.request import urlopen
>>> url1 = 'http://diveintopython3.org/examples/feed.xml'
>>> r = urlopen(url1)
>>> r.getheaders()
[('Cache-Control', 'no-cache'), ('Pragma', 'no-cache'), ('Content-Type', 'text/html; charset=utf-8'), ('Expires', '-1'), ('Server', 'ATS/3.2.4'), ('X-AspNet-Version', '4.0.30319'), ('X-Powered-By', 'ASP.NET'), ('Date', 'Fri, 31 May 2013 11:43:46 GMT'), ('Content-Length', '783'), ('Age', '0'), ('Connection', 'close')]
See the http.client documentation for more infos.

Resources