I am working on Ubuntu 12.04. This is my simple code for implementing HTTP GET method using URLConnection.
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
public class HttpURLConnectionExample {
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception {
HttpURLConnectionExample http = new HttpURLConnectionExample();
System.out.println("Testing 1 - Send Http GET request");
http.sendGet();
}
// HTTP GET request
private void sendGet() throws Exception {
String url = "https://www.google.com/search?q=flower";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
}
}
But when i compile and run this code from ubuntu terminal, the output of this code does not give the content of the page specified by the URL. Rather it gives the following output
Testing 1 - Send Http GET request
Sending 'GET' request to URL : http://www.google.com/search?q=flower
Response Code : 307
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>307 Temporary Redirect</title></head><body><h1>Temporary Redirect</h1><p>The document has moved here.</p><hr><address>Apache/2.2.22 (Fedora) Server at www.google.com Port 80</address></body></html>
This issue holds for any URL I specify in the code. Moreover, I tried to access web content using telnet client like
telnet www.google.com 80
GET /
and it gives the similar result not only for www.google.com but for every URL.
I am a student at IIT Bombay and may be it has something to do with https://ifwb.iitb.ac.in.
I also want to stick to java.net and not apache httpclient. So help me out of this.
It seems you're rejected by the server due incomplete request. It's a good idea to use any sniffer like Fiddler or Wireshark to "learn by example": compare your requests and requests from particular software like browsers.
Below is an excerpt from Wireshark dump, how IE10 sends GET request to interested URL. As you can see, there are various fields that describe capabilities and expectations of your client side so queried server can return the answer in most suitable and consumable form. Consult with Google/RFC to see the meaning of each parameter passed in:
GET /search?q=flower HTTP/1.1
Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64;
Trident/6.0)
Accept-Encoding: gzip, deflate
Host: www.google.com
DNT: 1
Connection: Keep-Alive
Cookie: [some private info here]
Related
I'm writing a GWT frontend for a personal project, and I'm having problems with some HTTP requests. When I do a CORS POST request, it works fine
String url = BASE_URL + "students/";
RequestBuilder builder = new RequestBuilder(RequestBuilder.POST, url);
builder.setHeader("Content-Type", "application/vnd.onelostlogician.student+json");
builder.setHeader("Accept", "application/json");
StringBuilder basicAuth = new StringBuilder();
basicAuth.append(username.getValue());
basicAuth.append(":");
basicAuth.append(password.getValue());
String basicAuthStr = basicAuth.toString();
builder.setHeader("Lambda-Authorization", "Basic " + toBase64(basicAuthStr.getBytes()));
StudentWriter studentWriter = GWT.create(StudentWriter.class);
try {
builder.sendRequest(studentWriter.write(student), new RequestCallback() {
public void onError(Request request, Throwable exception) {
addItemDialog.close();
responseDialog.open();
loadingIcon.setVisible(false);
responseHeading.setText("No response");
responseLabel.setText(request.toString());
}
public void onResponseReceived(Request request, Response response) {
loadingIcon.setVisible(false);
String responseText = response.getText();
List<Map.Entry<Integer, Student>> students = model.getList();
Integer studentId = Integer.parseInt(responseText);
students.add(new AbstractMap.SimpleEntry<>(studentId, student));
model.setList(students);
// clear text fields
className.setValue("");
additionLevel.setValue("");
additionProblems.setValue("");
subtractionLevel.setValue("");
subtractionProblems.setValue("");
multiplicationLevel.setValue("");
multiplicationProblems.setValue("");
divisionLevel.setValue("");
divisionProblems.setValue("");
addItemDialog.close();
}
});
} catch (RequestException _) {
// Code omitted for clarity
}
The options request gets a 200 response (chrome network inspection below):
General
Request URL:https://[redacted].execute-api.eu-west-1.amazonaws.com/v1/students/
Request Method:OPTIONS
Status Code:200
Remote Address:54.230.9.41:443
Referrer Policy:no-referrer-when-downgrade
Response Headers
access-control-allow-headers:content-type, lambda-authorization
access-control-allow-methods:post, get, put
access-control-allow-origin:*
content-length:0
content-type:application/json
date:Wed, 13 Sep 2017 14:56:32 GMT
status:200
via:1.1 5db82aafd9021b07695423274288b59e.cloudfront.net (CloudFront)
x-amz-cf-id:8nJ2gzqHFPiiDOOeEelzkpI7Ga9SFdEcljiLt2pvm7Z995_GicxPVw==
x-amzn-requestid:bb0e23db-9893-11e7-bbbe-9bea7d9d70bf
x-amzn-trace-id:sampled=0;root=1-59b94720-d892209d8c5c2a04832bdb85
x-cache:Miss from cloudfront
Request Headers
:authority:[redacted].execute-api.eu-west-1.amazonaws.com
:method:OPTIONS
:path:/v1/students/
:scheme:https
accept:*/*
accept-encoding:gzip, deflate, br
accept-language:en-GB,en-US;q=0.8,en;q=0.6
access-control-request-headers:content-type,lambda-authorization
access-control-request-method:POST
origin:http://127.0.0.1:8888
referer:http://127.0.0.1:8888/ArithmeticExerciseGeneratorClient.html
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
and then the POST request occurs as expected
General
Request URL:https://[redacted].execute-api.eu-west-1.amazonaws.com/v1/students/
Request Method:POST
Status Code:201
Remote Address:54.230.9.41:443
Referrer Policy:no-referrer-when-downgrade
Response Headers
access-control-allow-origin:*
content-length:1
content-type:application/json
date:Wed, 13 Sep 2017 14:56:33 GMT
status:201
via:1.1 5db82aafd9021b07695423274288b59e.cloudfront.net (CloudFront)
x-amz-cf-id:gxYrwctM75ObiPyS4nD69jXSO4dBaMAOZmXXX0mPE4wMgCdcjUSQsA==
x-amzn-requestid:bb381a33-9893-11e7-a1f1-17fd67ca388c
x-amzn-trace-id:sampled=0;root=1-59b94720-1c1e3a8d8c9ce2741c789241
x-cache:Miss from cloudfront
Request Headers
:authority:[redacted].execute-api.eu-west-1.amazonaws.com
:method:POST
:path:/v1/students/
:scheme:https
accept:application/json
accept-encoding:gzip, deflate, br
accept-language:en-GB,en-US;q=0.8,en;q=0.6
content-length:224
content-type:application/vnd.onelostlogician.student+json
lambda-authorization:Basic [redacted]
origin:http://127.0.0.1:8888
referer:http://127.0.0.1:8888/ArithmeticExerciseGeneratorClient.html
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Request Payload
{"className":"6T","additionProblemId":4,"additionNoOfProblems":5,"subtractionProblemId":3,"subtractionNoOfProblems":5,"multiplicationProblemId":2,"multiplicationNoOfProblems":5,"divisionProblemId":1,"divisionNoOfProblems":5}
Unfortunately, a PUT request to a very similar resource, on the same server, doesn't. The code is almost identical:
String url = BASE_URL + "students/" + studentId;
RequestBuilder builder = new RequestBuilder(RequestBuilder.PUT, url);
builder.setHeader("Content-Type", "application/vnd.onelostlogician.student+json");
builder.setHeader("Accept", "application/json");
StringBuilder basicAuth = new StringBuilder();
basicAuth.append(username.getValue());
basicAuth.append(":");
basicAuth.append(password.getValue());
String basicAuthStr = basicAuth.toString();
builder.setHeader("Lambda-Authorization", "Basic " + toBase64(basicAuthStr.getBytes()));
StudentWriter studentWriter = GWT.create(StudentWriter.class);
try {
builder.sendRequest(studentWriter.write(student), new RequestCallback() {
public void onError(Request request, Throwable exception) {
addItemDialog.close();
responseDialog.open();
loadingIcon.setVisible(false);
responseHeading.setText("No response");
responseLabel.setText(request.toString());
}
public void onResponseReceived(Request request, Response response) {
loadingIcon.setVisible(false);
responseDialog.open();
loadingIcon.setVisible(false);
responseHeading.setText("Response: " + response.getStatusCode());
responseLabel.setText(response.getText());
}
});
} catch (RequestException _) {
// Code omitted for clarity
}
The options request gets a 200 response:
General
Request URL:https://[redacted].execute-api.eu-west-1.amazonaws.com/v1/students/4
Request Method:OPTIONS
Status Code:200
Remote Address:54.230.9.41:443
Referrer Policy:no-referrer-when-downgrade
Response Headers
access-control-allow-headers:content-type, lambda-authorization
access-control-allow-methods:get, put
access-control-allow-origin:*
content-length:0
content-type:application/json
date:Wed, 13 Sep 2017 14:58:38 GMT
status:200
via:1.1 5db82aafd9021b07695423274288b59e.cloudfront.net (CloudFront)
x-amz-cf-id:0PoyOa6oDBSmU7iCWZyeSZFqWxZvumN8C4GtHn8rsoJK5AURbj3kxQ==
x-amzn-requestid:063270d4-9894-11e7-9d66-71b07b2689ef
x-amzn-trace-id:sampled=0;root=1-59b9479e-39be94b25784b92027fa2753
x-cache:Miss from cloudfront
Request Headers
:authority:[redacted].execute-api.eu-west-1.amazonaws.com
:method:OPTIONS
:path:/v1/students/4
:scheme:https
accept:*/*
accept-encoding:gzip, deflate, br
accept-language:en-GB,en-US;q=0.8,en;q=0.6
access-control-request-headers:content-type,lambda-authorization
access-control-request-method:PUT
origin:http://127.0.0.1:8888
referer:http://127.0.0.1:8888/ArithmeticExerciseGeneratorClient.html
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
…but after receiving a successful OPTIONS response, it does not make the PUT request at all.
In the Chrome console I get:
XMLHTTPRequest cannot load https://[redacted]/v1/students/5. Method
PUT is not allowed by Access-Control-Allow-Methods in preflight
response
I don't understand that error, given that we can see "put" in the access-control-allow-methods response header for the preflight OPTIONS request shown above.
Any ideas what I'm doing wrong?
In the POST response, the allowed methods header is
access-control-allow-methods:post, get, put
In the PUT response, the allowed methods header is
access-control-allow-methods:get, put
Note that the required method is the first in the list of the POST case, but second in the list in the PUT case. When I modified the server to put the method under consideration first in the list (and, also, to make it case sensitive, because HTTP method names are case sensitive), the browser then performed the required followup PUT request.
I need to set charset ISO-8859-1 for the responses of my web api controllers, and not UTF-8.
The controller for testing returns a POCO object like this:
public class StudyCaseController : ApiController
{
...
// GET: api/StudyCase/5
public Study Get(int id)
{
...
}
}
I've tried to set <globalization requestEncoding="iso-8859-1" responseEncoding="iso-8859-1"/> in the Web.config, but testing with a fiddler request like this:
GET http://localhost:45988/api/StudyCase/1 HTTP/1.1
User-Agent: Fiddler
Host: localhost:45988
Accept: text/xml
I've got a response like this:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/xml; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B? QzpcTUVESE9NRVxEZXNhcnJvbGxvQ1xQcm95ZWN0b3NcVmlld0NhcE1hblxWaWV3Q2FwTWFuXGFwaVxT dHVkeUNhc2VcMQ==?=
X-Powered-By: ASP.NET
Date: Tue, 24 Mar 2015 11:36:13 GMT
Content-Length: 1072
<?xml version="1.0"?>
... etc...
I've also tried to specify the charset at the request with Accept-Charset: iso-8859-1 but the same result.
For more info, i've tested it with IIS Express and IIS Server.
Thanks.
You can set supported encodings for formatters in HttpConfiguration class. 28591 is codepage for ISO-8859-1 (Latin 1).
config.Formatters.Add(new XmlMediaTypeFormatter());
config.Formatters[0].SupportedEncodings.Clear();
config.Formatters[0].SupportedEncodings.Add(Encoding.GetEncoding(28591));
Dealing with the problem my self, I found out that the problem was indeed the XML MediaTypeFormatter.
It does not support ISO-8859-1 and without the ability to change the server-side code I was forced to use another MediaTypeFormatter
https://www.nuget.org/packages/NetBike.Xml.Formatting/
or in nuget console
Install-Package NetBike.Xml.Formatting
This solved my problem (in a project using Web API 2).
Here is a demonstration of one way to set this using the HttpClient
private static HttpClient httpClient;
private static MediaTypeFormatter formatter;
private static List<MediaTypeFormatter> formatters;
public static void loadSettings()
{
//httpClient settings are set here
formatters = new List<MediaTypeFormatter>();
formatter = new NetBike.Xml.Formatting.NetBikeXmlMediaTypeFormatter();
formatters.Add(formatter);
}
public static async Task<HttpResponseMessage> GetContent(string requestMethod)
{
try
{
var returnValue = await httpClient.GetAsync(requestMethod);
return returnValue;
}
catch (Exception ex)
{
Debug.WriteLine("Communicator.GetXml: " + ex.Message);
}
return null;
}
public static async void testStuff()
{
var httpResponse = await GetContent("http://someDomain.com/someMethod");
MyModelObject myModel = await httpResponse.Content.ReadAsAsync<MyModelObject>(formatters);
}
What you want to do, is to set this as a default formatter for the whole project.
-EDIT-
Microsoft argues that this behavior is intended.
https://connect.microsoft.com/VisualStudio/feedback/details/958121
I have experienced issues with the NetBike Formatter, when formatting advanced XML objects, and even have a case where it writes BOM to the output.
Therefore I do not recommend it above the default MediaTypeSerializer, instead the solution falls back to the old "it depends"
That looks like this in code
List<MediaTypeFormatter> formatters = new List<MediaTypeFormatter>();
MediaTypeFormatter badencodingFormatter = new NetBike.Xml.Formatting.NetBikeXmlMediaTypeFormatter();
badencodingFormatter.SupportedEncodings.Clear();
badencodingFormatter.SupportedEncodings.Add(Encoding.GetEncoding("ISO-8859-1"));
MediaTypeFormatter defaultFormatter = new CustomXmlMediaTypeFormatter();
formatters.Add(badencodingFormatter);
formatters.Add(defaultFormatter);
This makes sure that anoying encoding is handled by NetBike, but only in those (hopefully rare) cases
I have to stream an ip camera inside a web application. Typically I do this by adding the url to an img tag. The problem this time is that the request requires digest authentication. Because of this, I want to create a proxy method to handle the authentication and stream out the data back to the client. I am currently trying to do this with an HttpHandler.
using System;
using System.Net;
using System.Web;
public class HelloWorldHandler : IHttpHandler
{
#region IHttpHandler Members
private const int BUFFERSIZE = 1048576;
public bool IsReusable
{
get { return false; }
}
public void ProcessRequest(HttpContext context)
{
HttpWebRequest webRequest = null;
#region Prepare the request object
webRequest = (HttpWebRequest)WebRequest.Create("http://url/cgi/image.php?type=live");
webRequest.Method = "GET";
webRequest.ContentType = "multipart/x-mixed-replace; boundary=------MJPEG FAME--------";
webRequest.Credentials = new NetworkCredential(username, password);
#endregion Prepare the request object
#region Cycle through and return output
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
// GETS STUCK HEAR FOR OBVIOUS REASONS. NOT SURE HOW TO BUFFER A CHUNK, STREAM THE CHUNK AND REPEAT
System.IO.Stream fileStream = webResponse.GetResponseStream();
Byte[] buffer = new byte[1048576];
int bytesRead = 1;
context.Response.ContentType = "image/png";
while (bytesRead > 0)
{
bytesRead = fileStream.Read(buffer, 0, BUFFERSIZE);
if (bytesRead == BUFFERSIZE)
context.Response.OutputStream.Write(buffer, 0, buffer.Length);
else if (bytesRead > 0)
{
byte[] endBuffer = new byte[bytesRead];
Array.Copy(buffer, endBuffer, bytesRead);
context.Response.OutputStream.Write(endBuffer, 0, endBuffer.Length);
}
}
fileStream.Dispose();
webResponse.Close();
#endregion Cycle through and return output
}
#endregion
}
If you look at my comments in the code, I have marked the obvious place where the code is failing. When I am getting the response stream, it will never end until an memory exception is thrown. This seems obvious enough but I am not sure how to actually handle the buffer issue. I think I need to be able to buffer a chunk, stream it, buffer anther and repeat.
For reference this an example of the headers when I make a direct connection through a browser.
REQUEST
GET http://url/cgi/image.php?type=live HTTP/1.1
Host: url
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Authorization: Digest something something something
RESPONSE
HTTP/1.1 200 OK
X-Powered-By: PHP/5.3.10-1ubuntu3.8
Content-type: multipart/x-mixed-replace; boundary=------MJPEG FAME--------
Transfer-Encoding: chunked
Date: Mon, 04 Nov 2013 23:39:22 GMT
Server: lighttpd/1.4.28
Any advice would be greatly appreciated.
Thanks,
Chad
As #Aristos mentioned, the key was to set the context.Repsonse.Flush(). Though this part of the problem has been solved, I am now presented with another problem.
I have opened up another thread specific to that problem. https://stackoverflow.com/questions/19849148/is-it-possible-to-close-an-httpcontext-object-that-is-inside-an-httphandler-from
Thanks,
Chad
Hi I have a simple restlet get method which returns a static string. It looks like the following:
#Get
public String represent() {
return "mystring\r\n";
}
A low level c app is invoking this get by going into a read loop. It never receives a finish confirmation signaling it that there is no more data left to read and times out after 20 seconds. Is there code I need to send to alert the client app that no more data is coming? Or that the get is finished?
[Note: The code written below is partly based on the samples available on http://www.restlet.org ]
HTTP 1.0 and 1.1 have a header named Content-Length. Whatever is the numeric value of that header, is the length of the body of the HTTP-response. Going a step further in HTTP 1.1, there is another header name-value, Tranfer-Encoding: chunked which indicates that the response-body is divided into parts (chunks) and each part's length is mentioned in a line just before that part is delivered.(I am not including other values of Transfer-Encoding to keep this answer concised.)
If this is my restlet server:
package restletapp;
import org.restlet.Component;
import org.restlet.data.Protocol;
import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;
public class RestletApp extends ServerResource {
public static void main(String[] args) throws Exception {
Component component = new Component();
component.getServers().add(Protocol.HTTP, 8182);
component.getDefaultHost().attach("/trace", RestletApp.class);
component.start();
}
#Get
public String toAtGet() {
return "Resource URI : " + getReference() + '\n'
+ "Root URI : " + getRootRef() + '\n'
+ "Routed part : " + getReference().getBaseRef() + '\n'
+ "Remaining part: " + getReference().getRemainingPart()
;
}
}
And this is my client (written using Sockets in Java. Just sends a minimal HTTP request, and prints each character of response on console.)
package restletapp;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.Socket;
import java.net.UnknownHostException;
public class Requester {
public static void main(String[] args) throws UnknownHostException, IOException {
Socket s=new Socket("localhost", 8182);
OutputStream os = s.getOutputStream();
os.write((
"GET /trace HTTP/1.1\r\n" //request
+ "host: localhost:8182\r\n" //request
+ "Connection-type: close\r\n\r\n" //request
).getBytes());
InputStream is = s.getInputStream();
for(int ch;(ch=is.read())!=-1;System.out.flush())
System.out.write(ch); //response, one char at a time.
is.close();
os.close();
s.close();
}
}
The client process never ends. But if I change my client program to this:
package restletapp;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.Socket;
import java.net.UnknownHostException;
public class Requester {
public static void main(String[] args) throws UnknownHostException, IOException {
Socket s=new Socket("localhost", 8182);
OutputStream os = s.getOutputStream();
os.write((
"GET /trace HTTP/1.1\r\n"
+ "host: localhost:8182\r\n"
+ "Connection-type: close\r\n\r\n"
).getBytes());
InputStream is = s.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
int bytesRead=0;
int contentLength=0;
//response headers.
for(String line;!(line=br.readLine()).isEmpty();System.out.flush()){
System.out.println(line);
String[] tokens = line.split(":| ");
if(tokens[0].equalsIgnoreCase("content-length")){
contentLength=Integer.parseInt(tokens[2]);
}
}
//response separator, between headers and body.
System.out.println();
//response body.
while(bytesRead<contentLength){
System.out.write(br.read());
System.out.flush();
bytesRead++;
}
is.close();
os.close();
s.close();
}
}
In the second version of Requester, you can see that the connection is closed by the client when response body's content-length-number of characters are read.
This is what i get using curl:
command line $ curl -i "http://localhost:8182/trace"
HTTP/1.1 200 OK
Date: Fri, 14 Jun 2013 11:54:32 GMT
Server: Restlet-Framework/2.0.15
Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept
Content-Length: 148
Content-Type: text/plain; charset=UTF-8
Resource URI : http://localhost:8182/trace
Root URI : http://localhost:8182/trace
Routed part : http://localhost:8182/trace
Remaining part:
command line $
You can see, that curl exits after reading the content.
Using Clojure 1.4 (Java 7) and the clj-http (0.6.0) library.
When doing a get request the Content-Length header is automatically included and set to zero. Some servers (lighttpd for instance) don't like this and respond with Bad Request. Is it possible to remove the said header or make the library not include it by default? I couldn't find anything relevant in the docs and googling gave me only this, which doesn't really help.
If I try:
(client/get "http://thepiratebay.se" {:debug true})
I get:
Request: nil
{:scheme :http,
:http-url "http://thepiratebay.se",
:request-method :get,
:query-string nil,
:uri "",
:server-name "thepiratebay.se",
:headers {"accept-encoding" "gzip, deflate"},
:debug true,
:body-type nil,
:server-port nil,
:body nil,
:user-info nil}
HttpRequest:
{:requestLine #<BasicRequestLine GET http://thepiratebay.se HTTP/1.1>,
:protocolVersion #<HttpVersion HTTP/1.1>,
:params
#<BasicHttpParams org.apache.http.params.BasicHttpParams#5b14a306>,
:method "GET",
:entity nil,
:class
clj_http.core.proxy$org.apache.http.client.methods.HttpEntityEnclosingRequestBase$0,
:allHeaders
[#<BasicHeader Connection: close>,
#<BasicHeader accept-encoding: gzip, deflate>],
:aborted false,
:URI #<URI http://thepiratebay.se>}
Which yields a 400 error. I tried to reproduce it in Java, using Apache HttpClient directly:
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.client.HttpClient;
import org.apache.http.HttpResponse;
import org.apache.http.Header;
public class Get {
public static void main(String args[]) throws Exception {
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(args[0]);
httpget.addHeader("Connection", "close");
httpget.addHeader("accept-encoding", "gzip, deflate");
Header[] headers = httpget.getAllHeaders();
for (Header h : headers) {
System.out.println(h.getName() + ", " + h.getValue());
}
System.out.println();
HttpResponse response = httpclient.execute(httpget);
System.out.println(response);
}
}
However, this works fine. My guess is that before calling HttpClient, clj-http is doing something that forces an empty body in the response, so HttpClient sets the header Content-Length to 0. The header is not set by clj-http if you look at the source. I would file this as an issue for clj-http.
https://github.com/dakrone/clj-http/issues