I am new to python (2.7) and trying to create a base class used for sending http requests using requests and I have drafted following code.
class BaseRequest(object):
def __init__(self):
self.s = requests.Session()
def do_post(self, url, payload, headers, timeout):
response = self.s.request('POST', url,
headers=headers,
data=json.dumps(payload),
timeout=timeout)
return self._handle_response(response)
}
What is the benefit of using Session? I googled little bit and one of the benefit using Session is to have cookie persistent which I don't really need. Should I just replace with requests.post()?
How do I know the actual connection is closed? If exception was thrown in s.request will the connection be closed?
Related
I am trying to use Java 11 HTTP Client against an authenticated service, using Basic Authentication.
The authentication occurs successfully but it makes an additional round-trip to the server, to understand it should send the authentication data.
Have searched around documentation and code and at some point internally it uses some kind of cache, but I am unable to set the cache value.
Here is my client code:
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("http://someurl.com"))
.build();
HttpClient client = HttpClient.newBuilder()
.authenticator(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("user", "pass".toCharArray());
}
})
.build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
What I expected was that somehow I could tell the client to preemptively sent the authentication data, not only when the server requests.
The HttpClient behaves in the same way than HttpURLConnection in what preemptive authentication is concerned: for basic authentication it will preemptively send the credentials if it finds them in its cache. However, the cache is populated after the first successful request (or more exactly after the response headers indicating that the authentication was successful are parsed).
If this is not satisfactory for you then a possibility is to handle authentication directly in your code by preemptively inserting the Authorization header in your request, and not setting any Authenticator.
Thanks to #daniel, this is the solution I came up with, adding the header to the HttpRequest and removing Authenticator.
String encodedAuth = Base64.getEncoder()
.encodeToString(("user" + ":" + "pass").getBytes(StandardCharsets.UTF_8));
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("http://someurl.com"))
.header("Authorization", "Basic " + encodedAuth)
.build();
HttpClient client = HttpClient.newHttpClient();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
Wish the client has some other way to tell to preemptively send authentication data rather than manually creating the Header, but this way it works.
I have a XML-RPC server (using XML-RPC.net) running as a .NET console application. I'm trying to connect to it via my ASP.NET Core (2.1.1) web app but the client keeps timing out. Postman also returns a response immediately without issues.
Here is how I'm calling it:
HttpClient client = _clientFactory.CreateClient();
client.Timeout = TimeSpan.FromSeconds(10);
var httpRequest = new HttpRequestMessage(HttpMethod.Post, instance.ServiceUrl);
var stringContent = new ByteArrayContent(Encoding.UTF8.GetBytes(request.ToString()));
httpRequest.Content = stringContent;
httpRequest.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("text/xml");
var httpResponse = await client.SendAsync(httpRequest);
var response = await httpResponse.Content.ReadAsByteArrayAsync();
I can see that the request was made successfully as the console app returns a response. Fiddler shows there was a 200 response but await client.SendAsync(httpRequest); times-out!
The request usually completes in under 10ms so the timeout value is just for debugging, if I leave it out it would take 60s. The response returns XML.
I've tried rewriting this to use StringContent and using PostAsync, same issue. I also attempted to rewrite this using WebClient but it returned The remote server returned an error: (100) Continue. not sure if that's relevant.
Been stuck on this for a whie, anyone know what could be happening?
OK I did some googling and it looks like I needed this line:
client.DefaultRequestHeaders.ExpectContinue = true;
It was definitely related to 100 status code returned not being handled properly.
Found it here:
https://social.msdn.microsoft.com/Forums/en-US/042016f0-d70e-42f9-9924-5febeb2bea86/excluding-the-quotexpect-100continuequot-header-from-httpwebrequest-posts?forum=winappswithcsharp
There is a third-party website that uses HTTPS and where the start page performs a POST upon login. I have inspected that POST request in my browser and then I have been able to manually create the request with Fiddler's composer. Thus, depending on the credentials, I could either successfully or unsuccessfully log in with Fiddler. The return code is always 302, which comes along with either a redirect (header "Location") to the user management page or a login failed page, respectively.
However, when I create that request using the Retrofit library, it does not work. I get response code 200, which in this specific case is not to be considered a success.
In order to inspect the POST request from Retrospect, I have directed it to Fiddler (http://localhost:8888) instead of the third-party URL. If I copy that request into the composer and adjust the URL to be the third-party one, the request does work. I.e., I could not find anything wrong with the request built by Retrofit.
Does anybody have an idea what could be wrong?
My code is written in Kotlin, but should be easily understandable if you know Java:
import okhttp3.ResponseBody
import retrofit2.Call
import retrofit2.Retrofit
import retrofit2.http.*
interface MyApi {
#POST("<relative login url>")
#FormUrlEncoded
#Headers(
//...
)
fun login(
#Field("username") username: String,
#Field("password") password: String
) : Call<ResponseBody>;
}
fun main(args: Array<String>) {
val baseUrl = "https://<url>"
val retrofit = Retrofit.Builder().baseUrl(baseUrl).build()
val myApi = retrofit.create(MyApi::class.java)
val code = myApi.login("<username>", "<password>").execute().code()
println(code)
}
As already stated in the comments, but to make it more understandable to others too here goes an answer.
When using retrofit with okhttp redirects will be followed by default. The reason for this is because the default okhttp client is set to follow redirects. This is why you never get a 302 - The redirect is followed automatically and you get the 200 from the followed url.
You can disable this behaviour by building your retrofit instance with a properly configured okhttp client:
OkHttpClient client = new OkHttpClient.Builder()
.followRedirects(false)
.followSslRedirects(false)
.build();
Retrofit retrofit = new Retrofit.Builder()
.client(client)
// other configurations for retrofit
.build();
Notice that here we create an instance of retrofit with a client configure not to follow redirects. This will effectively make you receive the 302 and other redirect codes.
(note that I didn't fully configure the retrofit instance here to focus the answer in the important part)
While calling Rest services(remote machine) from local machine i am getting httprequest exception,I found that this exception is coming while we are making asynchronous call using httpclient in the code.
var httpresponse = await _httpClient.SendAsync(httpClientRequests);
Sometimes i am able to get response from services but it is happening intermittently.
I have cross checked the request url also,it's correct.
Also i am getting data only in debug mode not in release.
Is this server side issue or client side?
Having some problem developing a SignalR client for a Hub hosted in asp.net website with gzip compression enabled. Since we are using IIS compression, the response from SignalR also gets compressed, but, the client does not understand the response and we get a Json parsing error on the client side.
SignalR internally uses HttpWebRequest to make make http requests and HttpWebRequest can be configured to automatically decompress the response using AutomaticDecompression property. So, if somehow I can get hold of the HttpWebRequest object used by SignalR to make the request, I should be able to set the enable automatic decompression.
I thought I should be able to get access to the HttpWebRequest by providing HubConnection.Start with my custom implementation of IHttpClient, IHttpClient.GetAsync takes a prepareRequest action which I thought should give me access to the HttpWebRequest, but, HttpHelper.GetAsync wraps the HttpWebRequest with HttpWebRequestWrapper before passing to prepareRequest and HttpWebRequestWrapper does not provide access to HttpWebRequest.
HttpHelper class is internal so can't use it as well, so, I am not exactly sure how to enable automatic decompression with SignalR.
I can expose the HttpWebRequest in HttpWebRequestWrapper, but, would prefer a simpler solution if one exists. Any thougths?
I am using SignalR version 0.5.1.10822
My auto decompression HttpClient:
public class HttpClientWithAutoDecompression : IHttpClient
{
readonly DefaultHttpClient _httpClient = new DefaultHttpClient();
private readonly DecompressionMethods _decompressionMethods;
public HttpClientWithAutoDecompression(DecompressionMethods decompressionMethods)
{
_decompressionMethods = decompressionMethods;
}
public Task<IResponse> GetAsync(string url, Action<IRequest> prepareRequest)
{
Task<IResponse> task = _httpClient.GetAsync(url,
request =>
{
[ERROR: request is actually HttpRequestWrapper and
does not expose HttpWebRequest]** ]
var httpWebRequest = (HttpWebRequest) request;
httpWebRequest.AutomaticDecompression = _decompressionMethods;
prepareRequest(request);
});
return task.ContinueWith(response =>
{
Log.Debug(this, "Response: {0}", response.Result.ReadAsString());
return response.Result;
});
}
....
}
To the best of my knowledge GZip encoding and streaming do not mix. In the case of the forever frame transport the client wouldn't be able to decode any on the streaming content until the entire response, or at least a significant block of data, is received (due to the way the data is decoded). In the case of web sockets there is not support for encoding of any kind at this time, although there is apparently an extension to the specification for per message encoding being worked on.
That said, if you wanted to attempt to provide support for the LongPolling transport, the only way I can see this being possible is to provide your own SignalR IHttpClient implementation. You can see right now that the DefaultHttpClient class uses HttpHelper::GetAsync which creates the HttpWebRequest internally and you can never get your hands on that because you only have access to the IRequest which is HttpWebRequestWrapper at that point.
By creating your own IHttpClient you can take over the initial instantiation of the HttpWebRequest, set the AutomaticDecompression and then wrap that up yourself with the HttpWebRequestWrapper.