I have an Android app that uses OkHttp to send Requests to various HTTP 1.1 servers.
The Problem
It works in almost all cases, but will eventually time out for one server that does not separate the status line and message body with a CRLF line break. Reviewing the intermediate response, it appears OkHttp is interpreting the message body as a list of headers.
The message body will always start with a D3 ASCII character. Is it possible to get OkHttp to treat a "header" that starts with this character as the response body?
Tried
I tried using an Interceptor that manually sends the request to the socket and reads from the socket until CRLF or D3 is received (also collecting the StatusLine and headers). But OkHttp requires Chain.proceed() to be called (which duplicates the request).
Unpleasant Solution
My goal is to solve this with a Call. If that is impossible, it appears my only option would be to get a socket manually from the Socket factory, send the request, and manually parse the response.
Really would rather not reinvent the wheel. Any other solutions would be greatly appreciated.
Never found a way to fix this directly with OkHttp. Thankfully, it accepts a SocketFactory. What I had to do was:
Add a socket factory.
Which created a custom socket.
Which had a custom FilteredInputStream that overrides all read methods.
Each read method calls the super read method and modifies the bytes.
If (byte)0xD6 is encountered before the first two CRLF pairs, it is replaced by CR. Then LF and D6 are buffered or appended to the result (depending on the method call).
You can't sadly just override read(). read(byte[] b, int off, int len) is also called (the parent implementation does not call read()).
This fixes the request structure, allowing it to be parsed by OkHttp, so the framework can still be taken advantage of and the request does not need to be sent directly.
Related
I'd like to accept and respond to JSON requests in Odoo from sources that may be out of my control. The reason this is not straightforward is because Odoo is forcing me to use JSON-RPC, which is not suitable for the source I'm interacting with.
For example, if I set the route type to http in the #http.route decorator, Odoo rejects the request if the mimetype is application/json but the body has no content. This isn't going to work in my case because I may not be able to choose what the other source sends to me. Additionally, I am unable to send back a custom JSON response unless the incoming request doesn't have the application/json mimetype, which again is not in my control.
I have done a lot of searching on the internet and read much of Odoo's HTTP source code. The "solution" I keep seeing everywhere is to monkey patch the JsonRequest class one way or another. This allows me to indeed respond with whatever I want, however it doesn't allow me to accept whatever the service may send me.
One specific case I need to be able to handle is incoming application/json GET requests with no body. How can I achieve this despite Odoo's heavy handed JSON-RPC handing?
There is no correct way to accomplish this, I'd call the described method acceptable. It applies to versions of Odoo 10 through 15.
In my opinion, it would be better to leave JsonRequest class alone and let it do its JSON-RPC related job. There is odoo.http.Root.get_request method which constructs the json-rpc or http request object, depending on the content type:
class Root(object):
"""Root WSGI application for the OpenERP Web Client.
"""
# ...
def get_request(self, httprequest):
# deduce type of request
if httprequest.mimetype in ("application/json", "application/json-rpc"):
return JsonRequest(httprequest)
else:
return HttpRequest(httprequest)
This point seems to be the most relevant one to be patched, with returning the custom request class object from this method. There is an issue, though - this method is called prior to any route detection. You have to invent a suitable method to tell, which request class object to return.
To have an idea about a possible implementation, please, see OCA base_rest module.
I have a requirement where I need to make a HTTP request to a Flask server where the payload is a question(string) and a paragraph(string). The server uses machine learning to find the answer to the question within the paragraph and return it.
Now, the paragraph can be huge, as in thousands of words. So will a GET request with a JSON payload be appropriate? or should I be using POST?
will a GET request with a JSON payload be appropriate?
No - the problem here is that the payload of a GET request has no defined semantics; you have no guarantees that intermediate components will do the right thing with your request.
For example: caches are going to assume that the payload of the request is irrelevant, so your GET request might get a response for a completely different document.
should I be using POST?
Today, you should be using POST.
Eventually, you'll probably end up using the safe-method-with-body, once the HTTP-WG figures out the semantics of the new method and adoption has taken hold.
In a new Phoenix app the Plug.Head plug is present by default and I was intrigued about its significance.
I know that "the HEAD method is identical to GET except that the server MUST NOT send a message body in the response".
I think the official Phoenix guides are top-notch but this threw me off in the Routing guide:
Plug.Head - converts HEAD requests to GET requests and strips the
response body
If HEAD requests are without a body then why is this needed? I thought maybe to rein in malformed requests but looking at the Plug.Head implementation, it just switches the HEAD method to GET.
def call(%Conn{method: "HEAD"} = conn, []), do: %{conn | method: "GET"}
def call(conn, []), do: conn
end
The closest thing I was able to find on this topic is a question on ServerFault but it was related to NGINX and a flawed application logic where HEAD requests needed to be converted to GET and the respective GET responses back to HEAD.
Since Phoenix is largely inspired by Rails, you can safely bet Plug.Head is inspired by Rack::Head.
A HEAD request returns the same response as GET but with headers only. So to produce the correct headers, they're routed to GET actions in your Phoenix app.
However to produce the correct (empty) body, the response's body must be stripped. Because Rack::Head is middleware, it gets to do so after it gets the response from controllers.
In contrast, Plug's architecture works more like a pipeline, Plug.Head modifies the method and passes conn along, but never sees it again.
If you see cdegroot's answer, the responsibility to strip the response's body is passed to the Plug.Conn.Adapter to implement (i.e. the webserver).
AFAICT, the idea is that Plug.Head just ensures that the request is processed as a GET; the second part of implementing HEAD, which is not sending a body, is done by the Plug connection adapters. The documentation for most callbacks, like send_resp, specifies that "If the request has method "HEAD", the adapter should not send the response to the client."
I know that XMLHttpRequest and ServletRequest are not different in theory by searching similar questions.
But some details confuse me.
Fox example, if I send an XMLHttpRequest to the server, how does the client know the response is for the XMLHttpRequest rather than for the ServletRequest?
How does the client distinguishes the response type?
Otherwise, on the server side, when I call method:
response.getWriter().write(str);
Will the argument str be present on the browser?
This may be of help to you.
while the standard HTTP request makes a 'synchronous' call and must wait for the response and makes a page-reload (you always get a new html-page to display) a XMLHttpRequest may be used sync (not typical) and async (the better way) without a page-reload. you may ask for the response with javascript and the response is usually xml- or json-data that you may process with js and update parts of your page through the use of dom-methods that manipulate your document ... so you don't need an entire page-reload because all of that is running in the 'background' ...
This should also help with how the two requests are treated differently by servers and clients.
How do I handle a server error in the middle of an Http message?
Assuming I already sent the header of the message and I am streaming the
the body of the message, what do I do when I encounter an unexpected error.
I am also assuming this error was caused while generating the content and not a connection error.
(Greatly) Simplified Code:
// I can define any transfer encoding or header fields i need to.
send(header); // Sends the header to the Http client.
// Using an iterable instead of stream for code simplicity's sake.
Iterable<String> stream = getBodyStream();
Iterator<String> iterator = stream.iterator();
while (iterator.hasNext()) {
String string;
try {
string = iterator.next();
catch (Throwable error) { // Oops! an error generating the content.
// What do i do here? (In regards to the Http protocol)
}
send(string);
}
Is there a way to tell the client the server failed and should either retry or abandon the connection or am I sool?
The code is greatly simplified but I am only asking in regards to the protocol and not the exact code.
Thank You
One of the following should do it:
Close the connection (reset or normal close)
Write a malformed chunk (and close the connection) which will trigger client error
Add a http trailer telling your client that something went wrong.
Change your higher level protocol. Last piece of data you send is a hash or a length and the client knows to deal with it.
If you can generate a hash or a length (in a custom header if using http chunks) of your content before you start sending you can send it in a header so your client knows what to expect.
It depends on what you want your client to do with the data (keep it or throw it away). You may not be able to make changes on the client side so the last option will not work for example.
Here is some explanation about the different ways to close. TCP option SO_LINGER (zero) - when it's required.
I think the server should return a response code start with 5xx as per RFC 2616.
Server Error 5xx
Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents SHOULD display any included entity to the user. These response codes are applicable to any request method.