I'm implementing an HTTP proxy. Sometimes when a browser makes a request via my proxy, I get an error such as ECONNRESET, Address not found, and the like. These indicate errors below the HTTP level. I'm not talking about bugs in my program -- but how other servers behave when I send them an HTTP request.
Some servers might simply not exist, others close the socket, and still others not answer at all.
What is the best way to report these errors to the caller? Is there a standard method that, if I use it, browsers will convert my HTTP message to an appropriate error message? (i.e. they get a reply from the proxy that tells them ECONNRESET, and they act as though they received the ECONNRESET themselves).
If not, how should it be handled?
Motivations
I really want my proxy to be totally transparent and for the browser or other client to work exactly as if it wasn't connected to it, so I want to replicate the organic behavior of errors such as ECONNRESET instead of sending an HTTP message with an error code, which would be totally different behavior.
I kind of thought that was the intention when writing an HTTP proxy.
There are several things to keep in mind.
Firstly, if the client is configured to use the proxy (which actually I'd recommend) then fundamentally it will behave differently than if it were directly connecting out over the Internet. This is mostly invisible to the user, but affects things like:
FTP URLs
some caching differences
authentication to the proxy if required
reporting of connection errors etc <= your question.
In the case of reporting errors, a browser will show a connectivity error if it can't connect to the proxy, or open a tunnel via the proxy, but for upstream errors, the proxy will be providing a page (depending on the error, e.g. if a response has already been sent the proxy can't do much but close the connection). This page won't look anything like your browser page would.
If the browser is NOT configured to use a proxy, then you would need to divert or intercept the connection to the proxy. This can cause problems if you decide you want to authenticate your users against the proxy (to identify them / implement user-specific rules etc).
Secondly HTTPS can be a real pain in the neck. This problem is growing as more and more sites move to HTTPS only. There are several issues:
browsers configured to use a proxy, for HTTPS URLS will firstly open a tunnel via the proxy using the CONNECT method. If your proxy wants to prevent this then any information it provides in the block response is ignored by the browser, and instead you get the generic browser connectivity error page.
if you want to provide any other benefits one normally wishes from a proxy (e.g. caching / scanning etc) you need to implement a MitM (Man-in-the-middle) and spoof server SSL certificates etc. In fact you need to do this if you just want to send back a block-page to deny things.
There is a way a browser can act a bit more like it was directly connected via a proxy, and that's using SOCKS. SOCKS has a way to return an error code if there's an upstream connection error. It's not the actual socket error code however.
These are all reasons why we wrote the WinGate Internet Client, which is a LSP-based product for our product WinGate. Client applications then learn the actual upstream error codes etc.
It's not a favoured approach nowadays though, as it requires installation of software on the client computer.
I wouldn't provide them too much info. Report what you need through internal logs in case you have to solve the problem. Return a 400, 403 or 418. Why? Perhaps the're just hacking.
Related
Is there a posibillity to check if a remote server supports https?
Currently im requesting https, if it doesnt work retry http and then display an error if this still does not work.
Is there a feature embedded in HTTP which indicates if https is supported?
By this I dont mean redirect etc. because these must be implemented on the server and arent always.
Silently falling back to HTTP sounds dangerous. An attacker (i.e. man-in-the-middle) might be able to force you to use the insecure channel by blocking your requests to HTTPS. Thus, I would not recommend this approach in general.
In general, you should let your users decide which protocol to use. If they specify https, you should not silently downgrade but throw an error. If they specify http however, it might be possible to also try https first and silently fall back to http if that fails (since they requested http in the first place).
An a general answer to your request: you can only try https to check if the server supports https. There is an HTTP(s) extension called HTTP Strict Transport Security (HSTS) which allows servers to indicate that all requests to them should always be performed via secure channels only. If you receive such a header in a response for an HTTPS request, you can force https in the future for the host. Note though that you have to ignore such headers receive over insecure HTTP.
In general, you can't trust any information you received over plaintext HTTP to give you any indication about security options (such as support for TLS) of the server since this information could be arbitrarily spoofed by man-in-the-middle attackers. In fact, preventing such undetectable changes is one of the main reasons to use TLS / HTTPS in the first place.
For a while now many apps/application in almost all platforms (example this on android) that can be used to "inject a certain url or text" (usually of that one allowed address) to an blocked outgoing connection and magically bypass a firewall.
I have intercepted one of these connections with fiddler but cant seem to see the difference between an injected an non injected connection especially HTTPS. how to they fool ISPs? what does it take to forge connection like those?
I need to build a Web site which would have a secure connection (HTTPS) on some pages. I need to know if there will be a difference for me (as a developer) while I will write the code? I must treat differently some data or what? What is the main difference from back-end view?
From the backend point of view, there is no difference. The difference between the two is the TCP connection between the server and the client. Https will be encrypted, http is not of course, but it's all decrypted by the time it hits your code. The server will have some flags available so you can determine whether the connection is http or https (names vary depending on the server) but unless you're using that information to change the behavior of the page, you don't need to worry about it.
we need to protect our webservices with SSL (https) or any other security mechanism. Our problem is that current clients (delphi exe's) have references to our http webservices fixed in code and can not change that code.
I've tried to implement URL redirection rule from http to https but that didn't work because of the "hand shake"...Changing client to use https reference did work but saddly we can not do that for every client.
I know this question is in contradiction with encription theories but i'll fire this question anyway if anyone has any type of suggestion/idea to at least make connection or data transfer more secured (either with or without SSL protocol) without changing client side.
Thanks,
Luke
You need some kind of transparent TCP tunneling software/hardware on the clients, so the encryption occurs without the delphi clients noticing it.
My Google search using "transparent encrypted tunneling" keywords got this vendor of such solutions. There's must other vendors with similar solutions.
This is really an networking question.
PS.: hardcoding the URL is the real problem here. After the tunneling palliative is done, change that because this really will cause more headaches in future.
The client will be connecting over a port (non SSL) that will need to remain. What you could possibly do is that if you allow access both http and https you could possibly only allow http from specific IP addresses if you know them? its still not secure, but least you know where the calls are coming from and can do something about that?
In a developement environement (where often the browser and the http server are on the same machine) i want to study the exact detail of authentication schemas. So i need to trace down every http request/response.
I've tried WireShark, that is very promising. But actually on
windows machines there is a problem in sniffing the traffic on
loopback interface.
Then i've tried a browser plugin, HttpFox
0.8.10 of Firefox 12. It is good in showing requests and responses, but in the specific case of authentication, it doesn't correctly
show the "double hop" authentication, it "collapses" the first
request (the Unauthorized status code) with the next, successful
one.
Then i've tried to work with the logs of httpd, that is my
actual server, but it is required a not trivial effort to create a
log that contains all the request such as headers (the authorization
header).So it doesn't seem a good "debug" technique.
Are there other possibilities?
Go with Wireshark. The answer to this question will address the loopback issue. Wireshark is the best because it really understands the formatting of everything related to HTTP (so long as you are not using HTTPS).