Is there a posibillity to check if a remote server supports https?
Currently im requesting https, if it doesnt work retry http and then display an error if this still does not work.
Is there a feature embedded in HTTP which indicates if https is supported?
By this I dont mean redirect etc. because these must be implemented on the server and arent always.
Silently falling back to HTTP sounds dangerous. An attacker (i.e. man-in-the-middle) might be able to force you to use the insecure channel by blocking your requests to HTTPS. Thus, I would not recommend this approach in general.
In general, you should let your users decide which protocol to use. If they specify https, you should not silently downgrade but throw an error. If they specify http however, it might be possible to also try https first and silently fall back to http if that fails (since they requested http in the first place).
An a general answer to your request: you can only try https to check if the server supports https. There is an HTTP(s) extension called HTTP Strict Transport Security (HSTS) which allows servers to indicate that all requests to them should always be performed via secure channels only. If you receive such a header in a response for an HTTPS request, you can force https in the future for the host. Note though that you have to ignore such headers receive over insecure HTTP.
In general, you can't trust any information you received over plaintext HTTP to give you any indication about security options (such as support for TLS) of the server since this information could be arbitrarily spoofed by man-in-the-middle attackers. In fact, preventing such undetectable changes is one of the main reasons to use TLS / HTTPS in the first place.
Related
I understand that HTTTPS is secured and it requires SSL certificate issued by CA authority to make the application secure. But what I do not understand is that its in-depth difference with HTTP.
My question, as a user, if I make a request to an application with HTTP or if I make same request to HTTPS what is the actual difference? The traffic remains same to both. Is there any traffic filtering happening if I use HTTPS?
Thanks
HTTPS, as an application protocol is just HTTP over TLS, so there are very few differences, the s in the URL and some consequences for proxy, that is all.
Now you are speaking about the traffic and the filtering. Here you have a big difference because using TLS adds confidentiality and integrity: passive listeners will see nothing about the HTTP data exchanged, including headers. The only thing visible will be the hostname (taken from the https:// URL) as this is needed at the TLS level before HTTP even happens, through a mechanism called SNI (Server Name Indication) that is now used everywhere to be able to install multiple services using TLS under different names but with a single IP address.
I'm implementing an HTTP proxy. Sometimes when a browser makes a request via my proxy, I get an error such as ECONNRESET, Address not found, and the like. These indicate errors below the HTTP level. I'm not talking about bugs in my program -- but how other servers behave when I send them an HTTP request.
Some servers might simply not exist, others close the socket, and still others not answer at all.
What is the best way to report these errors to the caller? Is there a standard method that, if I use it, browsers will convert my HTTP message to an appropriate error message? (i.e. they get a reply from the proxy that tells them ECONNRESET, and they act as though they received the ECONNRESET themselves).
If not, how should it be handled?
Motivations
I really want my proxy to be totally transparent and for the browser or other client to work exactly as if it wasn't connected to it, so I want to replicate the organic behavior of errors such as ECONNRESET instead of sending an HTTP message with an error code, which would be totally different behavior.
I kind of thought that was the intention when writing an HTTP proxy.
There are several things to keep in mind.
Firstly, if the client is configured to use the proxy (which actually I'd recommend) then fundamentally it will behave differently than if it were directly connecting out over the Internet. This is mostly invisible to the user, but affects things like:
FTP URLs
some caching differences
authentication to the proxy if required
reporting of connection errors etc <= your question.
In the case of reporting errors, a browser will show a connectivity error if it can't connect to the proxy, or open a tunnel via the proxy, but for upstream errors, the proxy will be providing a page (depending on the error, e.g. if a response has already been sent the proxy can't do much but close the connection). This page won't look anything like your browser page would.
If the browser is NOT configured to use a proxy, then you would need to divert or intercept the connection to the proxy. This can cause problems if you decide you want to authenticate your users against the proxy (to identify them / implement user-specific rules etc).
Secondly HTTPS can be a real pain in the neck. This problem is growing as more and more sites move to HTTPS only. There are several issues:
browsers configured to use a proxy, for HTTPS URLS will firstly open a tunnel via the proxy using the CONNECT method. If your proxy wants to prevent this then any information it provides in the block response is ignored by the browser, and instead you get the generic browser connectivity error page.
if you want to provide any other benefits one normally wishes from a proxy (e.g. caching / scanning etc) you need to implement a MitM (Man-in-the-middle) and spoof server SSL certificates etc. In fact you need to do this if you just want to send back a block-page to deny things.
There is a way a browser can act a bit more like it was directly connected via a proxy, and that's using SOCKS. SOCKS has a way to return an error code if there's an upstream connection error. It's not the actual socket error code however.
These are all reasons why we wrote the WinGate Internet Client, which is a LSP-based product for our product WinGate. Client applications then learn the actual upstream error codes etc.
It's not a favoured approach nowadays though, as it requires installation of software on the client computer.
I wouldn't provide them too much info. Report what you need through internal logs in case you have to solve the problem. Return a 400, 403 or 418. Why? Perhaps the're just hacking.
If a client is requesting a domain that does not have a valid CA signed certificate and the server not intend on supporting HTTPS but does support HTTP for this domain, what is the best way to handle this in the web server. Note, the server does handle requests for SSL (HTTPS) on other domains so it is listening on 443.
Example where this would apply is for multi sub-domains where the sub-domains are dynamically created and thus making it extremely difficult to register CA signed certificates.
I've seen people try to respond with HTTP error codes but these seem moot as the client (browser) will first verify the certificate and will present the hard warning to the user before processing any HTTP. Therefore the client will only see the error code if they "proceed" past the cert warning.
Is there a canonical way of handling this scenario?
There is no canonical way for this scenario. Clients don't automatically downgrade to HTTP if HTTPS is broken and it would be a very bad idea to change clients in this regard - all what an attacker would need to do to attack HTTPS would be to infer with the HTTPS traffic to make a client downgrade to unprotected HTTP traffic.
Thus, you need to make sure that the client either does not try to attempt to access URL's which do not work properly (i.e. don't publish such URL's) or to make sure that you have a working certificate for these subdomains, i.e. adapt the processes for creation of subdomains so that they not only have an IP address but also a valid certificate (maybe use wildcard certificates).
Considering these websites don't have to work with SSL, the webserver should close all SSL connections for them in a proper way.
There is no canonical way for this, but RFC 5246 implicitly suggests to interrupt the handshake on the server side by using the user_cancel + close_notify alerts. How to achieve this is another question, it will be a configuration of the default SSL virtual host.
user_canceled
This handshake is being canceled for some reason unrelated to a
protocol failure. If the user cancels an operation after the
handshake is complete, just closing the connection by sending a
close_notify is more appropriate. This alert should be followed
by a close_notify. This message is generally a warning.
If you are dealing with subdomains, you probably can use a wildcard certificate for all of your subdomains.
Adding the CA certificate to your client will remove the warning (that's what companies do, no worry).
When hosting with Apache, for example, you can use VirtualDocumentRoot to add domains without editing your configuration. Have a look at the solution provided here : Virtual Hosting in SSL with VirtualDocumentRoot
I was just wondering this small little question. I know it is irreverent to coding, but I just had to know quickly.
If you type in http:// for a https:// will it still take you to the correct place?
That is mostly dependent on the server configuration. The server has to accept the initial HTTP request and be configured to redirect the client to an appropriate HTTPS url.
That being said, there are some Internet standards related to automating HTTP-to-HTTPS upgrades. HTTP Strict Transport Security and Upgrade Insecure Requests allow an HTTP/S server to tell clients that it wants them to automatically use HTTPS for all subsequent requests. If a client visits an HSTS/UIR-enabled server, it will receive a normal HTTP response with additional HSTS/UIR-related headers. If the client supports HSTS/UIR, it will then know to automatically send all subsequent HTTP requests to that same server using HTTPS, and in the case of UIR also treat any received HTTP URLs as if they were HTTPS URLs.
Wikipedia defines HTTP(S) or S-HTTP as a security layer over HTTP:
Technically, it is not a protocol in and of itself; rather, it is the
result of simply layering the Hypertext Transfer Protocol (HTTP) on
top of the SSL/TLS protocol, thus adding the security capabilities of
SSL/TLS to standard HTTP communications.
Logically, it implies that every feature and aspect of HTTP (e.g. methods and status codes) exists in HTTPS.
Should I expect any caveats or differences when switching an existing HTTP REST interface to HTTPS?
There doesn't seem to be any limitation of what you can do with HTTP but not HTTPS. The only limitations/differences relate to the fact that the connection is encrypted. As Eugene mentioned, this includes the fact that HTTPS cannot be proxy-cached. There are however some caveats:
HTTP inline content inside HTTPS page
If you start using HTTPS for sites where you originally used HTTP, problems might arise with HTTP inline content, e.g. if you use 3rd party HTTP services or cross-domain content:
scripts: google maps API
iframes: other webs, facebook, google ads, ...
images, static google maps, ...
In that case, many browsers will disable the "insecure" HTTP content inside HTTPS page! For the user, it is very hard to switch this off (especially in Firefox).
The only reliable way around that is to use protocol-relative URLs. So, instead of:
<script src="http://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false"></script>
which would break on HTTPS page, you will just use
<script src="//maps.googleapis.com/maps/api/js?v=3.exp&sensor=false"></script>
which will work as HTTP on HTTP page and as HTTPS on HTTPS page. This fixes the problem.
The downside of course is that it is useless encryption of large amount of network traffic, that is not vulnerable and wouldn't normally have to be encrypted. This is the cost of the paranoid browser approach to security (like year ago, there was no warning from FF in this situation, and I was completely happy. World changes ...)
If you don't have signed SSL certificate for your domain
Another caveat of course is that if you don't have SSL certificate for your domain which is signed by trusted CA authority, then if your users will use HTTPS, they will have to pass a terrible scary 4-5 step procedure to accept the certificate. It is almost impossible and unprofessional to expose an average user (unaware of the problematics) to this. You will have to buy certificate in this case. Many times you end up using HTTP instead of HTTPS because of this. So if you cannot afford to buy the certificate, the browser paranoia forces you many times to use insecure HTTP protocol instead of HTTPS. Again, 6-7 years ago, it wasn't the case.
Mixing HTTP and HTTPS - cookie and authorization problems
If you use both HTTP and HTTPS within the same session, you might run into problems because sometimes they will be treated as separate sites (even if the rest of the URL is the same). This might be the case of cookies - in some cases they will not be shared between HTTP and HTTPS. Also, the HTTP authentication - RFC2617 will not be shared between HTTP and HTTPS. However, this type of authentication is now very rare on the Web, possibly due to lack of customization of the login form.
So, if you start using HTTPS, easiest way is then to use HTTPS only.
After several years of running HTTP over HTTPS, I am not aware of any other caveats.
Performance Considerations
HTTP vs HTTPS performance
HTTPS vs HTTP speed comparison
HTTPS Client/Broswer Caching
Top 7 Myths about HTTPS - Note commentary on HTTPS caching that is handled differently in browsers. It's from 2011 though, the browsers might have changed.
Will web browsers cache content over https
More on why there is no HTTPS proxy caching
Can a proxy server cache SSL GETs? If not, would response body encryption suffice?
UPGRADE command in Websockets via HTTPS
While the WebSocket protocol itself is unaware of proxy servers and firewalls, it features an HTTP-compatible handshake so that HTTP servers can share their default HTTP and HTTPS ports (80 and 443) with a WebSocket gateway or server. The WebSocket protocol defines a ws:// and wss:// prefix to indicate a WebSocket and a WebSocket Secure connection, respectively. Both schemes use an HTTP upgrade mechanism to upgrade to the WebSocket protocol.
http://en.wikipedia.org/wiki/WebSocket
As a coder of REST, I do not see any possible caveats when you switch HTTP REST to HTTPS. In times if you find some, you would definitely have them in normal HTTP REST too.