I understand that HTTTPS is secured and it requires SSL certificate issued by CA authority to make the application secure. But what I do not understand is that its in-depth difference with HTTP.
My question, as a user, if I make a request to an application with HTTP or if I make same request to HTTPS what is the actual difference? The traffic remains same to both. Is there any traffic filtering happening if I use HTTPS?
Thanks
HTTPS, as an application protocol is just HTTP over TLS, so there are very few differences, the s in the URL and some consequences for proxy, that is all.
Now you are speaking about the traffic and the filtering. Here you have a big difference because using TLS adds confidentiality and integrity: passive listeners will see nothing about the HTTP data exchanged, including headers. The only thing visible will be the hostname (taken from the https:// URL) as this is needed at the TLS level before HTTP even happens, through a mechanism called SNI (Server Name Indication) that is now used everywhere to be able to install multiple services using TLS under different names but with a single IP address.
Related
Is there a posibillity to check if a remote server supports https?
Currently im requesting https, if it doesnt work retry http and then display an error if this still does not work.
Is there a feature embedded in HTTP which indicates if https is supported?
By this I dont mean redirect etc. because these must be implemented on the server and arent always.
Silently falling back to HTTP sounds dangerous. An attacker (i.e. man-in-the-middle) might be able to force you to use the insecure channel by blocking your requests to HTTPS. Thus, I would not recommend this approach in general.
In general, you should let your users decide which protocol to use. If they specify https, you should not silently downgrade but throw an error. If they specify http however, it might be possible to also try https first and silently fall back to http if that fails (since they requested http in the first place).
An a general answer to your request: you can only try https to check if the server supports https. There is an HTTP(s) extension called HTTP Strict Transport Security (HSTS) which allows servers to indicate that all requests to them should always be performed via secure channels only. If you receive such a header in a response for an HTTPS request, you can force https in the future for the host. Note though that you have to ignore such headers receive over insecure HTTP.
In general, you can't trust any information you received over plaintext HTTP to give you any indication about security options (such as support for TLS) of the server since this information could be arbitrarily spoofed by man-in-the-middle attackers. In fact, preventing such undetectable changes is one of the main reasons to use TLS / HTTPS in the first place.
If a client is requesting a domain that does not have a valid CA signed certificate and the server not intend on supporting HTTPS but does support HTTP for this domain, what is the best way to handle this in the web server. Note, the server does handle requests for SSL (HTTPS) on other domains so it is listening on 443.
Example where this would apply is for multi sub-domains where the sub-domains are dynamically created and thus making it extremely difficult to register CA signed certificates.
I've seen people try to respond with HTTP error codes but these seem moot as the client (browser) will first verify the certificate and will present the hard warning to the user before processing any HTTP. Therefore the client will only see the error code if they "proceed" past the cert warning.
Is there a canonical way of handling this scenario?
There is no canonical way for this scenario. Clients don't automatically downgrade to HTTP if HTTPS is broken and it would be a very bad idea to change clients in this regard - all what an attacker would need to do to attack HTTPS would be to infer with the HTTPS traffic to make a client downgrade to unprotected HTTP traffic.
Thus, you need to make sure that the client either does not try to attempt to access URL's which do not work properly (i.e. don't publish such URL's) or to make sure that you have a working certificate for these subdomains, i.e. adapt the processes for creation of subdomains so that they not only have an IP address but also a valid certificate (maybe use wildcard certificates).
Considering these websites don't have to work with SSL, the webserver should close all SSL connections for them in a proper way.
There is no canonical way for this, but RFC 5246 implicitly suggests to interrupt the handshake on the server side by using the user_cancel + close_notify alerts. How to achieve this is another question, it will be a configuration of the default SSL virtual host.
user_canceled
This handshake is being canceled for some reason unrelated to a
protocol failure. If the user cancels an operation after the
handshake is complete, just closing the connection by sending a
close_notify is more appropriate. This alert should be followed
by a close_notify. This message is generally a warning.
If you are dealing with subdomains, you probably can use a wildcard certificate for all of your subdomains.
Adding the CA certificate to your client will remove the warning (that's what companies do, no worry).
When hosting with Apache, for example, you can use VirtualDocumentRoot to add domains without editing your configuration. Have a look at the solution provided here : Virtual Hosting in SSL with VirtualDocumentRoot
The question is about HTTP vs HTTPS.
If I want to anonymously load a website that forces HTTPS, like Google.com, do I need an HTTPS proxies, or can I get away with HTTP proxies?
If your proxy is SOCKS it will not care what kind of socket is connecting through it. It has its own handshake and it does not care about what happens after the handshake. Whether after the SOCKS handshake an SSL handshake (HTTPS) is started it is not a SOCKS proxy problem, it will just pass through.
Several HTTP proxies on the other hand expect HTTP headers to guide them, such a HTTP proxy will not allow HTTPS since it needs to read the headers.
On the third hand (ekhm... well, foot?), an HTTP proxy that supports HTTP CONNECT can also setup the transfer of arbitrary data. Therefore such a proxy can setup any type of socket, which can have an SSL handshake, which can then be used for HTTPS transfer.
HTTP Proxy Server supports CONNECT verb which supports HTTPS connections within HTTP Proxy. You don't need special HTTPS proxy server or any other setup.
CONNECT verb allows you to create binary socket tunnel to any given IP:Port address. So any HTTP client (all browsers), will open secure tunnel and communicate securely over proxy server. However, no one cant control or see anything that is going through the tunnel unless they implement man in middle attack by sending you self-signed certificates.
Most firewall these days automatically implement man in middle self signed certificates that are deployed in work network, so you have to probably dig more to identify whether it is really secure or not. So it may not be that anonymous.
If you're trying to access a service anonymously, you won't get this by running your own proxy. It's not clear from the original question what is meant by "proxy", e.g. local service, or remote service. You won't get anonymity by surfing through a proxy that's on your network, unless it's something like a TOR proxy which relays out through the TOR network.
As for whether proxies can support HTTPS or not, that's been covered here, it would be unusual to find a proxy that doesn't support CONNECT. However if it's a remote anonymizing service you're using, I doubt they would do MitM, since you'd need to install the signing cert into your trusted root store, so they couldn't do that surreptitiously.
Wikipedia defines HTTP(S) or S-HTTP as a security layer over HTTP:
Technically, it is not a protocol in and of itself; rather, it is the
result of simply layering the Hypertext Transfer Protocol (HTTP) on
top of the SSL/TLS protocol, thus adding the security capabilities of
SSL/TLS to standard HTTP communications.
Logically, it implies that every feature and aspect of HTTP (e.g. methods and status codes) exists in HTTPS.
Should I expect any caveats or differences when switching an existing HTTP REST interface to HTTPS?
There doesn't seem to be any limitation of what you can do with HTTP but not HTTPS. The only limitations/differences relate to the fact that the connection is encrypted. As Eugene mentioned, this includes the fact that HTTPS cannot be proxy-cached. There are however some caveats:
HTTP inline content inside HTTPS page
If you start using HTTPS for sites where you originally used HTTP, problems might arise with HTTP inline content, e.g. if you use 3rd party HTTP services or cross-domain content:
scripts: google maps API
iframes: other webs, facebook, google ads, ...
images, static google maps, ...
In that case, many browsers will disable the "insecure" HTTP content inside HTTPS page! For the user, it is very hard to switch this off (especially in Firefox).
The only reliable way around that is to use protocol-relative URLs. So, instead of:
<script src="http://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false"></script>
which would break on HTTPS page, you will just use
<script src="//maps.googleapis.com/maps/api/js?v=3.exp&sensor=false"></script>
which will work as HTTP on HTTP page and as HTTPS on HTTPS page. This fixes the problem.
The downside of course is that it is useless encryption of large amount of network traffic, that is not vulnerable and wouldn't normally have to be encrypted. This is the cost of the paranoid browser approach to security (like year ago, there was no warning from FF in this situation, and I was completely happy. World changes ...)
If you don't have signed SSL certificate for your domain
Another caveat of course is that if you don't have SSL certificate for your domain which is signed by trusted CA authority, then if your users will use HTTPS, they will have to pass a terrible scary 4-5 step procedure to accept the certificate. It is almost impossible and unprofessional to expose an average user (unaware of the problematics) to this. You will have to buy certificate in this case. Many times you end up using HTTP instead of HTTPS because of this. So if you cannot afford to buy the certificate, the browser paranoia forces you many times to use insecure HTTP protocol instead of HTTPS. Again, 6-7 years ago, it wasn't the case.
Mixing HTTP and HTTPS - cookie and authorization problems
If you use both HTTP and HTTPS within the same session, you might run into problems because sometimes they will be treated as separate sites (even if the rest of the URL is the same). This might be the case of cookies - in some cases they will not be shared between HTTP and HTTPS. Also, the HTTP authentication - RFC2617 will not be shared between HTTP and HTTPS. However, this type of authentication is now very rare on the Web, possibly due to lack of customization of the login form.
So, if you start using HTTPS, easiest way is then to use HTTPS only.
After several years of running HTTP over HTTPS, I am not aware of any other caveats.
Performance Considerations
HTTP vs HTTPS performance
HTTPS vs HTTP speed comparison
HTTPS Client/Broswer Caching
Top 7 Myths about HTTPS - Note commentary on HTTPS caching that is handled differently in browsers. It's from 2011 though, the browsers might have changed.
Will web browsers cache content over https
More on why there is no HTTPS proxy caching
Can a proxy server cache SSL GETs? If not, would response body encryption suffice?
UPGRADE command in Websockets via HTTPS
While the WebSocket protocol itself is unaware of proxy servers and firewalls, it features an HTTP-compatible handshake so that HTTP servers can share their default HTTP and HTTPS ports (80 and 443) with a WebSocket gateway or server. The WebSocket protocol defines a ws:// and wss:// prefix to indicate a WebSocket and a WebSocket Secure connection, respectively. Both schemes use an HTTP upgrade mechanism to upgrade to the WebSocket protocol.
http://en.wikipedia.org/wiki/WebSocket
As a coder of REST, I do not see any possible caveats when you switch HTTP REST to HTTPS. In times if you find some, you would definitely have them in normal HTTP REST too.
The JVM allows proxy properties http.proxyHost and http.proxyPort for specifying a HTTP proxy server and https.proxyHost and https.proxyPort for specifying a HTTPS proxy server .
I was wondering whether there are any advantages of using a HTTPS proxy server compared to a HTTP proxy server ?
Is accessing a https url via a HTTPS proxy less cumbersome than accesing it from a HTTP proxy ?
HTTP proxy gets a plain-text request and [in most but not all cases] sends a different HTTP request to the remote server, then returns information to the client.
HTTPS proxy is a relayer, which receives special HTTP request (CONNECT verb) and builds an opaque tunnel to the destination server (which is not necessarily even an HTTPS server). Then the client sends SSL/TLS request to the server and they continue with SSL handshake and then with HTTPS (if requested).
As you see, these are two completely different proxy types with different behavior and different design goals. HTTPS proxy can't cache anything as it doesn't see the request sent to the server. With HTTPS proxy you have a channel to the server and the client receives and validates server's certificate (and optionally vice versa). HTTP proxy, on the other hand, sees and has control over the request it received from the client.
While HTTPS request can be sent via HTTP proxy, this is almost never done because in this scenario the proxy will validate server's certificate, but the client will be able to receive and validate only proxy's certificate, and as name in the proxy's certificate will not match the address the socket connected to, in most cases an alert will be given and SSL handshake won't succeed (I am not going into details of how to try to address this).
Finally, as HTTP proxy can look into the request, this invalidates the idea of security provided by HTTPS channel, so using HTTP proxy for HTTPS requests is normally done only for debugging purposes (again we omit cases of paranoid company security policies which require monitoring of all HtTPS traffic of company employees).
Addition: also read my answer on the similar topic here.
There are no pros or cons.
And there are no "HTTPS proxy" server.
You can tell the protocol handlers which proxy server to use for different protocols. This can be done for http, https, ftp and socks. Not more and not less.
I can't tell you if you should use a different proxy for https connections or not. It depends.
I can only explain the difference of an http and https request to a proxy.
Since the HTTP Proxy (or web proxy) understands HTTP (hence the name), the client can just send the request to the proxy server instead of the actual destenation.
This does not work for HTTPS.
This is because the proxy can't make the TLS handshake, which happens at first.
Therefore the client must send a CONNECT request to the proxy.
The proxy establishes a TCP connection and just sends the packages forth and back without touching them.
So the TLS handshake happens between the client and destenation.
The HTTP proxy server does not see everything and does not validate destenation servers certificate whatsoever.
There can be some confusion with this whole http, https, proxy thing.
It is possible to connect to a HTTP proxy with https.
In this case, the communication between the client and the proxy is encrypted.
There are also so called TLS terminating or interception proxy servers like Squid's SSL Peek and Splice or burp, which see everything.
But this should not work out of the box, because the proxy uses own certificates which are not signed by trusted CAs.
References
https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
https://parsiya.net/blog/2016-07-28-thick-client-proxying---part-6-how-https-proxies-work/
http://dev.chromium.org/developers/design-documents/secure-web-proxy
https://www.rfc-editor.org/rfc/rfc2817#section-5
https://www.rfc-editor.org/rfc/rfc7231#section-4.3.6
If you mean connecting to a HTTP proxy server over TLS by saying HTTPS proxy, then
I was wondering whether there are any advantages of using a HTTPS
proxy server compared to a HTTP proxy server ?
The advantage is that your client's connection to proxy server is encrypted. E.g. A firewall can't not see which host you use CONNECT method connect to.
Is accessing a https url via a HTTPS proxy less cumbersome than
accesing it from a HTTP proxy ?
Everything is the same except that with HTTPS proxy, brower to proxy server connection is encrypted.
But you need to deploy a certificate on your proxy server, like how a https website does, and use a pac file to configure the brower to enable Connecting to a proxy over SSL.
For more details and a practical example, check my question and answer here HTTPs proxy server only works in SwitchOmega
Unfortunately, "HTTPS proxy" has two distinct meanings:
A proxy that can forward HTTPS traffic to the destination. This proxy itself is using an HTTP protocol to set up the forwarding.
In case the browser is trying to connect to a website using HTTPS, the browser will send a CONNECT request to the proxy, and the proxy will set up a TCP connection with the website and mirror all TCP traffic sent on the connection from the browser to the proxy onto the connection between the proxy and the website, and similarly mirror the response TCP packet payload from the webite to the connection with the browser. Hypothetically, the same mechanism using CONNECT could be used with HTTP traffic, but practically speaking browsers don't do that. For HTTP traffic, they send the actual HTTP request to the proxy, including the full path in the HTTP command (as well as setting the Host header): https://stackoverflow.com/a/38259076/10026
So, by this definition, HTTPS Proxy is a proxy that understands the CONNECT directive and can support HTTPS traffic going between the browser and the website.
A proxy that uses HTTPS protocol to secure client communication.
In this mode (sometimes referred to as "Secure Proxy"), the browser uses the proxy's own certificate to perform TLS handshake with the proxy, and then sends either HTTP or HTTPS traffic, (including CONNECT requests), on that connection as per (1). So, the connection between the browser and the proxy is always protected with a TLS key derived using the proxy's certificate, regardless of whether the traffic itself is encrypted with a key negotiated between the browser and the website. If HTTPS traffic is proxied via a secure proxy, it is double-encrypted on the connection between the browser and the proxy.
For example, the Proxy Switcher Chrome plugin has two separate settings to control each of these funtionalities:
As of 2022, the option to use a secure proxy is not available in MacOS and Windows manual proxy configuration UI. But a secure proxy may be specified in a PAC file used in automatic proxy configuration using the HTTPS proxy directive. It is up to the consuming application to support the HTTPS directive; most major browsers, except Safari, and many desktop apps support it.
NOTE: Things get a bit more complicated because some proxies that proxy HTTPS traffic don't simply forward TCP packet payload, as described in (1), but act as Intercepting Proxies. Using a spoofed website certificate, they effectively perform a Man-in-the-Middle attack (well, it's not necessarily an attack because it's expected behavior). Whereas the browser thinks it's using the website's certificate to set up a TLS tunnel with a website, it's actually using a spoofed certificate to set up TLS tunnel with the proxy, and the proxy sets up the TLS tunnel with the website. Then proxy has visibility into the HTTPS requests/responses. But all of that is completely orthogonal to whether the proxy is acting as a secure proxy as per (2).