I've got Fiddler to decrypt https traffic. which works just fine.
http://fiddler2.com/documentation/Configure-Fiddler/Tasks/DecryptHTTPS
But there are some requests that are shown as "Tunnel to" port 443. Also the protocol of these requests are shown as HTTP instead of HTTPS. I do not see the decrypted content of these requests.
on the response details pane, it's shown as below:
Can someone let me know how I can get fiddler to decrypt these tunnelled requests too. Also appreciate if you could provide me some details about what's going on in this case.
Cheers
As noted in the inspector description the line Tunnel to example.org:443 is the first line of a successful captured HTTPS call.
It is notes as HTTP because at that time the HTTPS channel has not been established yet.
If afterwards no further HTTPS requests to that server are shown this means that the program performing the request has not accepted the presented server certificate generated by Fiddler and therefore the HTTPS requests failed.
There are two possible explanations:
The program performing the requests does not trust the Fiddle root CA certificate.
For programs that don't use the Windows certificate store (e.g. Firefox) you have to export this root ca certificate from Fiddler and install it manually into the program's certificate trusted store (list of trusted root-CA certificates).
A second possibility is that the site is certificate pinned. For web browsers this can happen is the web site uses the HSTS configuration.
Related
I wrote a proxy server which works well. But when looking at the log, there are some weird requests like:
POST https://vortex.data.microsoft.com/collect/v1 HTTP/1.1
Also some GET over https. I think only CONNECT is allowed over https, am I wrong? If I am wrong, how to deal with these request? (I just dropped these requests in my app.)
Another thing maybe unrelated is all these requests are related to microsoft from the log.
There isn't any problem handling any HTTP Method with HTTPS within a proxy.
All the requests with https://-protocol will be automatically received and sent to port 443 if not indicated otherwise.
Independently if you have a server where you deployed a HAProxy, NGINX, Apache Web Server or that you literally wrote a proxy like this one in JavaScript, only thing you have to do is to literally proxy the requests to the destination server address.
Regarding the encryption, precisely HTTPS ensures that there are no eavesdroppers between the client and the actual target, so the Proxy would act as initial target and then this would transparently intercept the connection.
Client starts HTTPS session to Proxy
Proxy intercepts and returns its certificate, signed by a CA trusted by the client.
Proxy starts HTTPS session to Target
Target returns its certificate, signed by a CA trusted by the Proxy.
Proxy streams content, decrypt and re-encrypt with its certificate.
Basically it's a concatenation of two HTTPS sessions, one between the client and the proxy and other between the proxy and the final destination.
I understand that HTTTPS is secured and it requires SSL certificate issued by CA authority to make the application secure. But what I do not understand is that its in-depth difference with HTTP.
My question, as a user, if I make a request to an application with HTTP or if I make same request to HTTPS what is the actual difference? The traffic remains same to both. Is there any traffic filtering happening if I use HTTPS?
Thanks
HTTPS, as an application protocol is just HTTP over TLS, so there are very few differences, the s in the URL and some consequences for proxy, that is all.
Now you are speaking about the traffic and the filtering. Here you have a big difference because using TLS adds confidentiality and integrity: passive listeners will see nothing about the HTTP data exchanged, including headers. The only thing visible will be the hostname (taken from the https:// URL) as this is needed at the TLS level before HTTP even happens, through a mechanism called SNI (Server Name Indication) that is now used everywhere to be able to install multiple services using TLS under different names but with a single IP address.
If a client is requesting a domain that does not have a valid CA signed certificate and the server not intend on supporting HTTPS but does support HTTP for this domain, what is the best way to handle this in the web server. Note, the server does handle requests for SSL (HTTPS) on other domains so it is listening on 443.
Example where this would apply is for multi sub-domains where the sub-domains are dynamically created and thus making it extremely difficult to register CA signed certificates.
I've seen people try to respond with HTTP error codes but these seem moot as the client (browser) will first verify the certificate and will present the hard warning to the user before processing any HTTP. Therefore the client will only see the error code if they "proceed" past the cert warning.
Is there a canonical way of handling this scenario?
There is no canonical way for this scenario. Clients don't automatically downgrade to HTTP if HTTPS is broken and it would be a very bad idea to change clients in this regard - all what an attacker would need to do to attack HTTPS would be to infer with the HTTPS traffic to make a client downgrade to unprotected HTTP traffic.
Thus, you need to make sure that the client either does not try to attempt to access URL's which do not work properly (i.e. don't publish such URL's) or to make sure that you have a working certificate for these subdomains, i.e. adapt the processes for creation of subdomains so that they not only have an IP address but also a valid certificate (maybe use wildcard certificates).
Considering these websites don't have to work with SSL, the webserver should close all SSL connections for them in a proper way.
There is no canonical way for this, but RFC 5246 implicitly suggests to interrupt the handshake on the server side by using the user_cancel + close_notify alerts. How to achieve this is another question, it will be a configuration of the default SSL virtual host.
user_canceled
This handshake is being canceled for some reason unrelated to a
protocol failure. If the user cancels an operation after the
handshake is complete, just closing the connection by sending a
close_notify is more appropriate. This alert should be followed
by a close_notify. This message is generally a warning.
If you are dealing with subdomains, you probably can use a wildcard certificate for all of your subdomains.
Adding the CA certificate to your client will remove the warning (that's what companies do, no worry).
When hosting with Apache, for example, you can use VirtualDocumentRoot to add domains without editing your configuration. Have a look at the solution provided here : Virtual Hosting in SSL with VirtualDocumentRoot
Is there a way to completely encrypt the outgoing HTTPS requests from the software to the server ? i mean there are apps like Charles and Fiddler that can capture the HTTPS traffic and see everything like the Headers,URL,...
i don not want anyone to see or capture the traffic going from my app.
i'm using Delphi 10.1 VCL App
If you're using HTTPS and you are properly validating certificates as a browser would, there is no way for an intermediate to view URLs or headers, or content. All they can see is which server you're communicating with.
The way Charles gets around this is that it presents its own non-genuine certificate, which won't validate, and proxies the communication. If your app is validating certificates it would refuse to communicate with the Charles proxy. If you viewed the Charles proxy with a web browser it would present an SSL certificate error.
If you trust that particular Charles proxy and want to add an exception in your client or browser, you can. But it only allows that particular one - it doesn't mean anyone else can intercept your HTTPS, or read URLs, etc, using their own Charles proxy or similar.
The question is about HTTP vs HTTPS.
If I want to anonymously load a website that forces HTTPS, like Google.com, do I need an HTTPS proxies, or can I get away with HTTP proxies?
If your proxy is SOCKS it will not care what kind of socket is connecting through it. It has its own handshake and it does not care about what happens after the handshake. Whether after the SOCKS handshake an SSL handshake (HTTPS) is started it is not a SOCKS proxy problem, it will just pass through.
Several HTTP proxies on the other hand expect HTTP headers to guide them, such a HTTP proxy will not allow HTTPS since it needs to read the headers.
On the third hand (ekhm... well, foot?), an HTTP proxy that supports HTTP CONNECT can also setup the transfer of arbitrary data. Therefore such a proxy can setup any type of socket, which can have an SSL handshake, which can then be used for HTTPS transfer.
HTTP Proxy Server supports CONNECT verb which supports HTTPS connections within HTTP Proxy. You don't need special HTTPS proxy server or any other setup.
CONNECT verb allows you to create binary socket tunnel to any given IP:Port address. So any HTTP client (all browsers), will open secure tunnel and communicate securely over proxy server. However, no one cant control or see anything that is going through the tunnel unless they implement man in middle attack by sending you self-signed certificates.
Most firewall these days automatically implement man in middle self signed certificates that are deployed in work network, so you have to probably dig more to identify whether it is really secure or not. So it may not be that anonymous.
If you're trying to access a service anonymously, you won't get this by running your own proxy. It's not clear from the original question what is meant by "proxy", e.g. local service, or remote service. You won't get anonymity by surfing through a proxy that's on your network, unless it's something like a TOR proxy which relays out through the TOR network.
As for whether proxies can support HTTPS or not, that's been covered here, it would be unusual to find a proxy that doesn't support CONNECT. However if it's a remote anonymizing service you're using, I doubt they would do MitM, since you'd need to install the signing cert into your trusted root store, so they couldn't do that surreptitiously.