Is there a way to detect if someone has charles proxy or a network tracking app, tracking your program's requests/apis. And if so is there a way to prevent such a thing.
Just wanted a bit of insight into this, any information is much appreciated.
It's possible to achieve it: Here are two solutions I could think of
After the SSL finish, you can get the certificate and check whether or not it's a self-signed certificate. Then, you might reject the Request connection.
Use the SSL-Pinning technique that only accepts the certificate from your own and rejects others. It will prevent the app is intercepted from Charles Proxy or other Proxy tools like Proxyman, Fiddler, and Wireshark.
Related
I've some discussion with my colleague whether we can make HTTPS connection without using SSL certificate. I'm quite confused about this. Can anyone explain this?
It's not possible to do HTTPS connections without SSL. The way it works is complicated and as mentioned in the comments there is a great answer here.
But to summarize the reason why it's not possible:
HTTPS needs an SSL certificate, you can think of SSL as the S in HTTPS, since the secure connection is made using the certificate. A client sends a request to a server, they do a secure handshake (which requires the certificate in order to create and verify the keys) and then proceed to business as usual.
Any browser will throw a million red flaws if you try to circumvent this behavior. You could trick your own server to believe data is coming as HTTPS rather than HTTP in some cases (if you have control over it), but you won't be able to fool browsers.
I'm implementing an HTTP proxy. Sometimes when a browser makes a request via my proxy, I get an error such as ECONNRESET, Address not found, and the like. These indicate errors below the HTTP level. I'm not talking about bugs in my program -- but how other servers behave when I send them an HTTP request.
Some servers might simply not exist, others close the socket, and still others not answer at all.
What is the best way to report these errors to the caller? Is there a standard method that, if I use it, browsers will convert my HTTP message to an appropriate error message? (i.e. they get a reply from the proxy that tells them ECONNRESET, and they act as though they received the ECONNRESET themselves).
If not, how should it be handled?
Motivations
I really want my proxy to be totally transparent and for the browser or other client to work exactly as if it wasn't connected to it, so I want to replicate the organic behavior of errors such as ECONNRESET instead of sending an HTTP message with an error code, which would be totally different behavior.
I kind of thought that was the intention when writing an HTTP proxy.
There are several things to keep in mind.
Firstly, if the client is configured to use the proxy (which actually I'd recommend) then fundamentally it will behave differently than if it were directly connecting out over the Internet. This is mostly invisible to the user, but affects things like:
FTP URLs
some caching differences
authentication to the proxy if required
reporting of connection errors etc <= your question.
In the case of reporting errors, a browser will show a connectivity error if it can't connect to the proxy, or open a tunnel via the proxy, but for upstream errors, the proxy will be providing a page (depending on the error, e.g. if a response has already been sent the proxy can't do much but close the connection). This page won't look anything like your browser page would.
If the browser is NOT configured to use a proxy, then you would need to divert or intercept the connection to the proxy. This can cause problems if you decide you want to authenticate your users against the proxy (to identify them / implement user-specific rules etc).
Secondly HTTPS can be a real pain in the neck. This problem is growing as more and more sites move to HTTPS only. There are several issues:
browsers configured to use a proxy, for HTTPS URLS will firstly open a tunnel via the proxy using the CONNECT method. If your proxy wants to prevent this then any information it provides in the block response is ignored by the browser, and instead you get the generic browser connectivity error page.
if you want to provide any other benefits one normally wishes from a proxy (e.g. caching / scanning etc) you need to implement a MitM (Man-in-the-middle) and spoof server SSL certificates etc. In fact you need to do this if you just want to send back a block-page to deny things.
There is a way a browser can act a bit more like it was directly connected via a proxy, and that's using SOCKS. SOCKS has a way to return an error code if there's an upstream connection error. It's not the actual socket error code however.
These are all reasons why we wrote the WinGate Internet Client, which is a LSP-based product for our product WinGate. Client applications then learn the actual upstream error codes etc.
It's not a favoured approach nowadays though, as it requires installation of software on the client computer.
I wouldn't provide them too much info. Report what you need through internal logs in case you have to solve the problem. Return a 400, 403 or 418. Why? Perhaps the're just hacking.
Is there a way to completely encrypt the outgoing HTTPS requests from the software to the server ? i mean there are apps like Charles and Fiddler that can capture the HTTPS traffic and see everything like the Headers,URL,...
i don not want anyone to see or capture the traffic going from my app.
i'm using Delphi 10.1 VCL App
If you're using HTTPS and you are properly validating certificates as a browser would, there is no way for an intermediate to view URLs or headers, or content. All they can see is which server you're communicating with.
The way Charles gets around this is that it presents its own non-genuine certificate, which won't validate, and proxies the communication. If your app is validating certificates it would refuse to communicate with the Charles proxy. If you viewed the Charles proxy with a web browser it would present an SSL certificate error.
If you trust that particular Charles proxy and want to add an exception in your client or browser, you can. But it only allows that particular one - it doesn't mean anyone else can intercept your HTTPS, or read URLs, etc, using their own Charles proxy or similar.
we need to protect our webservices with SSL (https) or any other security mechanism. Our problem is that current clients (delphi exe's) have references to our http webservices fixed in code and can not change that code.
I've tried to implement URL redirection rule from http to https but that didn't work because of the "hand shake"...Changing client to use https reference did work but saddly we can not do that for every client.
I know this question is in contradiction with encription theories but i'll fire this question anyway if anyone has any type of suggestion/idea to at least make connection or data transfer more secured (either with or without SSL protocol) without changing client side.
Thanks,
Luke
You need some kind of transparent TCP tunneling software/hardware on the clients, so the encryption occurs without the delphi clients noticing it.
My Google search using "transparent encrypted tunneling" keywords got this vendor of such solutions. There's must other vendors with similar solutions.
This is really an networking question.
PS.: hardcoding the URL is the real problem here. After the tunneling palliative is done, change that because this really will cause more headaches in future.
The client will be connecting over a port (non SSL) that will need to remain. What you could possibly do is that if you allow access both http and https you could possibly only allow http from specific IP addresses if you know them? its still not secure, but least you know where the calls are coming from and can do something about that?
I've got a tool which I need to use with a https site, but it only supports http. I need some kind of proxy tool that can work as a man in the middle between my tool and the https site, making the requests seem like http to the tool. The connection would look like this:
Tool -> HTTP -> Proxy -> HTTPS -> Site -> HTTPS -> Proxy -> HTTP -> Tool
Does anyone know of a tool or something I can set up that will work for this purpose? I've already asked in a few IRC channels and searched Google, but I was honestly not sure how to word it when searching Google, and not sure which channel to ask in via IRC.
Any help would be greatly appreciated!
Fiddler (www.fiddler2.com) has this capability; you can change the URL from HTTP to HTTPS on the fly.
Try cUrl. If your tool works with stdin/out, You can pipe your tool's stdin/out through curl. Also there is the library (libCurl) that you can use from within your code, if needed.
WebScarab might do it for you. It can act as a man-in-the-middle and you can alter requests on the fly.
Got any objection to an entire copy of Apache httpd with mod_proxy operating as a reverse proxy? Plausible instructions here.
"Stunnel can allow you to secure non-SSL aware daemons and protocols (like POP, IMAP, LDAP, etc) by having Stunnel provide the encryption, requiring no changes to the daemon's code."
There is also a guide on their site to wrap HTTP requests with Stunnel.
Looks like this SSL MITM proxy can do exactly what you're asking.