Suppose I am logged in and connected to a website in firefox (or any other browser) now I can make download requests in the browser. Suppose I want to use wget or curl using the connection of firefox. Is there a way to use firefox as a systemwide proxy for port 443 and 80? Here is a usage scenario: This would be interesting for a download manager, if the requests are proxied and made by the browser, all the credentials stored in the browser could be used.
So the browser would receive the request on port 443 and replicate it or forward it. Proxy and forwarding are probably not the right words in this context.
I am not aware of any feature of Firefox (or any other mainstream browser) that allows to really use it as some kind of proxy, sorry.
You cannot somehow "use the connection firefox already has", since there is no permanent connection between client and server in an http communication. http is a stateless protocol without some socket permanently kept open. Instead each http request is sent separately, each time a new socket is opened.
However something similar might be "half possible" using a crude workaround:
What you can try however is to simply start a new instance of the browser for each request you want to do. In reality this does not start a new instance, but reuses an already existing instance and typically opens a new tab in there. That way you can "remote control" your already started browser in a primitive way and do downloads, if and only if the url you specify will result in a download. However that all depends on the browser settings, so for example downloads will be stored in files in your local file system where you have to read the payload from again.
This all is not really efficient and convenient which is why it probably does not make much sense. Instead you should create a simple script for such communication. The effort for that is not that high.
Related
I'm implementing an HTTP proxy. Sometimes when a browser makes a request via my proxy, I get an error such as ECONNRESET, Address not found, and the like. These indicate errors below the HTTP level. I'm not talking about bugs in my program -- but how other servers behave when I send them an HTTP request.
Some servers might simply not exist, others close the socket, and still others not answer at all.
What is the best way to report these errors to the caller? Is there a standard method that, if I use it, browsers will convert my HTTP message to an appropriate error message? (i.e. they get a reply from the proxy that tells them ECONNRESET, and they act as though they received the ECONNRESET themselves).
If not, how should it be handled?
Motivations
I really want my proxy to be totally transparent and for the browser or other client to work exactly as if it wasn't connected to it, so I want to replicate the organic behavior of errors such as ECONNRESET instead of sending an HTTP message with an error code, which would be totally different behavior.
I kind of thought that was the intention when writing an HTTP proxy.
There are several things to keep in mind.
Firstly, if the client is configured to use the proxy (which actually I'd recommend) then fundamentally it will behave differently than if it were directly connecting out over the Internet. This is mostly invisible to the user, but affects things like:
FTP URLs
some caching differences
authentication to the proxy if required
reporting of connection errors etc <= your question.
In the case of reporting errors, a browser will show a connectivity error if it can't connect to the proxy, or open a tunnel via the proxy, but for upstream errors, the proxy will be providing a page (depending on the error, e.g. if a response has already been sent the proxy can't do much but close the connection). This page won't look anything like your browser page would.
If the browser is NOT configured to use a proxy, then you would need to divert or intercept the connection to the proxy. This can cause problems if you decide you want to authenticate your users against the proxy (to identify them / implement user-specific rules etc).
Secondly HTTPS can be a real pain in the neck. This problem is growing as more and more sites move to HTTPS only. There are several issues:
browsers configured to use a proxy, for HTTPS URLS will firstly open a tunnel via the proxy using the CONNECT method. If your proxy wants to prevent this then any information it provides in the block response is ignored by the browser, and instead you get the generic browser connectivity error page.
if you want to provide any other benefits one normally wishes from a proxy (e.g. caching / scanning etc) you need to implement a MitM (Man-in-the-middle) and spoof server SSL certificates etc. In fact you need to do this if you just want to send back a block-page to deny things.
There is a way a browser can act a bit more like it was directly connected via a proxy, and that's using SOCKS. SOCKS has a way to return an error code if there's an upstream connection error. It's not the actual socket error code however.
These are all reasons why we wrote the WinGate Internet Client, which is a LSP-based product for our product WinGate. Client applications then learn the actual upstream error codes etc.
It's not a favoured approach nowadays though, as it requires installation of software on the client computer.
I wouldn't provide them too much info. Report what you need through internal logs in case you have to solve the problem. Return a 400, 403 or 418. Why? Perhaps the're just hacking.
I was playing around with fiddler (http proxy) and I noticed that some apps are making http get/post requests in the background and sending data and stats to and from the web. This got me interested and a little concerned to see what data various apps were sending but it seems that most of them are not doing it on port 80 via http but presumably on another port so you can't see the data in fiddler. Is there some way to view and/or potentially block the data being sent?
You're asking: "Using Fiddler, I saw that traffic was being sent by clients to servers. How can I see that traffic?"
Might I suggest you use Fiddler?
You can see the process sending the traffic in the Process column, and you can view the contents of the requests and responses using the Inspectors tab.
I would check out burp suite. It is a proxy you set up in your web browser and shows all of the data that passes through it. There's plenty of tutorials online. Check it out here
I need to build a Web site which would have a secure connection (HTTPS) on some pages. I need to know if there will be a difference for me (as a developer) while I will write the code? I must treat differently some data or what? What is the main difference from back-end view?
From the backend point of view, there is no difference. The difference between the two is the TCP connection between the server and the client. Https will be encrypted, http is not of course, but it's all decrypted by the time it hits your code. The server will have some flags available so you can determine whether the connection is http or https (names vary depending on the server) but unless you're using that information to change the behavior of the page, you don't need to worry about it.
In a developement environement (where often the browser and the http server are on the same machine) i want to study the exact detail of authentication schemas. So i need to trace down every http request/response.
I've tried WireShark, that is very promising. But actually on
windows machines there is a problem in sniffing the traffic on
loopback interface.
Then i've tried a browser plugin, HttpFox
0.8.10 of Firefox 12. It is good in showing requests and responses, but in the specific case of authentication, it doesn't correctly
show the "double hop" authentication, it "collapses" the first
request (the Unauthorized status code) with the next, successful
one.
Then i've tried to work with the logs of httpd, that is my
actual server, but it is required a not trivial effort to create a
log that contains all the request such as headers (the authorization
header).So it doesn't seem a good "debug" technique.
Are there other possibilities?
Go with Wireshark. The answer to this question will address the loopback issue. Wireshark is the best because it really understands the formatting of everything related to HTTP (so long as you are not using HTTPS).
I'm new to low-level HTTP stuff and am not sure what to make of what I am seeing.
If I go to a particular internet web server (let's call it www.someserver.com for now... I'll give the real one if it's really needed), Firefox happily pulls up its home page. If, however, I do a
telnet www.someserver.com 80
GET / HTTP/1.0
...what I get returned appears to be the Apache default "It works" page. Trying to GET another page on the server that Firefox will happily pull up receives a 404 from telnet. It's like they're hitting different servers, but these requests are both coming from the same machine, so I'm not sure how.
What could cause such behavior?
It could be serving different sites based on the host header sent by the browser. Your telnet connection wouldn't send that header unless you explicitly typed it.
http://support.microsoft.com/kb/308163