As far as i know, squid can be as proxy forward proxy. for example
docker run -d -p 3128:3128 --name squid0 wernight/squid
curl --proxy http://localhost:3128/ example.com
Above example is for http, i have used google to search, didn't find any tcp forward proxy software can implement above similar function.
Existing any tcp forward proxy can implement similar function?
Related
I have an application that needs to use a proxy (call it proxy1) to access some https endpoints outside of its network. The application doesn't support proxy settings, so I'd like to provide it a reverse proxy url, and I would prefer not to provide tls certs for proxy1, so I would use http for application -> proxy1.
I don't have access to the application host or forward proxy mentioned below, so I cannot configure networking there.
The endpoints the application needs are https, so proxy1 must make its outbound connections via https.
Finally, this whole setup is within a corporate network that requires a forward proxy (call it proxy2) for outbound internet, so my proxy1 needs to chain to proxy2 / use it as a parent.
I tried squid and it worked well for http only, but I couldn't get it to accept http inbound while using https outbound. Squid easily supported the parent proxy2.
I tried haproxy, but had the same result as with squid.
I tried nginx and it did what I wanted with http -> proxy -> https, but doesn't support a parent proxy. I considered setting up socat as in this answer, or using proxy_pass and proxy_set_header as in this answer, but I can't shake the feeling there's a cleaner way to achieve the requirements.
This doesn't seem like an outlandish setup, is it? Or is there a preferred approach for it? Ideally one using squid or nginx.
You can achive this without the complexity by using a port forwarder like socat. Just install it on a host to do the forwarding (or locally on the app server if you wish to) and create a listener that forwards connections through the proxy server. Then on your application host use a local name resolution overide to map the FQDN to the forwarder.
So, the final config should be the app server using a URI that points to the forwarding server (using its address if no name resolution excists), which has a socat listener that points to the the corporate proxy. No reverse proxy required.
socat TCP4-LISTEN:443,reuseaddr,fork \
PROXY:{proxy_address}:{endpoint_fqdn}:443,proxyport={proxy_port}
Just update with your parameters.
I'm trying to set up a proxy_pass while also using a socks5 proxy. I can access my backing service with curl using the following:
curl -x socks5h://localhost:8001 -svo /dev/null -I http://[abcd:1234::]:8000
So what I've currently got in my nginx config which doesn't work is:
location / {
proxy_pass http://[abcd:1234::]:8000
proxy_redirect http://localhost:8001 /;
}
It also seems like nginx has no notion of ALL_PROXY|HTTP(S)_PROXY environment variables which other applications can use.
Any idea how I can get this to work?
I did find a related question - socks5 proxy/tunnel for nginx upstream? but it's now 6 years old and I'm not sure works still.
Why it does not work?
To my knowledge, proxy_pass, proxy_redirect and other functionality in the ngx_http_proxy_module is meant to act as a HTTP/HTTPS proxy only. This seems to be confirmed by the 'As a protocol, “http” or “https” can be specified.' note in proxy_pass documentation (no mention of SOCKS).
The proxy_pass directive allows you to tell NGINX to take whatever requests it receives at specific location and blindly send them to another HTTP server, wait for the response from said server and return the response to the client. Other directives from the module (for example proxy_redirect) allow slight modifications to requests/responses. What is important is that the entire process is very simple and there is no tunneling (aside from TLS when location is https) or wrapping in additional protocols.
In contrast, SOCKS proxies require implementation of the SOCKS proxy protocol and using it to wrap all the connections. This additional work cannot be performed using the ngx_http_proxy_module.
How to make it work?
Unfortunately, using SOCKS proxies in NGINX does not seem to be supported by any of the core modules (listed here below 'Module reference'). It also does not seem to be a popular use case, so i would not expect support for it in NGINX core anytime soon. In another question you linked one of the answers references a third party nginx module which is also listed on the nginx.com website (the list has no anchors, so CTRL + F for "SOCKS" and you will find it). Last commit is from 2016 but it is possible it will still work.
If you can't change the way you access your backend service, i would say your best bet is either using the module mentioned above and trying to fix it if it does not work or writing your own module. Alternatively, maybe you could establish port forwarding to the backend service over the SOCKS proxy, and just proxy_pass to your local port. If you have an ssh server running on your backend service host, you could set up a simple proof-of-concept like this:
ssh <YOUR-SSH-LOGIN>#<BACKEND-HOST> \
-L 8081:localhost:80 \
-o "ProxyCommand=nc -X 5 -x <YOUR-SOCKS-PROXY-IP>:<YOUR-SOCKS-PROXY-PORT> %h %p"
The -L argument creates port forwarding between your local 8081 port and port 80 (http) on backend host. The -o argument adds a ProxyCommand option which uses netcat to forward traffic over a SOCKS proxy (not all netcat versions support the -X and -x arguments, the one i am using is openbsd-netcat on Arch Linux). After using that you should be able to just proxy_pass to localhost:8081 in NGINX. This setup is not very performant and serves only as a proof-of-concept, if you decide to go this way you should find another method of forwarding ports over the proxy.
Finally, in my personal opinion, if you can you should change the way you access your backend service. If you were the one to set up the connection then SOCKS proxy is an overkill when all you want to do is to connect to few hosts. If it is a proxy put in place by your company or someone else above you then i would discuss it with the network administrators.
I want to print the dynamic port created by Nginx in reverse proxy mode, so as to capture the specific packet.
I have go through the nginx docs, but not find something useful. by the way, any other alternatives are welcome
I am using squid for the very first time , actually i am forwarding from privoxy to squid and then to server.
I could able to see privoxy logs when i hit a request not the Squid log.
I have checked in access.log but no luck.
Can anyone help me with this.
Unless you have configured Squid otherwise, all requests (whether successful or not) will be written to the access.log file after completion. In the case of a successful request, this happens almost immediately, but an unsuccessful request might take up to 30 seconds to appear in the log. Are you waiting long enough for this to happen before checking the file?
Assuming you're running on Linux, perhaps you could send some requests through the proxy using a tool like wget, then check the log. This will confirm that logging is actually working. Use this syntax:
http_proxy=http://localhost:3128/ wget www.google.com --debug
Prepending the "http_proxy=" at the start of the command tells wget to use a proxy listening at localhost on port 3128. What does wget show you? Does Squid write anything to the access.log file? If so, that would seem to indicate your proxy is logging fine, but privoxy isn't sending anything to it.
Another thing you could try is to run a packet capture on the TCP port you expect privoxy to send traffic to Squid over. I don't know how you configure privoxy, but I'm assuming the sequence goes something like this:
Client > Privoxy > Squid > Server
In that case, lets say privoxy connects to Squid on localhost over TCP port 3128. You could run a packet capture to see if privoxy is even connecting to Squid, like this:
tcpdump -i localhost port 3128 -vv
Or, if you see packets being transferred but want to know what's inside them, then the excellent tcpflow is your friend:
tcpflow -c -i localhost port 3128
If you edit your question to provide more context, it might be possible to provide a better answer.
The following configuration i have added and i could see the logs now.
access_log /var/log/squid/access.log squid
access_log syslog:daemon.debug squid
I need to send (duplicate) traffic from one machine (port) and to two different machines (ports). I need to take care of TCP session as well.
In the beginnig I used em-proxy, but it seems to me that the overhead is quite large (it goes over 50% of cpu).
Then I installed haproxy and I managed to redirect traffic (not to duplicate). The overhead is reasonable (less than 5%).
The problem is that I could not say in haproxy config file the following:
- listen on specific address:port and whatever you find send on the two different
machines:ports and discard the answers from one of them.
Em-proxy code for this is quite simple, but it seems to me that EventMachine generates
a lot of overhead.
Before I dig in haproxy code and try to change (duplicate traffic) I would like
to know is there something similar out there?
Thanks.
I have created a proxy just for this purpose.
https://github.com/chrislusf/teeproxy
Usage
./teeProxy -l :8888 -a localhost:9000 -b localhost:9001
tee-proxy is a reverse proxy. For each incoming request, it clones the request into 2 and then forwards them to 2 servers. The results from server a is returned as usual, but the results from server b is ignored.
tee-proxy handles both GET, POST, and other HTTP methods.
How about the iptables experimental ROUTE target? It has a "tee" option for mirroring traffic:
http://www.netfilter.org/projects/patch-o-matic/pom-external.html#pom-external-ROUTE
Which would let you mirror traffic with something like:
iptables -A PREROUTING -t mangle -p tcp --dport 80 -j ROUTE --gw 1.2.3.4 --tee
iptables -A POSTROUTING -t mangle -p tcp --sport 80 -j ROUTE --gw 1.2.3.4 --tee
The second machine would need to be on the same subnet and would either need to listen on the target IP address (and not reply to arps) or listen promiscuously.
Try https://github.com/agnoster/duplicator.
I tried teeproxy but got strange results with some requests other than GET's.
I have also written a reverse proxy / load balancer for a similar purpose with Node.js (it is just for fun, not production ready at the moment).
https://github.com/losnir/ampel
It is very opinionated, and currently supports:
GET Using round-robin selection (1:1)
POST Using request splitting. There is no concept of "master" and "shadow" -- the first backend that responds is the one that will serve the client request, and then all of the other responses will be discarded.
If someone finds it useful then I can improve it to be more flexible.
I needed something that could tee the TCP traffic as well, but being not intrusive, thus not being able to put something in-between as a reverse proxy for example.
What I did is basically did is use the tcpdump/wireshark logic (packet sniffing) wrap it in a Go process that you can configure to do some things.
For whom it may be helpful the code can be found here: https://github.com/RobinUS2/teecp