I've recently been trying to self-host a HTTP proxy server for OpSec reasons. I want to make the proxy in a fashion that I have a singular proxy I can connect to, and upon each request it selects a different proxy from a 50k proxy list I will load manually. Is there any way I can do this, I'm willing to use any application or tool that allows me to do so.
Thank you!
Related
I have an application set up like this:
There is a server, with a reverseproxy/load balancer that acts as the HTTPS termination (this is the one that has a server certificate), and several applications behind it(*)
However, some applications require authentication of the client with a certificate. Authentication cannot happen in the reverse proxy. Will the application be able to see the user certificate, or will it be jettisoned by the HTTPS->HTTP transfer?
(*) OK, so this is a Kubernetes ingress, and containers/pods.
It will be lost. I think you need to extract it in the reverse proxy (i.e. Nginx) and pass it in as a HTTP header if you really must. See for example https://serverfault.com/questions/788895/nginx-reverse-proxy-pass-through-client-certificate. Not very secure as the cert is passed in the clear!
I don't know if we have that level of control over the ingress, personally I'm using a normal Nginx server for incoming traffic instead.
I have server with nginx and one working app. I want to add several apps to this servers. I would like to assimilate a few things for myself.
What is the difference between load balancer and reverse proxy?
In which situations should I use the first, and in which situations should I use the second?
What should I use if my sites are static, and what if not static?
And additionally it would be a big plus to hear about containers in the context of several sites for nginx
Differences between load balancer and reverse proxy
A reverse proxy accepts a request from a client, forwards it to a server that can fulfill it, and returns the server’s response to the client.
A load balancer distributes incoming client requests among a group of servers, in each case returning the response from the selected server to the appropriate client.
Taken from nginx docs
TL;DR :
Reverse proxying is about : routing requests to the correct server using the domain name
Load balancing is about : distributing load to multiple instances
What should I use if my sites are static, and what if not static?
You can combine an HTTP reverse proxy + load balancing with both static and non static web apps, so it depends.
And additionally it would be a big plus to hear about containers in the context of several sites for nginx
I recommend one nginx container per app / site + a dynamic reverse proxy, traefik in particular (http://traefik.io)
You need a reverse proxy to route the incoming traffic to the proper application taking into account the content of the original request (and rules that you may define).
When the target application(s) is determined, you will need to load balance them in order to distribute the amount of work across them.
Both tasks can be done by software like classic nginx, apache, haproxy, etc or by those that are designed for the microservices world, like fabio, traefik and others.
My application is based on Spring framework and is used to transmit very confidential data.During testing of the application with with proxy tools like Fiddler, Paros Proxy etc. it was found that these tools are intercepting the request data, and data is easily modified before reaching the server.
My application is not currently not integrated with SSL . We will implement SSL /HTTPS. but is this because SSL has not been integrated ?
Is it normal for Proxy tools to intercept the data from a web application with out HTTPS ?
There are two things here.
1) If you don't use SSL the communication is not encrypted, which means that anybody who is able to intercept the traffic, will be able to see the content. You don't necessarily need a proxy for that.
2) With an intercepting HTTP proxy you can see SSL encrypted traffic as well. What the proxy does is building two separate SSL tunnels one between the server and the proxy and one between the client and the proxy. This way the proxy itself can see the whole traffic. Of course the proxy can only provide a fake SSL certificate which will trigger a notification in the browser for the user, but he will probably ignore it.
Yes. If you don't use https, Proxy see everything that the application send or received.
To prevent that, you must use https.
To prevent sslstrip you must use HSTS.
These are two phrases I hear about very often, mainly associated with Nginx. Can someone give me a laymans defintion?
Definitions are often difficult to understand. I guess you just need some explanation for their use case.
A short explanation is: load balancing is one of the functionalities of reverse proxy, and reverse proxy is one of the softwares that can do load balancing.
And a long explanation is given below.
For example a service of your company has customers in UK and German. Because the policy is different for these two countries, your company has two web servers, uk.myservice.com for UK and de.myservice.com for German, each with different business logic. In addition, your company wants there to be only one unified endpoint, myservice.com for the service. In this case, you need to set up a reverse proxy as the unified endpoint. The proxy takes the url myservice.com, and rewrites the url of incoming requests so that requests from UK(determined by source ip) go to uk.myservice.com and requests from German go to de.myservice.com. From the view of a client from UK, it never knows the response is actually generated from uk.myservice.com.
In this case, the load of request traffic to the service is actually balanced to servers on uk.myservice.com and de.myservice.com as a side effect. So we normally don't call it used as a load balancer, just say it as a reverse proxy.
But lets say if your company uses the same policy for all countries, and has 2 servers, a.myservice.com and b.myservice.com, only for the reason that the work load is to heavy for one server machine. In this case, we normally call the reverse proxy as load balancer to emphasize the reason why it is being used.
Here is the basic definition:
Reverse Proxy is a proxy host, that receives requests from a client, and sends it to one of the servers behind itself. Nginx and apache httpd are commonly used as reverse proxies. These are in the administrative network of the web server that a servers a request.
This is in contrast with a (forward) Proxy, which sits in front of a client, and sends requests on behalf of a client to a web server. As an example, your corporate network address translator is a forward proxy. These are in the administrative network of the client from where the request originates.
Load balancing is a function performed by reverse proxies. The client requests are received by a load balancer, and the load balancer tries to send that request to one of the nodes (hosts) in the server pool, in an attempt to balance the load across various nodes.
I see both of them as a functionality of a HTTP/Web Server.
Load balancer’s job is to distribute the workload between servers node in a way that makes the best use of it.
Reverse proxy is a interface for external world ,forwarding request to a server node (even when we have a single node)
Its other use cases are caching of static content ,compression etc
I have a running HTTP web application and I am facing problems to make it run over HTTPS.
I am thinking of bringing some HTTPS Proxy that accepts user requests and forward it to the HTTP web app.
What do you think of that? and How can I accomplish that?
Setting up stunnel is a no-brainer - and its available for Unix/Linux/Posix/MSWindows (you might have mentioned what OS you are using).
(Also you can run the program to encrypt or decrpyt, at the server or at the client side)
It's possible to run Apache Httpd (for example) using HTTPS and use mod_proxy_http as a reverse proxy to forward the requests to your existing HTTP server. Of course, for this to be of any use, you'd need the reverse proxy and the target server to be connected in such a way that connections cannot be sniffed or altered.
You may find that the existing server needs certain extra settings for it to be aware it's using HTTPS (for example, special Valves in Apache Tomcat to set the HTTPS flag to true).
Apache httpd reverse-proxy?