I need to have 3 varnish instances, I follow this guide creating-varnish-cluster and now I have those systems
3 servers that have varnish, each varnish has vcl like that vcl-cluster
1 backend server, that contains application and Nginx for load balancing between varnish servers, Nginx config: nginx-config
but requests received in Nginx and passes through varnish but varnish server can't forward request to the backend. when I call directly backend it perfectly works.
how can I fix this issue? and is there any better solution for varnish scaling? (except varnish plus:D)
server.identity is the hostname of the machine, by default. I assume your machines are not called node1, node2 and node3, so you need to edit your VCL to reflect that.
Related
I have a Windows PC with some Flask webapps running on various ports 5000, 5001, ...
I have mapped subdomains with CNAMEs to point to the server and Nginx handles the rerouting (correct wording?) to the appropriate ports: x.server.net -> ip:5000, y.server.net -> ip:5001
It works
Now I want to enable https so I would like to redirect users from http to https.
Further, I discovered that Nginx doesn't handle when users supply a non-standard port, so x.server.net:5001 actually points to the wrong Flask app. So I would also like to redirect non-default ports to the default (80 or 443 depending on http or https). Some of the apps don't need https, so I might mix it.
Can this be done with Nginx or should I use something else? I found others asking this, but the replies are only for Linux as far as I understand (iptables?).
And last but not least, is redirects a safe approach? Can it be ignored by a malicious client?
I am thinking to install varnish cache on a small VPS configuration. My VPS configuration is 2 core CPU & 2 GB RAM with 50 GB SSD.
I would like to know on such small VPS configuration what should be best to install
Varnish Cache, Nginx Web server (PHP & MySQL also)
Nginx Cache (for static file caching) on top of Nginx webserver along with PHP & MySQL
Fyi I also need to install SSL on all virtual host. What I know by default Varnish doesn't work on SSL. There should be additional things has to be installed to support SSL. So if I have to install multiple service on server to support varnish cache does it not better to use the nginx only with nginx cache remembering the server specs are low.
I am waiting for an open discussion to find best solution. Thanks in advance.
Nginx is a great webserver, but isn't a specialized reverse caching proxy. Varnish is specifically built for caching and its caching policies are far more granular.
Components
Use Hitch as your TLS proxy
Use Varnish for caching
Use Nginx as your webserver and connect to Nginx from Varnish
See https://www.varnish-software.com/developers/tutorials/terminate-tls-varnish-hitch/ for a tutorial on how to install Hitch for TLS termination.
See https://www.varnish-software.com/developers/tutorials/#installations for a list of tutorials (per platform and distribution) on how to install Varnish.
Port configuration
Configure Nginx to listen on port 8080
Configure Varnish to listen on port 80
Add an additional listening interface in Varnish on port 8443 for PROXY protocol traffic
Configure Hitch to listen on port 443
Let Hitch connect to Varnish over port 8443 via the PROXY protocol
Let Varnish connect to Nginx over port 8080 via a backend definition
I have this install from this guide https://www.linode.com/docs/websites/varnish/use-varnish-and-nginx-to-serve-wordpress-over-ssl-and-http-on-debian-8
is there any advantage for using proxoy protocol? from
https://info.varnish-software.com/blog/five-steps-to-secure-varnish-with-hitch-and-lets-encrypt
to this setup (I have varnish 5)
if so, what is the modification needed for the setup in the linode link above?
best.
To begin with, it will not be possible to use PROXY protocol in the linked setup.
Nginx supports PROXY protocol only on the client side (i.e. when there is another proxy forwarding requests to it). It doesn't support PROXY protocol with proxy_pass, where it would make more sense for Varnish + Nginx SSL setup. Sorry about that.
I was looking at using HAProxy and Nginx for load balancing, and I had some questions:
Should I use JUST HAProxy over Nginx for the proxy server?
Is there any reason to have HAProxy and Nginx installed on the same proxy server?
Haproxy is a "load balancer" it doesn't know to serve files or dynamic content. nginx is a web server capable of many interesting things. if you only need to load balance + HA some third web server then haproxy is enough. if you need to implement some static content or some logic in routing of the requests before terminating them on a third server then you may need nginx.
The reason you can see haproxy+nginx on the same host is that it allows you to bring down single nginx instances while haproxy continues to serve requests from other hosts. Imagine having a RR DNS using A records:
myapp.com IN A 1.1.1.1
myapp.com IN A 1.1.1.2
Where 1.1.1.1 and 1.1.1.2 are two hosts with haproxy+nginx configured to load balance between them. Now for some reason your 1.1.1.1's nginx goes down. The browsers that come to 1.1.1.1 are still being served by haproxy on it which in turn gets data from 1.1.1.2's nginx.
HAProxy is definitely the better, more fully featured loadbalancer (compared to the free nginx, not nginx plus (but one could argue that as well).
One thing that HAProxy sadly still can't do is generic UDP connections. So we used HAProxy and nginx on our logging lbs. But HAProxy released support for syslog/udp in 2.3 so we are about to change that. :)
We use HAProxy together with nginx. There are a number of reasons.
Nginx can do everything (more or less) but you don't want your load balancer serving web pages. Some error in config (which might have nothing to do with load balancing) and your entire setup comes to a screeching halt. Imagine that you have a Nodejs app, a Dotnet Core app, static files served by Nginx, and a php app. You just make some mistake and your 4 apps come to a standstill. You have lost your redundancy too if you have multiple instances of each app.
Even if you say that Nginx will only do the load balancing, Nginx doesn't support PROXY Protocol which is problematic if you forward to other servers who are also not serving the pages.
In addition there is something to be said for doing one thing and doing it well. Nginx is the master toolbox today. It does almost everything. Your load balancer is supposed to be the most stable part of your setup. Wouldn't you prefer to use something that was built just for load balancing?
If you use varnish then HAProxy works well with it and in fact they are made by the same people.
If you want an added level of balance then you can also use dns as a load balancer with multiple HAPROXY instances. Dns is not meant for this perse but you will always have some weak link. Your load balancer can crash too even if it's managed by your cloud provider. Most web browsers today will try other servers if there is more than one in your dns entry so it's like a load balancer. Your dns should be very reliable thus increasing your uptime.
We use 2 haproxy instances with 2 varnish instances with two dns entries.
From what I understand Node.js doesnt need NginX to work as a http server (or a websockets server or any server for that matter), but I keep reading about how to use NginX instead of Node.js internal server and cant find of a good reason to go that way
Here http://developer.yahoo.com/yui/theater/video.php?v=dahl-node Node.js author says that Node.js is still in development and so there may be security issues that NginX simply hides.
On the other hand, in case of a heavy traffic NginX will be able to split the job between many Node.js running servers.
In addition to the previous answers, there’s another practical reason to use nginx in front of Node.js, and that’s simply because you might want to run more than one Node app on your server.
If a Node app is listening on port 80, you are limited to that one app. If nginx is listening on port 80 it can proxy the requests to multiple Node apps running on other ports.
It’s also convenient to delegate TLS/SSL/HTTPS to Nginx. Doing TLS directly in Node is possible, but it’s extra work and error-prone. With Nginx (or another proxy) in front of your app, you don’t have to worry about it and there are tools to help you securely configure it.
But be prepared: nginx don't support http 1.1 while talking to backend so features like keep-alive or websockets won't work if you put node behind the nginx.
UPD: see nginx 1.2.0 - socket.io - HTTP/1.1 - Proxy websocket connections for more up-to-date info.