I am a DevOps guy and presently I am running my Ruby on Rails application on ubuntu ec2 where the app and also the web server reside inside the same box but we are using mysql RDS cluster. I can see lot of spikes due to more traffic to the web site. So I am planning to change the system. I wanna put web server nginx in a separate instance and web app in a separate instance. But this needs a load balancer which should reside in nginx box, but once the traffic goes up, the nginx instance can be configured to auto scale. What about the app server instance? It can be configured to auto scale but it needs to attach itself to the web server and web server needs to discover the new app server which was created. How can achieve this? Kindly help me out to get this done.
When you are using one single web server at the moment, a transition to using nginx as static webserver and proxy for another backend webserver on another instance really makes sense and will give you performance boost.
However I am not sure if you really need autoscaling. Autoscaling mostly makes sense if you want to react on fast traffic spikes etc. If you have a more or less continuous workload that might increase over time, it should be easier to manually launch and add another backend server in the nginx config. If this does not work for you, you can still have a look at Amazon's Elastic Loadbalancers and Autoscaling afterwards.
Related
I have a simple flask application deployed on an AWS EC2 instance. The flask app accepts the incoming HTTP request, then do some (potentially heavy and lengthy) computations with the request, and then return the results.
Based on my limited understanding, it is recommended to always use nginx + gunicorn stack for a true flask app. As I try to keep things on the simple side, I just used gunicorn with 8 workers. The app works just fine, as I can query the EC2 instance, and got the result as expected. There is no (or very little) static contents for the app.
As for the traffic, I won't expect many simultaneous requests to the site (maybe ~10 requests at the same time), since it is for internal use. My question is, given my use case, will this (no nginx) harm me in the near future?
have you deployed using Elastic Beanstalk or EC2?
If the later, i recommend for this app using Elastic Beanstalk as it handles a lot of the configuration for you.
From AWS:
Elastic Beanstalk uses nginx as the reverse proxy to map your application to your load balancer on port 80. If you want to provide your own nginx configuration, you can override the default configuration provided by Elastic Beanstalk by including the .ebextensions/nginx/nginx.conf file in your source bundle. If this file is present, Elastic Beanstalk uses it in place of the default nginx configuration file.
Otherwise, at this stage not having NGINX isn't going to affect your App performance however since it's not best practise/future proof, there's no harm in including it. There's a lot of content out there, describing how to do just that.
Cheers!
I've found at that Instagram share their technology implementation with other developers trough their blog. They've some great solutions for the problems they run into. One of those solutions they've is an Elastic Load Balancer on Amazon with 3 nginx instances behind it. What is the task of those nginx servers? And what is the task of the Elastic Load balancers, and what is the relation between them?
Disclaimer: I am no expert on this in any way and am in the process of learning about AWS ecosystem myself.
The ELB (Elastic load balancer) has no functionality on its own except receiving the requests and routing it to the right server. The servers can run Nginx, IIS, Apache, lighthttpd, you name it.
I will give you a real use case.
I had one Nginx server running one WordPress blog. This server was, like I said, powered by Nginx serving static content and "upstreaming" .php requests to phpfpm running on the same server. Everything was going fine until one day. This blog was featured on a tv show. I had a ton of users and the server could not keep up with that much traffic.
My first reaction would be to just use the AMI (Amazon machine image) to spin up a copy of my server on a more powerful instance like m1.heavy. The problem was I knew I would have traffic increasing over time over the next couple of days. Soon I would have to spin an even more powerful machine, which would mean more downtime and trouble.
Instead, I launched an ELB (elastic load balancer) and updated my DNS to point website traffic to the ELB instead of directly to the server. The user doesn’t know server IP or anything, he only sees the ELB, everything else goes on inside amazon’s cloud.
The ELB decides to which server the traffic goes. You can have ELB and only one server on at the time (if your traffic is low at the moment), or hundreds. Servers can be created and added to the server array (server group) at any time, or you can configure auto scaling to spawn new servers and add them to the ELB Server group using amazon command line, all automatically.
Amazon cloud watch (another product and important part of the AWS ecosystem) is always watching your server’s health and decides to which server it will route that user. It also knows when all the servers are becoming too loaded and is the agent that gives the order to spawn another server (using your AMI). When the servers are not under heavy load anymore they are automatically destroyed (or stopped, I don’t recall).
This way I was able to serve all users at all times, and when the load was light, I would have ELB and only one Nginx server. When the load was high I would let it decide how many servers I need (according to server load). Minimal downtime. Of course, you can set limits to how many servers you can afford at the same time and stuff like that so you don’t get billed over what you can pay.
You see, Instagram guys said the following - "we used to run 2 Nginx machines and DNS Round-Robin between them". This is inefficient IMO compared to ELB. DNS Round Robin is DNS routing each request to a different server. So first goes to server one, second goes to server two and on and on.
ELB actually watches the servers' HEALTH (CPU usage, network usage) and decides which server traffic goes based on that. Do you see the difference?
And they say: "The downside of this approach is the time it takes for DNS to update in case one of the machines needs to get decommissioned."
DNS Round robin is a form of a load balancer. But if one server goes kaput and you need to update DNS to remove this server from the server group, you will have downtime (DNS takes time to update to the whole world). Some users will get routed to this bad server. With ELB this is automatic - if the server is in bad health it does not receive any more traffic - unless of course the whole group of servers is in bad health and you do not have any kind of auto-scaling setup.
And now the guys at Instagram: "Recently, we moved to using Amazon’s Elastic Load Balancer, with 3 NGINX instances behind it that can be swapped in and out (and are automatically taken out of rotation if they fail a health check).".
The scenario I illustrated is fictional. It is actually more complex than that but nothing that cannot be solved. For instance, if users upload pictures to your application, how can you keep consistency between all the machines on the server group? You would need to store the images on an external service like Amazon s3. On another post on Instagram engineering – “The photos themselves go straight to Amazon S3, which currently stores several terabytes of photo data for us.”. If they have 3 Nginx servers on the load balancer and all servers serve HTML pages on which the links for images point to S3, you will have no problem. If the image is stored locally on the instance – no way to do it.
All servers on the ELB would also need an external database. For that amazon has RDS – All machines can point to the same database and data consistency would be guaranteed.
On the image above, you can see an RDS "Read replica" - that is RDS way of load balancing. I don't know much about that at this time, sorry.
Try and read this: http://awsadvent.tumblr.com/post/38043683444/using-elb-and-auto-scaling
Can you please point the blog entry out?
Load balancers balance load. They monitor the Web servers health (response time etc) and distribute the load between the Web servers. On more complex implementations it is possible to have new servers spawn automatically if there is a traffic spike. Of course you need to make sure there is a consistency between the servers. THEY CAN share the same databases for instance.
So I believe the load balancer gets hit and decides to which server it will route the traffic according to server health.
.
Nginx is a Web server that is extremely good at serving a lot of static content for simultaneous users.
Requests for dynamic pages can be offloaded to a different server using cgi. Or the same servers that run nginx can also run phpfpm.
.
A lot of possibilities. I am on my cell phone right now. tomorrow I can write a little more.
Best regards.
I am aware that I am late to the party, but I think the use of NGINX instances behind ELB in Istagram blogpost is to provide high available load balancer as described here.
NGINX instances do not seem to be used as web servers in the blogpost.
For that role they mention:
Next up comes the application servers that handle our requests. We run Djangoon Amazon High-CPU Extra-Large machines
So ELB is used just as a replacement for their older solution with DNS Round-Robin between NGINX instances that was not providing high availability.
I am trying to improve the user experience while a backend service is down due to maintenance, shutdown manually.
We do had a frontend web proxy, which happens to be nginx but it could also be something else like a NetScaler instance. An important note is that the frontend proxy is running on a different machine than the backend application.
Now, for the backend service it takes a lot of time to start, even more than 10 minutes in some cases.
Note, I am asking this question on StackOverflow, as opposed to ServerFault because providing a solution for this problem is more likely to require writing some bash code inside the daemon startup script.
What we want to achive:
service mydaemon stop should enable the maintenance page on the frontend proxy
service mydaemon start should disabled the maintenance page on the frontend proxy
In the past we used to create a maintenance.html page and had nginx configured to check the existence of this page using try, before falling back to the backend.
Still, because we decided to move nginx to another machine we cannot do this and doing this using ssh raises security concerns.
We already considered writing this file to a NFS drive which would be accessible by both machine, but even this solution does not scale for a service that has a lot of traffic. Nginx will end-up trying this for every request, slowing down the responses quite a lot.
We are looking for another solution for this problem, one that would ideally be more flexible.
As a note, we still want to be able to trigger this behaviour from inside the daemon script of the backend application. So if the backend application stops responsing for other reasons, we do expect to see the same from the frontend.
We are looking into setting up our application on AWS. This will run on 3 load balanced web servers. We have been looking into how to prevent DDOS attacks and how to serve a static page during maintenance and are looking at going with Nginx. So the setup would be Nginx in front of an elastic load balancer.
As with our setup when there is an upgrade to the application we update Nginx to serve a static maintenance page while a whole new stack comes online with its own elastic load balancer and to switch to the new application stack we will update the Nginx config to point to the new elastic load balancer.
Does this make any sense? the reason i am asking is that I cannot find anything on this type of setup online.
Thanks,
Colin.
The elastic load balancer works by using many ip-addresses. If you do dig amazon.com you can see how it's distributed with a low ttl. Once your nginx-server runs out of open ports it will go down.
It's however a very good idéa to use balancers in-front of your app-server. This scales very well with auto-scaling groups.
When you do the transition, just lower your CNAME ttl and point it to a new load balancer.
I see zero value going in this path-
Putting a single instance of nginx will just increase the complexity of the setup, cost more and most important will introduce a new single point of failure and performance/latency bottleneck (and will make your env much more vulnerable to DDOS).
AWS infrastructure is 100% programmable - learn how to control the ELB programatically : how to direct the traffic to a static site during maintenance (could be nginx hosted on one of your instances), and how to support your upgrade workflows.
I'm currently using Nginx as a web server for Openerp. It's used to handle SSL and cache static data.
I'm considering extending it's use to also handle fail over and load balancing with a second server, using the upstream module.
In the process, it occurred to me that Nginx could also do this on multiple Openerp servers on the same machine, so I can take advantage of multiple cores. But Gunicorn seems to the the preferred tool for this.
The question is: can Nginx do a good job handling traffic to multiple local OpenERP servers, bypassing completely the need for Gunicorn?
Let first talk what they both are bascially.
Nginx is a pure web server that's intended for serving up static content and/or redirecting the request to another socket to handle the request.
Gunicorn is based on the pre-fork worker model. This means that there is a central master process that manages a set of worker processes. The master never knows anything about individual clients. All requests and responses are handled completely by worker processes.
If you see closely Gunicorn is Designed from Unicron, Follow the link for the detail more diff
which show the ngix and unicrom same model work on Gunicron also.
nginx is not a "pure web server" :) It's rather a web accelerator capable of doing load balancing, caching, SSL termination, request routing AND static content. A "pure web server" would be something like Apache - historically a web server for static content, CGIs and later for mod_something.