I've developed a 3rd party app that I have integrated on a site that gets about 2-3 million hits per day
I'm serving the required assets (css, js and image file) from AWS Cloudfront
The costs are starting to add up, and I'm wondering the cheapest way to deliver static assets
I have a micro instance, but I'm not sure if that could handle that many requests (even though it's static content). Think it could?
Or have any recommendations on mirror cdns like cloudfront that are cheaper?
Maybe Cloudflare is cheaper for your requirements, take a look:
Cloudflare Plans
Even if you change your DNS to point to CloudFalare, if your hosting server fails to respond your site will be unavailable, so at the end of the day if your hosting server is unreliable, you won't be having much success.
Related
I know by experience that with load balancers L7 we can redirect traffic to others endpoints if applications become unhealthy.
As Firebase Hosting is using CDN to deliver static content, is there a way to redirect traffic if this static content becomes unavailable?
I thought of using DNS rules that uses redirect but I am concerned about the propagation time that may be too much. (and that could even take more time then Google resolving the incident)
Is there a way to manage that with this service or should I switch to another kind of architecture?
This would depend on your DNS manager itself, Firebase Hosting is delivered via Fastly and if Fastly is down, a good ~30% of the internet is down in general. This was seen a few weeks ago when multiple social media sites and apps broke. As such, you should ensure your host is not on Fastly as the first condition.
From there, you would have to manage some basic logic through a 3rd party DNS manager. A popular one is Cloud Flare which allows the configuration of load balancer pools where if your primary pool is unavailable, you can connect to your backup pool (costs do apply)
https://developers.cloudflare.com/load-balancing/create-load-balancer-ui
I've found at that Instagram share their technology implementation with other developers trough their blog. They've some great solutions for the problems they run into. One of those solutions they've is an Elastic Load Balancer on Amazon with 3 nginx instances behind it. What is the task of those nginx servers? And what is the task of the Elastic Load balancers, and what is the relation between them?
Disclaimer: I am no expert on this in any way and am in the process of learning about AWS ecosystem myself.
The ELB (Elastic load balancer) has no functionality on its own except receiving the requests and routing it to the right server. The servers can run Nginx, IIS, Apache, lighthttpd, you name it.
I will give you a real use case.
I had one Nginx server running one WordPress blog. This server was, like I said, powered by Nginx serving static content and "upstreaming" .php requests to phpfpm running on the same server. Everything was going fine until one day. This blog was featured on a tv show. I had a ton of users and the server could not keep up with that much traffic.
My first reaction would be to just use the AMI (Amazon machine image) to spin up a copy of my server on a more powerful instance like m1.heavy. The problem was I knew I would have traffic increasing over time over the next couple of days. Soon I would have to spin an even more powerful machine, which would mean more downtime and trouble.
Instead, I launched an ELB (elastic load balancer) and updated my DNS to point website traffic to the ELB instead of directly to the server. The user doesn’t know server IP or anything, he only sees the ELB, everything else goes on inside amazon’s cloud.
The ELB decides to which server the traffic goes. You can have ELB and only one server on at the time (if your traffic is low at the moment), or hundreds. Servers can be created and added to the server array (server group) at any time, or you can configure auto scaling to spawn new servers and add them to the ELB Server group using amazon command line, all automatically.
Amazon cloud watch (another product and important part of the AWS ecosystem) is always watching your server’s health and decides to which server it will route that user. It also knows when all the servers are becoming too loaded and is the agent that gives the order to spawn another server (using your AMI). When the servers are not under heavy load anymore they are automatically destroyed (or stopped, I don’t recall).
This way I was able to serve all users at all times, and when the load was light, I would have ELB and only one Nginx server. When the load was high I would let it decide how many servers I need (according to server load). Minimal downtime. Of course, you can set limits to how many servers you can afford at the same time and stuff like that so you don’t get billed over what you can pay.
You see, Instagram guys said the following - "we used to run 2 Nginx machines and DNS Round-Robin between them". This is inefficient IMO compared to ELB. DNS Round Robin is DNS routing each request to a different server. So first goes to server one, second goes to server two and on and on.
ELB actually watches the servers' HEALTH (CPU usage, network usage) and decides which server traffic goes based on that. Do you see the difference?
And they say: "The downside of this approach is the time it takes for DNS to update in case one of the machines needs to get decommissioned."
DNS Round robin is a form of a load balancer. But if one server goes kaput and you need to update DNS to remove this server from the server group, you will have downtime (DNS takes time to update to the whole world). Some users will get routed to this bad server. With ELB this is automatic - if the server is in bad health it does not receive any more traffic - unless of course the whole group of servers is in bad health and you do not have any kind of auto-scaling setup.
And now the guys at Instagram: "Recently, we moved to using Amazon’s Elastic Load Balancer, with 3 NGINX instances behind it that can be swapped in and out (and are automatically taken out of rotation if they fail a health check).".
The scenario I illustrated is fictional. It is actually more complex than that but nothing that cannot be solved. For instance, if users upload pictures to your application, how can you keep consistency between all the machines on the server group? You would need to store the images on an external service like Amazon s3. On another post on Instagram engineering – “The photos themselves go straight to Amazon S3, which currently stores several terabytes of photo data for us.”. If they have 3 Nginx servers on the load balancer and all servers serve HTML pages on which the links for images point to S3, you will have no problem. If the image is stored locally on the instance – no way to do it.
All servers on the ELB would also need an external database. For that amazon has RDS – All machines can point to the same database and data consistency would be guaranteed.
On the image above, you can see an RDS "Read replica" - that is RDS way of load balancing. I don't know much about that at this time, sorry.
Try and read this: http://awsadvent.tumblr.com/post/38043683444/using-elb-and-auto-scaling
Can you please point the blog entry out?
Load balancers balance load. They monitor the Web servers health (response time etc) and distribute the load between the Web servers. On more complex implementations it is possible to have new servers spawn automatically if there is a traffic spike. Of course you need to make sure there is a consistency between the servers. THEY CAN share the same databases for instance.
So I believe the load balancer gets hit and decides to which server it will route the traffic according to server health.
.
Nginx is a Web server that is extremely good at serving a lot of static content for simultaneous users.
Requests for dynamic pages can be offloaded to a different server using cgi. Or the same servers that run nginx can also run phpfpm.
.
A lot of possibilities. I am on my cell phone right now. tomorrow I can write a little more.
Best regards.
I am aware that I am late to the party, but I think the use of NGINX instances behind ELB in Istagram blogpost is to provide high available load balancer as described here.
NGINX instances do not seem to be used as web servers in the blogpost.
For that role they mention:
Next up comes the application servers that handle our requests. We run Djangoon Amazon High-CPU Extra-Large machines
So ELB is used just as a replacement for their older solution with DNS Round-Robin between NGINX instances that was not providing high availability.
I'm just learning about CDNs, so please forgive if this is a dumb question.
Would implementing a CDN involve moving images and changing paths?
Yes a CDN (Content Delivery Network) is at it basis nothing more that a set of webservers.
If you want to host files on a CDN you must copy your files to the CDN servers and then use the full CDN address that points to those files on those servers on your own webpage.
You can use a CDN on the same server but different URI. For instance, having your page in: www.example.com with cdn: cdn.example.com (with cdn.example.com as a vhost alias) should be faster then getting all data only from www.example.com, i think it's because of the number of http connections related to the address.
Of course it's best if you have it in another server, in this case you have to copy everything.
Not necessarily. You can use a service such as CloudFlare which requires only a modification of some of your DNS settings. In short, the service determines which files being served are static, and caches those in its network, generally reducing overall traffic to your servers. You also get the benefit of any geographical distribution the service provides that your own hosting service might not.
I have very recently started development on a multiplayer browser game that will use nowjs to synchronize player states from the server state. I am new to server-side development (so many of the things I'm saying are probably being said incorrectly), and while I understand how node.js works on its own I have seen discussions about proxying HTTP requests through another server technology (a la NGinx or Apache) for efficiency.
I don't understand why it would be beneficial to do so, even though I've seen plenty of explanations of how to do so. My current plan is to have the game's website and info on the same server as the game itself, so if there is any gain from proxying node I'd love to know why.
In the context of your question it seems you are looking for an answer on the benefits of implementing a reverse proxy in front of your node.js webserver. In summary, a reverse proxy (depending on implementation) can provide the following features out of the box:
Load balancing
Caching of static content
Failover
Compression of responses (e.g gzip)
SSL support
All these features are cross-cutting concerns that you should not need to accommodate in your application tier/code. By implementing these features within the proxy it allows you to focus on developing the code for your application and leaves the web server to do what it's good at, serving the HTTP requests for your application.
nginx appears to be a common choice in a reverse proxy/node configuration and if you take a look at the modules reference you should get a feel for what features the proxy can provide.
When you say "through another technology" I assume you mean through a dedicated web server such as NGinx or Apache.
The reason you do that is b/c in a production environment there are a number of considerations you don't want your application to have to do on its own. Caching, domain (or sub-domain) mapping, perhaps security, SSL, load balancing, and serving static files to name a few.
The web servers are already built to do all those things for you, and so they can handle them and then pass only the requests on to your app that actually need to be handled by your app. They're also optimized for doing those things and will probably do them as well or better than the average developer can.
Hope that helps.
Another issue that people haven't added in here is that with a front-end proxy, when you need to take your service down for maintenance (or even just restart it), nginx can serve up a pretty "YourCompanyName is currently under maintenance" page, making for a much more pleasant user experience.
I have a website with a lot of large files. However, I don't want users to start downloading like 10 files at a time. I noticed there are website out there where they only allow 2 simultaneous downloads.
My website is programmed using ASP.net running on IIS. Does anyone know how I can limit simultaneous downloads?
The Dynamic IP Restrictions module from Microsoft (currently in beta) will do this.
For details and a download: http://www.iis.net/download/DynamicIPRestrictions
I think the only problem with max concurrent in IIS is it might block page requests rather than just download requests.
I'd say write an HTTP Handler which actually does the download and can then decide (based on IP or Cookie) if a download is allowed to be sent back to the browser. Pretty straight forward code I'd think.
Do you want to do it programatically? Otherwise I believe there is a setting for max conncurrent connections from an ip address for IIS.
I think the only problem with max concurrent in IIS is it might block page requests rather than just download requests.
I'm no IIS expert but, if this setting is per domain / virtual host, you are set. If you can serve your downloads from a sub-domain that isn't used for anything, the setup it will not interfere with browsers that fetch several page elements at once.