I'm building an ASP.NET MVC application that will be hosted on relatively small dedicated server and I'm currently in a process of optimizing its performances (trying to leverage most of the points stated in this post, except for the load balancing part, no scaling).
Now I'm expecting that most of the time the traffic will be small and consistent, but on some very rare occasion it could peak.
The question is can this be handled gracefully?
Is there a way how I could detect when there are too much requests that the server cannot handle?
Also in case of detection, is there a way how I could perhaps send a static HTML to new user(s), that would notify him(them) about too much traffic?
I'm not exactly sure but I believe you're looking for request throttling.
Check this question and the provided answers, in short you can use Dynamic IP Restrictions to block unusually high number of concurrent requests and you can use some throttle implementation, if interested see Throttling Pattern for more information.
In case you're not worried about concurrent requests but with the amount of requests in general then the only thing that I would suggest is to limit the number of simultaneous executions of some expensive controller's actions.
Related
I'm looking for a mechanism to limit the number of concurrent connections to a service exposed using ASP.NET WebAPI.
Why? Because this service is performing operations that are expensive on the hardware resources and I would like to prevent degradation under stress.
More info:
I don't know how many requests will be issued per period of time.
This service runs in its own IIS application pool and limiting the maximum connections on the parent site in IIS is not an option.
I found this suite, but the supported algorithms do not include the one that I'm interested in.
I'm looking for something out of the box (something as straightforward as an IIS config setting) but I could not find exactly what I need.
Any clues?
Thanks!
Scaling your service would probably be a better idea than limiting the number of requests. You could send the heavy processing to some background jobs and keep your API servicing requests.
But assuming the above cannot be done, you will need to use one of the throttling package available or write your own if none meets your requirements.
I suggest starting with the ThrottlingHandler from WebApiContrib
You might be able to meet your needs by properly implementing the GetUserIdentifier method.
If not, you will need to implement your own MessageHandler and the handler mentioned would be a good starting point.
TL;DR
Is it appropriate for a (dedicated) web server to be sending many requests out to other servers every second (naturally with permission from said server)?
I'm asking this purely to save myself spending a long time implementing an idea that won't work, as I hope that people will have some more insight into this than me.
I'm developing a solution which will allow clients to monitor the status of their server. I need to constantly (24/7) obtain more recent logs from these servers. Unfortunately, I am limited to getting the last 150 entries to their logs. This means that for busy clients I will need to poll their servers more.
I'm trying to make my solution scalable so that if it gets a number of customers, I won't need to concern myself with rewriting it, so my benchmark is 1000 clients, as I think this is a realistic upper limit.
If I have 1000 clients, and I need to poll their servers every, let's give it a number, two minutes, I'm going to be sending requests off to more than 8 servers every second. The returned result will be on average about 15,000 characters, however it could go more or less.
Bearing in mind this server will also need to cope with clients visiting it to see their server information, and thus will need to be lag-free.
Some optimisations I've been considering, which I would probably need to implement relatively early on:
Only asking for 50 log items. If we find one already stored (They are returned in chronological order), we can terminate. If not, we throw out another request for the other 100. This should cut down traffic by around 3/5ths.
Detecting which servers get more traffic and requesting their logs less commonly (i.e. if a server only gets 10 logged events every hour, we don't want to keep asking for 150 every few minutes)
I'm basically asking if sending out this many requests per second is considered a bad thing and whether my future host might start asking questions or trying to throttle my server. I'm aiming to go shared for the first few customers, then if it gets popular enough, move to a dedicated server.
I know this has a slight degree of opinion enabled, so I fear that it might be a candidate for closure, but I do feel that there is a definite degree of factuality required in the answer that should make it an okay question.
I'm not sure if there's a networking SE or if this might be more appropriate on SuperUser or something, but it feels right on SO. Drop me a comment ASAP if it's not appropriate here and I'll delete it and post to a suggested new location instead.
You might want to read about the C10K Problem. The article compares several I/O strategies. A certain number of threads that each handle several connections using nonblocking I/O is the best approach imho.
Regarding your specific project I think it is a bad idea to poll for a limited number of log items. When there is a peak in log activity you will miss potentially critical data, especially when you apply your optimizations.
It would be way better if the clients you are monitoring pushed their new log items to your server. That way you won't miss something important.
I am not familiar with the performance of ASP.NET so I can't answer if a single dedicated server is enough. Especially because I do not know what the specs of your server are. Using a reasonable strong server it should be possible. If it turns out to be not enough you should distribute your project across multiple servers.
Sorry for banal question, but i would like to know, while i launch many many http requests to a site, will site goes down or just i will be banned after n request limit (according to server settings)?
As is most always the case the answer is: it depends :)
Now, seriously, it depends on a few things but most likely if you're just hitting "a" server with a single client computer you most likely will not be able to make it go down as servers usually have more bandwidth (networking & processing powers) available to them than most client computers. Plus, when you're making requests to a website you might actually be making requests to a load-balanced network of servers which definitely means that a single client will not be able to outperform the site. If this is the case your IP will probably get blacklisted or maybe just ignored.
The other possibility is to make a lot of requests with a number of different clients at the same time - using different IPs. This is what is usually called a coordinated or distributed denial of service attack. In that case you might be able to make the web server(s) go down for a while.
This is just a simple answer but I hope you get the point.
From all the articles I've read so far about Mochiweb, I've heard this over and over again that Mochiweb provides very good scalability. My question is, how exactly does Mochiweb get its scalability property? Is it from Erlang's inherent scalability properties or does Mochiweb have any additional code that explicitly enables it to scale well? Put another way, if I were to write a simple HTTP server in Erlang myself, with a simple 'loop' (recursive function) to handle requests, would it have the same level of scalability as a simple web server built using the Mochiweb framework?
UPDATE: I'm not planning to implement a full blown web-server supporting every feature possible. My requirements are very specific - to handle POST data from a HTML form with fixed controls.
Probably. :-)
If you were to write a web server that handles each request in a separate process (light weight thread in Erlang) you could reach the same kind of "scalability" easily. Of course the feature set would be different, unless you implement everything Mochiweb has.
Erlang also has great built in support for distribution among many machines, this might be possible to use to gain even more scalability.
MochiWeb isn't scalable itself, as far as I understand it. It's a fast, tiny server library that can handle thousands of requests per second. The way in which it does that has nothing to do with "scalability" (aside from adjusting the number of mochiweb_acceptors that are listening at any given time).
What you get with MochiWeb is a solid web server library, and Erlang's scalability features. If you want to run a single MochiWeb server, when a request comes in, you can still offload the work of processing that request to any machine you want, thanks to Erlang's distributed node infrastructure and cheap message passing. If you want to run multiple MochiWeb servers, you can put them behind a load balancer and use mnesia's distributed features to sync session data between machines.
The point is, MochiWeb is small and fast (enough). Erlang is the scalability power tool.
If you roll your own server solution, you could probably meet or beat MochiWeb's efficiency and "scalability" out of the box. But then you'd have to rethink everything they've already thought of, and you'd have to battle test it yourself.
I would like to create a browser-based network game. To ensure it can be used by as many people as possible, I'd like to embed all the traffic in standard HTTP packages.
Assuming I use IIS as my back end, how should I code this to minimize latency?
Is it reasonable to start with an ASP application of some kind (ASP.NET MVC perhaps) using shared state in memory? Or should I plan from the outset on writing some kind of IIS plugin of my own? Or should I abandon IIS and write a custom server instead?
Is it reasonable to start with each client repeatedly querying, say ten times per second, or should I make the data more stream-like somehow (and if so how)?
This can work just fine, but you're going to have to do something less "conventional."
To make this work, don't do individual requests, keep the request open and then transmit data with the open connection.
You could try using a protocol like Bayeux, but there are no rules here.
Here's an example with ASP.NET using COMET.
Designing an app to hit a web server 10 times a second is not a good idea. Web servers are designed for less frequent client requests and probably larger processing times and reponse sizes than your game will be using. That's not to say a web server wouldn't be able to cope just that it would not be an efficient client-server match.
For any type of app that requires multiple packets per second you really should think about a lighter protocol than HTTP which is fairly verbose. For example if your game needs to send 4 bytes to register a user's battleship co-ordinates you don't really want to transmit an extra few hundred bytes of HTTP headers.
I'd recommend a browser plugin technology like Siverlight of Flash. Both of those support TCP socket connections. You would need to write your own server to sit at the other end of the TCP socket. With that approach you'd also have the advantage of being able to push data out to the clients without having to rely on client polling mechanisms which are required with HTTP, e.g. AJAX.
One problem you will face with standard HTTP messages is the size (and lack of control) over the header data.
I was involved in the design of a flash game which was played by several million people. We needed to communicate with the server every few seconds and ended up using ultra-lightweight JSON messages to save on bandwidth and keep it snappy.
Size of JSON message: 16 bytes
Size of HTTP Header: 200+ bytes
HTTP is not really a good protocol for high traffic, low latency requirements. It was designed for larger, less frequent request/response models and has status codes like 304 Not Modified for this very reason.
You'll probably want to drop down to a custom TCP/IP implementation.