I'm looking for a mechanism to limit the number of concurrent connections to a service exposed using ASP.NET WebAPI.
Why? Because this service is performing operations that are expensive on the hardware resources and I would like to prevent degradation under stress.
More info:
I don't know how many requests will be issued per period of time.
This service runs in its own IIS application pool and limiting the maximum connections on the parent site in IIS is not an option.
I found this suite, but the supported algorithms do not include the one that I'm interested in.
I'm looking for something out of the box (something as straightforward as an IIS config setting) but I could not find exactly what I need.
Any clues?
Thanks!
Scaling your service would probably be a better idea than limiting the number of requests. You could send the heavy processing to some background jobs and keep your API servicing requests.
But assuming the above cannot be done, you will need to use one of the throttling package available or write your own if none meets your requirements.
I suggest starting with the ThrottlingHandler from WebApiContrib
You might be able to meet your needs by properly implementing the GetUserIdentifier method.
If not, you will need to implement your own MessageHandler and the handler mentioned would be a good starting point.
Related
I'm building an ASP.NET MVC application that will be hosted on relatively small dedicated server and I'm currently in a process of optimizing its performances (trying to leverage most of the points stated in this post, except for the load balancing part, no scaling).
Now I'm expecting that most of the time the traffic will be small and consistent, but on some very rare occasion it could peak.
The question is can this be handled gracefully?
Is there a way how I could detect when there are too much requests that the server cannot handle?
Also in case of detection, is there a way how I could perhaps send a static HTML to new user(s), that would notify him(them) about too much traffic?
I'm not exactly sure but I believe you're looking for request throttling.
Check this question and the provided answers, in short you can use Dynamic IP Restrictions to block unusually high number of concurrent requests and you can use some throttle implementation, if interested see Throttling Pattern for more information.
In case you're not worried about concurrent requests but with the amount of requests in general then the only thing that I would suggest is to limit the number of simultaneous executions of some expensive controller's actions.
Our client requirement is to develop a WCF which can withstand with 1-2k concurrent website users and response should be around 25 milliseconds.
This service reads couple of columns from database and will be consumed by different vendors.
Can you suggest any architecture or any extra efforts that I need to take while developing. And how do we calculate server hardware configuration to cope up with.
Thanks in advance.
Hardly possible. You need network connection to service, service activation, business logic processing, database connection (another network connection), database query. Because of 2000 concurrent users you need several application servers = network connection is affected by load balancer. I can't imagine network and HW infrastructure which should be able to complete such operation within 25ms for 2000 concurrent users. Such requirement is not realistic.
I guess if you simply try to run the database query from your computer to remote DB you will see that even such simple task will not be completed in 25ms.
A few principles:
Test early, test often.
Successful systems get more traffic
Reliability is usually important
Caching is often a key to performance
To elaborate. Build a simple system right now. Even if the business logic is very simplified, if it's a web service and database access you can performance test it. Test with one user. What do you see? Where does the time go? As you develop the system adding in real code keep doing that test. Reasons: a). right now you know if 25ms is even achievable. b). You spot any code changes that hurt performance immediately. Now test with lots of user, what degradation patterns do you hit? This starts to give you and indication of your paltforms capabilities.
I suspect that the outcome will be that a single machine won't cut it for you. And even if it will, if you're successful you get more traffic. So plan to use more than one server.
And anyway for reliability reasons you need more than one server. And all sorts of interesting implementation details fall out when you can't assume a single server - eg. you don't have Singletons any more ;-)
Most times we get good performance using a cache. Will many users ask for the same data? Can you cache it? Are there updates to consider? in which case do you need a distributed cache system with clustered invalidation? That multi-server case emerging again.
Why do you need WCF?
Could you shift as much of that service as possible into static serving and cache lookups?
If I understand your question 1000s of users will be hitting your website and executing queries on your DB. You should definitely be looking into connection pools on your WCF connections, but your best bet will be to avoid doing DB lookups altogether and have your website returning data from cache hits.
I'd also look into why you couldn't just connect directly to the database for your lookups, do you actually need a WCF service in the way first?
Look into Memcached.
I've got an asp.net web page that is making 7 async requests to a WCF service on another server. Both boxes are clean without anything else installed.
I've also increased maxconnections in web.config to 20.
I run a single call through the system and the page returns in 800ms. The long and short of it is I think that the threadpool is being being overwhelmed as, once placed underload I cannot get more that 8 requests per second, even though both quad core boxes are running at 20% CPU load and the sql server it's connected to is returning the querys in under 10ms per call.
I've changed the service behaviour to concurrency.multiple but that's not seeming to help.
Any ideas anyone.
There are many different factors that could be in play here. Taking a stab at the remark that changing your instancing model on the service had zero effect (big IF here) then its possible the 'bottleneck' is upstream from the service. Either at the web server, or the client load generator.
You've got several areas to review for tuning: client, web server, wcf service server - that's assuming there are no network devices in the middle. Pick an end and work towards the other end. Since I'm already making an assumption that its not the service, then I'd start at the client and work my way towards the wcf service.
Client
What machine is driving the load against the web server? A laptop? A desktop? A dedicated test agent, or a shared one? The client acting as the load generator for purposes of this test is also susceptible to maxConnections limitation as this is a client setting.
What is the CPU utilization of the client generating load? Could it be that the test driver is just unable to generate enough load to push these boxes? Can you add additional test clients to your test?
Web Server
What does the system.net/processModel element look like in machine.config on the ASP.NET web server? Try setting autoConfig = true. This will allow the configuration to auto size based on the 'size' of the machine its running on.
WCF Service
Review WCF service for any throttling defaults that might be in play and tweak appropriately. See ServiceThrottlingBehavior on MSDN.
Let us know any changes in behavior you might observe (if any) if you make any changes!
The real answer here that everyone missed is that you're using an ASP.NET web page. That means your client is some form of web browser. Modern web browsers have a limit of 2 concurrent async requests at any time. This means that 5 of your requests were queued up and waiting for the first two to finish. Once those first two, it served the next two, then the next two, then the last one.
All of these round-trips and handshakes simply take time. I'm guessing that your roundtrip time is around 200ms, unfortunately you have to do it 4 times.
I also really dislike the "max 2" browser limitation on making webservice calls.
Is this service hosted in IIS, WAS or a Windows Service?
You should try to set Windows to run services on a higher priority. Your WCF Service is probably creating the threads it needs but they should be running at a low priority.
Hope that helps.
I have two diffrent web services(running on local machine) and pointing to one application pool(1.Can I do that?Is it any performance concern?).I have not much knowledge about how the applicationpool will works.
the other .Net application is using two webservices,but frequently one webservice is not responding which internally calling by ssis package with in the .Net application.
what might be the reason and how to make sure it responds all the time, is there any better way to improve the performance?
if am missing or any further information, Comments Welcome
Yes you can have multiple web applications using to the same application pool.
Is it a performance concern? If it is really high traffic or is faulty code, then perhaps.
Application pools allow pushing sites to different processes, reducing the risk of each affecting the other. If one app pool contains an application/web application that has a memory leak, the leak will only affect that particular process, at least directly. Each process can be recycled either by time or system parameters, which mitigates risks of having something in a bad state.
Performance? Another benefit to app pools is the ability to have multiple instances running simultaneously (a similar thing when putting each app in its own pool). The benefit of this is that more request can be handled at a time. The down-side is that you cannot use in-process session state and your application state will be duplicated for each instance of the process. You would need to consider how much 'stuff' you keep in session and how your caching scheme would be effected, but, it has potential for giving a web application more scalability.
You mention call SSIS... I am assuming that is a long-running service, so you would probably want to push the call to that process to some sort of queue that can process outside of the web service request. MSMQ might work for you. If using a queue as such, you would initiate the running of the code, then have a way of checking on the status of the call to see if it is done.
I agree with Greg Ogle but one more point I think is worth mentioning. Splitting the applications into multiple app pools will also give you an added benefit when it comes to troubleshooting if there are any issues. If you have the various applications split out, you can tell specifically what app pool is related to what w3wp.exe process in the time of need. Like say when that w3wp.exe process is taking 98% of your cpu.
This might be a bit of a silly question but;
If I have two people logging on to my site at exactly the same time, will the server side code be executed one after the other or will they be executed simultaneously in separate threads?
I'm curious in regards to a denial of service attack on a website login. Does the server slow down because it has a massive queue of logins or is it slow because it has a billion simultaneous logins!
This is not related to ASP.NET per se (I have very little knowledge in that area), but generally web servers. Most web servers use threads (or processes) to handle requests, so basically, whatever snippet of code you have will be executed for both connections in parallel. Of course, if you access a database or some other backend system where a lock is placed, allowing just one session to perform queries, you might have implicitly serialized all requests.
Web servers typically have a minimum and maximum number of workers, which are tuned to the current hardware (CPUs, memory, etc). If these are exhausted, new requests will be queued waiting for a worker to become available, or until a maximum queue length of pending requests has been reached at which point it disregards new connections, effectively denying service (if this is on purpose, it's called a denial of service or DoS attack).
So, in your terms it's a combination, it's a huge number of simultaneous requests filling up the queue.
It should use a thread pool. Note that they are still in the same application, so application level items like static variables are still shared between them.
from this article
"Remember ISAPI is multi-threaded so requests will come in on multiple threads through the reference that was returned by ApplicationDomainFactory.Create(). Listing 1 shows the disassembled code from the IsapiRuntime.ProcessRequest method that receives an ISAPI ecb object and server type as parameters. The method is thread safe, so multiple ISAPI threads can safely call this single returned object instance simultaneously."
So yes, in the case of a DOS attack, it would be slow because of the large number of connections
As others said, most webservers use multiple processes or threads (better) to serve multiple requests at a time. In particular, you can set each ASP.NET application pool with a max number of queued requests and max worker processes. Each process has multiple threads up to a maximum (not configurable AFAIK, I may be wrong), and incoming requests are processed on a first-in-first-out basis.
Moreover, ASP.NET processes one single request for each session - but a malicious user can open as many sessions as she wants.
Multiple logins will probably hit the database and bring it to its knees probably before the webserver itself.
As far as I know, there is not a built-in way to throttle ASP.NET requests other than setting the max number of queued requests (waiting to be processed). This number should be ideally very small. You can monitor the number of queued ASP.NET requests using performance counters. Say you find that, on peak traffic, this number is 100. You can then update application so that it refuses login attempts when this number is above 100 so that the database is not hit (never did that, just a thought).