I'm having There are too many outstanding HTTP requests with a total 100 outstanding requests error each time I try to login to apache NiFi and sometimes when I'm working on the interface, memory and cpu are available on the cluster machines and NiFi flows work normally even when I can't login, is there a reason I keep getting this? And what can I try to prevent it?
You can change nifi.cluster.node.max.concurrent.requests to a greater number. As stated here, the number of requests is limited to 100 by default, this would increase the possible number of requests.
It might not fix the problem that makes your cluster experience so many requests, but it could at least let you login to your cluster.
Related
I am using Scrapy to build a broad crawler which will crawl a few thousand pages from 50-60 different domains.
I have encountered 429 status code sometimes. I am thinking of ways of dealing with it. I am trying to set polite policies regarding Concurrent requests per domain and autothrottle settings. This is a worst-case situation.
By default, Scrapy drops the request.
If we add 429 to RETRY_HTTP_CODES, Scrapy will use the default retry middleware which will schedule the request at the end of the queue. This will still allow other requests to the same domain to ping the server - Does this prolong the temporary block in place due to rate limiting? If not, why not use this approach only instead of trying other complex solutions as described below?
Another approach is to block the spider when it encounters a 429. However, one of the comments mentions that this will lead to a timeout in other active requests. Also, this would block requests to all the domains (which is an inefficient way as requests to other domains should continue normally). Does it make sense to temporarily reschedule requests to a particular domain instead of pinging the server continuously with other requests to the same domain? If yes, how to implement it in Scapy?
Does this solve the issue?
When Rate Limiting is already triggered - does sending more requests (which will receive a 429 response) prolong the time period for which rate limiting is applied? Or will it have no effect on rate limiting's time period?
How to pause scrapy to send requests to a particular domain, while continuing its other tasks (including requests to other domains)?
EDIT:
The default Retry Middleware cannot be used as it has a max retry counter - RETRY_TIMES. After this has expired for a particular request, that request is dropped - something that we don't want in the case of a 429.
With HTTP/1.0, there used to be a recommended limit of 2 connections per domain. More recent HTTP RFCs have relaxed this limitation but still warn to be conservative when opening multiple connections:
According to RFC 7230, section 6.4, "a client ought to limit the number of simultaneous open connections that it maintains to a given server".
More specifically, besides HTTP/2, these days, browsers impose a per-domain limit of 6-8 connections when using HTTP/1.1. From what I'm reading, these guidelines are intended to improve HTTP response times and avoid congestion.
Can someone help me understand what would happen with congestion and response times if many connections were opened by domain? It doesn't sound like an HTTP server problem since the amount of connection they can handle seems like an implementation detail. The explanation above seems to say it's about TCP performance? I can't find any more precise explanations for why HTTP clients limit the number of connections per domains.
The primary reasoning for this is resources on the server side.
Imagine that you have a server running Apache with the default of 256 worker threads. Imagine that this server is hosting an index page that has 20 images on it. Now imagine that 20 clients simultaneously connect and download the index page; each of these clients closes these connections after obtaining the page.
Since each of them will now establish connections to download the image, you likely see that the connections increase exponentially (or multiplicatively, I suppose). Consider what happens if every client is configured to establish up to ten simultaneous connections in parallel to optimize the display of the page with images. This takes us very quickly to 400 simultaneous connections. This is nearly double the number of worker processes that Apache has available (again, by default, with a pre-fork).
For the server, resources must be balanced to be able to serve the most likely load, but the clients help with this tremendously by throttling connections. If every client felt free to establish 100+ connections to a server in parallel, we would very quickly DoS lots of hosts. :)
In NiFi, I have an HTTP endpoint that accepts POST requests with payloads varying from 7kb to 387kb or larger (up to 4mb). The goal is to have a clustered implementation capable of handling approximately 10,000 requests per second. However, no matter if I have NiFi clustered with 3 nodes or a single instance, I've never been able to average more than 15-20 requests/second without the Jetty service returning a 503 error. I've tried reducing the time penalty and increasing the number of Maximum Outstanding Requests in the StandardHttpContextMap. No matter what I try, whether on my local machine or on a remote VM, I cannot get any impressive number of requests to go through.
Any idea why this is occurring and how to fix this? Even when clustering, I notice one node (not even the primary node) does the majority of the work and I think this explains why the throughput isn't much higher for a clustered implementation.
No matter at what bulletin level, this is the error I get in the nifi-app.log:
2016-08-09 09:54:41,568 INFO [qtp1644282758-117] o.a.n.p.standard.HandleHttpRequest HandleHttpRequest[id=6e30cb0d-221f-4b36-b35f-735484e05bf0] Sending back a SERVICE_UNAVAILABLE response to 127.0.0.1; request was POST 127.0.0.1
This is the same whether I'm running just two processors (HandleHttpRequest and HandleHttpResponse) as my only processors or I have my general flow where I'm routing on content, replacing some text, and writing to a database or jms messaging system. I can get higher throughput (up to 40 requests/sec) when I'm running just the web service without my entire flow, but it still has a KO rate of about 90%, so it's not much better - still seems to be an issue with the jetty service.
I have a question about the situation which arises during the flash sale in e-commerce websites. Assume there are only 5 items in stock and if 10000 requests hit the server at same instant, how does the server handle the requests and how does it manage to order the request?
Given the cpu speeds of current computers, like it says here
1 million requests per second, would come out as 1 request per 1000
cpu cycles.
Although requests come from many ends of the world, they are received through a single channel. Which means that two requests come after one another even if they are originated at the exact same time. The time of receipt would certainly not be the same, if routing conditions for the two requests are considered. It is impossible for them to hit the server at the exact same time. Because routing wouldn't allow it in order to prevent collisions.
Therefore the order in which the requests are handled is the order they are received at the network interface. After the request packets are through the application layer, each client will have a thread dedicated for itself. But the access of shared variables like the 5 items you mentioned will be synchronized. Therefore only the first 5 threads to acquire the lock on these shared variable will win.
I have a website that is frequently overloaded with multiple requests from thousands of clients. I cannot scale to infinity my servers and the application in current state is not possible to handle the traffic. For a better comfort I would like to let firstly the clients that started the transaction to complete it, and after that allow other clients to start the transaction. I am looking for a solution how to divide HTTP requests to two groups: the first requests that are able to finish the transaction and the others that should receive the 503 Server busy web page.
I can handle some amount of transactions concurrently. The rest transactions I would like to hold for a while with Server busy web page. I thought that I can use varnish for that. Bud I cannot think up the right condition in VCL for that.
I would like to find in varnish the number of current connections to the backend. If the current number of connections will be higher than some value (eg. 100) and the request didn't have a session cookie, the response will be 503 Server busy. If the number of connections will even greater than 100, but the session cookie exists, the requests will be passed to the backend.
AFAIK in varnish VCL I can get only the health of the backend (director) that should be true/false. But when backend is considered not healthy, the requests are not passed to it. When I use max_connections to the backend, all connections up to the limit will got 503 error.
Is there a way how to achive this behavior with varnish, ngingx, apache or any other tool?
Does your content have to be dynamic no matter what? I run a site that handles 3 to 4 million unique a day and use features like grace mode to handle invalidation.
Maybe another option is ESI, Edge Side Includes, that may help reduce load by caching everything that isn't dynamic.