Pretty much the tile. Got a node app behind nginx, and when i restart the app i would like nginx to delay the response, and retry doing the request a couple of times with some delay inbetween. Everything that i found would only instantly retry N times, but that obviously is no useful when the app is down for a restart, which is my use-case. Is there some way? I dont even care how hacky it is / if it is, i just need a solution that is not starting a second instance of the app, and killing the first one when the second one started.
Thanks!
You can add same server as multiple upstream and configure proxy_next_upstream, proxy_next_upstream_timeout and proxy_next_upstream_tries options as well. Reference
upstream node_servers {
server 127.0.0.1:12005;
server 127.0.0.1:12005;
}
...
proxy_next_upstream http_502;
proxy_next_upstream_timeout 60;
proxy_next_upstream_tries 3;
However, I would recommend you to use process managers like pm2 which support graceful reload/restart. They are relevant if you are using more than one CPU for your nodejs server in a clustered mode.
Related
I have a long-running request, which I want to handle in Play Framework which is behind an Nginx reverse proxy.
After about 3 minutes, I get a 502 Bad Gateway as a response. Play keeps working on the request, though.
I believe the Play Framework closes the socket. I tried to set the following configurations:
In Nginx
proxy_read_timeout 15m;
proxy_connect_timeout 15m;
In play framework's application conf:
play.server.http.idleTimeout = infinite
play.server.https.idleTimeout = infinite
play.server.akka.http.server.idle-timeout = infinite
play.server.akka.http.client.idle-timeout = infinite
play.server.akka.http.server.request-timeout = infinite
play.server.akka.http.client.connecting-timeout = infinite
play.server.akka.http.host-connection-pool.max-connection-lifetime = infinite
play.server.akka.http.host-connection-pool.idle-timeout = infinite
play.server.akka.http.host-connection-pool.client.idle-timeout = infinite
(I just added all settings that might be related and set them to infinite. Happy to throw all unrelated settings out.)
But that does not appear to change anything. After about 3 minutes I get a 502 BAd Gateway. Is there any way to keep the socket alive or is there another explanation for this behaviour?
Play Version: 2.7.2
Additional information:
I set all the values to extremely low values. All the play configuration doesn't seem to have any impact. The Nginx configurations works. So, I think it has nothing to do with those play settings.
I tried to confirm that it is actually happening in Play and not in Nginx. I took the time it takes to happen when the backend server could just start: 76 seconds. When it has to go through the get ready motions after being started (compiling, liquibase): 97 seconds. I believe that confirms that the issue is to be found in Play and not in Nginx or the frontend.
I want to request a Microsoft API which is not protected for concurrent access (pull the data with a GET, then Push it with a POST).
To prevent any weird behavior, I want to use a lock when I'm accessing this Api.
The easiest way I've found out (without messing up the code) is to create a middleware service (it will be targeted instead of the original one).
When requested, it could save a Lock in a redis, and forward the request to Microsoft.
When it's done, the lock is released.
Then if another request is coming to the server, it will be denied, and I'll be able to perform an exponential backoff until the lock is free.
My question is : Do I Have to code this thing, or is this a feature which could be found in an existing reverse proxy ?
To do this in a highly available way, I believe you would need to do what you wrote and use some sort of distributed lock to determine if the API was in use.
However, if high availability is not required, you could use a single HAProxy instance with maxconn set to 1 for that server. You would also want to set timeout queue to something short so that you could handle the 503 response and do the exponential backoff you mention.
backend microsoft_api_backend
timeout queue 2s
server microsoft_api 1.1.1.1:80 check maxconn 1
In Nginx, you could do something equivalent:
upstream microsoft_api {
server 1.1.1.1:80 max_conns=1
queue 1 timeout=2
}
server {
location / {
proxy_pass http://microsoft_api;
}
}
I'm looking into the built-in HTTP server of Rserve (1.8.5) after modifying .http.request() from FastRWeb. It's fine with the updated request function but the issue is, whenever # concurrent requests are high, some/most of them throw the following error.
WARNING: fork() failed in fork_http(): Cannot allocate memory
WARNING: fork() failed in Rserve_prepare_child(): Cannot allocate memory
This is due to there's not enough free memory remaining and it is necessary to limit # requests in one way or another.
I tried a couple of client layers (1) Python's requests + hug libraries, (2) Python's pyRserve + hug libraries where # worker processes are adjusted by # CPUs. Also I tried reverse proxy with Nginx both in a single/multiple container setup (3) (4).
In all the cases, I observe some overhead (~ 300 - 450 ms) compared to the setup of only Rserve with the built-in HTTP server.
I guess using it as it is would be the most efficient option but I'm concerned that it just keeps trying to fork and returns an error. (Besides errors are quickly thrown, it wouldn't be easy to auto-scale with some typical metrics such as CPU utilization or mean response time.)
Can anyone inform if there is a way to enforce rate limiting with/without relying on another tool, which doesn't sacrifice performance?
My Rserve config is roughly as following.
http.port 8000
socket /var/rserve/socket
sockmod 0666
control disable
Also here is a simplified nginx.conf.
worker_processes auto;
events {
worker_connections 1024;
}
http {
upstream backend {
server 127.0.0.1:8000;
}
server {
listen 80;
location / {
proxy_pass http://backend;
}
}
}
I was misguided by Locust (load testing tool) that it showed cached output for the setup of Rserve with the built-in HTTP server.
Manual investigation shows Rserve + Nginx returns a slightly improved result.
How I could call all upstreams at once and return a result from first that respond and the response will not be a 404?
Example:
Call to load balancer at "serverX.org/some-resource.png" creates two requests to:
srv1.serverX.org/some-resource.png
srv2.serverX.org/some-resource.png
srv2 responds faster and the response is shown to the user.
Is this possible at all? :)
Thanks!
Short answer, NO. You can't do exactly what you described with nginx. Come to think of it a bit, this operation can't be called load balancing since the whole back-end gets the total amount of traffic.
A good question is what do you think that you could accomplish with that? Better performance?
You can be sure that you will have better results with simple load balancing between your servers since the will have to handle the half of the traffic.
In case that you have a more complex architecture i.e. different loads from different paths to your backend servers we could discuss a more sophisticated load balancing method.
So if your purpose is sth else than performance there are some things that you can do:
1) After you sent the request to first server you can send it using the post_action to another one.
location ~ ^/*.png {
proxy_pass http://srv1.serverX.org;
...
post_action #mirror_to_srv2;
...
}
location #mirror_to_srv2 {
proxy_ignore_client_abort on;
...
proxy_pass http://srv2.serverX.org;
}
2) The request is available to you in nginx as a variable so with some lua scripting you can send it where ever you want.
Note that the above methods are not useful to tackle performance issues but to enable you to do things like mirroring live traffic to dev servers for test/debug purposes.
Last this one seems to provide the functionality you want but remember that isn't built for the use that you seem to have in mind.
We're using Nginx to load balance between two upstream app servers and we'd like to be able to take one or the other down when we deploy to it. We're finding that when we shut one down, Nginx is not failing over to the other. It keeps sending requests and logging errors.
Our upstream directive has the form:
upstream app_servers {
server 10.100.100.100:8080;
server 10.100.100.200:8080;
}
Our understanding from reading the Nginx docs is that we don't need to explictly specify the "max_fails" or "fail_timeout" because they have reasonable defaults. (ie. max_fails of 1).
Any idea what we might be missing here?
Thanks much.
As per the documentation...
max_fails = NUMBER - number of unsuccessful attempts at communicating
with the server within the time period (assigned by parameter
fail_timeout) after which it is considered inoperative. If not set,
the number of attempts is one. A value of 0 turns off this check. What
is considered a failure is defined by proxy_next_upstream or
fastcgi_next_upstream (except http_404 errors which do not count
towards max_fails).
As per the documentation, a failure is define by proxy_next_upstream or fastcgi_next_upstream.
It keeps sending requests and logging errors.
Please check the log what type of errors were logged, if it is not the default (error or timeout), then you may want to define it exclusively in proxy_next_upstream or fastcgi_next_upstream.