My Setup :
Curl --> AWS ELB --> Nginx --> NodeJS
When number of request High then I face this issue.
Nginx access logs
xx.xx.xx.xx - - [30/Oct/2014:13:23:40 +0000] "POST /some/calls/object HTTP/1.1" 499 0 "-" "curl/7.27.0"
xx.xx.xx.xx - - [30/Oct/2014:13:23:40 +0000] "POST /some/calls/object HTTP/1.1" 499 0 "-" "curl/7.27.0"
Nginx error logs
2014/10/30 13:23:40 [info] 11181#0: *17811 client xx.xx.xx.xx closed keepalive connection
2014/10/30 13:23:40 [info] 11181#0: *17631 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: xx.xx.xx.xx, server: example.com, request: "POST /some/calls/objects HTTP/1.1", upstream: "http://xx.xx.xx.xx:xx/some/calls/object", host: "example.com"
NodeJS logs
2014-10-30T13:23:40.074Z : info : req.method GET
2014-10-30T13:23:40.074Z : info : req.url /some/calls/objects
2014-10-30T13:23:40.074Z : info : error { message: 'Request Timed Out!' }
2014-10-30T13:23:40.075Z : info : req.method GET
2014-10-30T13:23:40.075Z : info : req.url /some/calls/objects
2014-10-30T13:23:40.075Z : info : error { message: 'Request Timed Out!' }
2014-10-30T13:23:40.075Z : info : error.stack undefined
2014-10-30T13:23:40.076Z : info : error.stack undefined
Question
as per this link 499 means client closed the connection, but my question is, who is the client in this scenario, is ELB closing the connection or curl or nodejs ?
I've also observed that nodejs takes 60+ seconds to respond calls, if nodejs encounter Timed out issue, then it keep occurring again and again.
Related
Trying to setup mailgun in wordpress with mailgun plugin.
Installs fine but the 'mailgun lists' settings section shows:
"There has been a critical error on this website. Please check your site admin email inbox for instructions."
Error:
#0 /home/runcloud/webapps/example/wp-content/plugins/mailgun/mailgun.php(241): Mailgun->api_call()
Stack trace:
2021/03/29 22:59:07 [error] 266087#266087: *179493 FastCGI sent in stderr: "PHP message: PHP Fatal error: Uncaught Error: Call to undefined function mg_api_get_region() in /home/runcloud/webapps/example/wp-content/plugins/mailgun/mailgun.php:188
2021/03/29 22:58:18 [error] 266087#266087: *179488 access forbidden by rule, client: 158.801.119.239, server: _, request: "GET /.env HTTP/1.1", host: "159.203.922.254"
2021/03/29 22:35:56 [error] 266087#266087: *179452 FastCGI sent in stderr: "PHP message: PHP Warning: escapeshellcmd() has been disabled for security reasons in /home/runcloud/webapps/example/wp-includes/PHPMailer/PHPMailer.php on line 1779" while reading response header from upstream, client: 262.258.622.926, server: _, request: "POST /wp-login.php?action=lostpassword HTTP/1.1", upstream: "fastcgi://unix:/var/run/example.sock:", host: "example.net", referrer: "https://example.net/wp-login.php?action=lostpassword"
Any ideas on how to fix this?
I have a flask service for external and internal usages.
Recently, in the last month, when deploying to production internal only, one endpoint is timing out and the nginx is reporting Resource temporarily unavailable.
it returns to normal after a few restarts. When trying to investigate:
I didn't see any increase in the number of requests during the last month
I have new relic installed but I don't see anything out of the ordinary (suggestions are welcomed)
full error message is :
2018/01/04 17:46:23 [error] 19#19: *259 connect() to unix:/run/server.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 10.0.1.181, server: , request: "POST /mobile_push_data HTTP/1.1", upstream: "uwsgi://unix:/run/server.socket:", host: "mobile.production.my_domain.com"
the service is running in a docker file running on alpine:3.6
I've installed OSticket application on my nginx server.
The webpage is opened only first time, if I simply refresh the page, it gives connection reset by peer upstream error..
I tried to change fastcgi_read_timeout and max_execution time as described in https://laracasts.com/discuss/channels/forge/502-bad-gateway-with-large-file-uploads and https://www.scalescale.com/tips/nginx/configure-max_execution_time-in-php-fpm-using-nginx/#, that didn't help.
nginx error log:
2017/08/07 22:15:08 [error] 26877#26877: *42 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.231.1.79, server: web.com, request: "GET /ticket/logo.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm/php-fpm.sock:", host: "web.com", referrer: "https://web.com/ticket/"
PHP-FPM error log:
WARNING: [pool www] child 26883 exited on signal 11 (SIGSEGV) after 4270.119171 seconds from start
I have a codebase that's running fine con HHVM.
I'm realizing some load tests with apache ab tool but when i test with 100 concurrent requests I start to get 502 gateway errors.
I generally only get two types of logs in nginx's error log:
2015/01/10 09:42:48 [crit] 29794#0: *302 connect() to
unix:/var/run/hhvm/hhvm.sock failed (2: No such file or directory)
while connecting to upstream, client: 192.168.56.211, server: ,
request: "GET /api/v2/checkaccess HTTP/1.1", upstream:
"fastcgi://unix:/var/run/hhvm/hhvm.sock:", host: "instela.com"
2015/01/10 09:42:26 [error] 29794#0: *264 connect() to
unix:/var/run/hhvm/hhvm.sock failed (111: Connection refused) while
connecting to upstream, client: 192.168.56.211, server: , request:
"GET /api/v2/checkaccess HTTP/1.1", upstream:
"fastcgi://unix:/var/run/hhvm/hhvm.sock:", host: "instela.com"
Under high load this error occurs on all load balanced backends servers, so it's not a server specific problem (I hope)
Mostly after some time HHVM returns back but many times I had to restart the daemon to bring the server back.
I am using the default configuration mostly. Here I've my configuration: http://pastie.org/9823561
I am using Nginx as Proxy for Websocket SSL upgrade to Asterisk backend.
However at times, my users just couldn't connect to Asterisk. On Asterisk end, I do not see any connection attempt.
Thus I was looking at nginx error log and I found a lot of such error
[error] 1049#0: *28726 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 103.246.xx.xx, server: xxx.xxx.io, request: "GET / HTTP/1.1", upstream: "http://0.0.0.0:8088/ws", host: "yyy.xxx.io"
Is there any clue on how can I solve this?
Is this correct? http://0.0.0.0:8088/ws
In the Internet Protocol version 4 the address 0.0.0.0 is a non-routable
meta-address used to designate an invalid, unknown or non applicable target.
To give a special meaning to an otherwise invalid piece of data is
an application of in-band signaling.
Source: http://en.wikipedia.org/wiki/0.0.0.0