I'm using nginx 1.12.2 (downloaded from official site) on windows 8.1 as a reverse proxy server. I got a problem that one of every two request is ignored.
nginx config:
server {
listen 80;
server_name my_fake_domain.com;
access_log C:/nginxlogs/access.txt;
location / {
proxy_pass http://localhost:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 1800;
proxy_connect_timeout 1800;
}
}
my nodejs server:
'use strict';
const http = require('http');
const httpServer = http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('okay');
});
httpServer.listen(8080, '0.0.0.0');
I'm using postman to send test requests. A request to http://my_fake_domain.com after a succeeded request will always loading. This always loading request does not actually ignored. If I cancel it, it will show up in the access log.
Note that a request to http://localhost:8080 always succeed.
Where might I get wrong?
Related
Hi everybody I have a problem with my reverse proxy configuration for my dotnet core application. My problem is that when I am using identityserver discovery endpoint the port number is missing from the end of my url.
I have a request for the discorvery document to
https://:8421/.well-known/openid-configuration
and the response is the following and the problem here is that the clients of the authentication service are using the document as a base for verification, using jwks_uri property from the response, the url listed in that property has the missing port number, so the clients cant call the openId configuration.
{
"issuer": "https://<servername>",
"jwks_uri": "https://<servername>/.well-known/openid-configuration/jwks",
"authorization_endpoint": "https://<servername>/connect/authorize",
"token_endpoint": "https://<servername>/connect/token",
"userinfo_endpoint": "https://<servername>/connect/userinfo",
"end_session_endpoint": "https://<servername>/connect/endsession",
"check_session_iframe": "https://<servername>/connect/checksession",
"revocation_endpoint": "https://<servername>/connect/revocation",
"introspection_endpoint": "https://<servername>/connect/introspect",
"device_authorization_endpoint": "https://<servername>/connect/deviceauthorization"
}
expected result for endpoint config:
{
"jwks_uri": "https://<servername>:<port>/.well-known/openid-configuration/jwks",
}
In my dotnet app I setup the reverse proxy settings
services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders =
ForwardedHeaders.XForwardedFor |
ForwardedHeaders.XForwardedHost |
ForwardedHeaders.XForwardedProto;
options.ForwardLimit = 2; //Limit number of proxy hops trusted
options.KnownNetworks.Clear();
options.KnownProxies.Clear();
});
app.UseForwardedHeaders();
nginx config is the following
server {
listen <ip>:8421 ssl;
server_name _;
ssl_certificate <certPath> ;
ssl_certificate_key <certPath>;
location / {
proxy_pass http://127.0.0.1:8421;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 8421;
}
}
Before updateing to Identityserver 4 4.0 it had the Public origin property but it became deprecated with 4.0. Now instead it is using the dotnet baseurl.
In the http pipeline I also logged all the headers from the request and got this result:
2020-07-03 11:40:15.109 +00:00;[INF];Cache-Control--no-cache Connection--upgrade Accept--*/* Accept-Encoding--gzip, deflate, br Host--<hostName> User-Agent--PostmanRuntime/7.26.1 X-Real-IP--<ip> X-Original-Proto--http X-Forwarded-Host--<domainName with port> X-Forwarded-Port--8421 Postman-Token--44dc6573-71eb-4b36-8b2a-9768d71e5b64 X-Original-For--<ip address> ;;
I had to add the port also to the nginx configuration. Setting the X-Forwarded-Host to $host was not enough also had to add the $proxy_port as well
The working confiugration for proxy
proxy_set_header X-Forwarded-Host $host:$proxy_port;
If the host is listening on non-default ports (not 80 for http and not 443 for https), use $http_host instead of $host to set the Host as shown below.
Change
change proxy_set_header Host $host;
to
proxy_set_header Host $http_host;
im trying to implement a system that comprise of nginx , daphne and gunicorn. So far , i have gotten gunicorn to work , however im facing an issue connecting the websockets for daphne. This issue does not arise when im in developing mode , only when i utilize nginx. This is the error code from my console :
reconnecting-websocket.js:199 WebSocket connection to 'ws://192.168.8.31/' failed: Error during WebSocket handshake: Unexpected response code: 200
This is my nginx config file :
upstream crm_server { server unix:/home/user/project/venv/run/gunicorn.sock fail_timeout=0;
}
upstream channels-backend {
server localhost:8001;
}
server {
listen 80;
# add here the ip address of your server
# or a domain pointing to that ip (like example.com or www.example.com)
server_name 192.168.8.31;
keepalive_timeout 5;
client_max_body_size 4G;
access_log /home/user/project/venv/logs/nginx-access.log;
error_log /home/user/project/venv/logs/nginx-error.log;
location /staticfiles/ {
root /home/user/roject/crm/staticfiles/;
}
# checks for static file, if not found proxy to app
location / {
try_files $uri #proxy_to_app; }
location /ws/ {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
include proxy_params;
proxy_pass http://channels-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_redirect off;
proxy_pass http://crm_server;
}
This is my front end thats generating the socket
<script type="text/javascript">
var loc = window.location
var wsStart = 'ws://'
if (loc.protocol == 'https'){
wsStart = 'wss://' } var endpoint = wsStart + loc.host + loc.pathname var socket = new ReconnectingWebSocket(endpoint)
This is my routing:
from channels.routing import ProtocolTypeRouter , URLRouter
from django.urls import path
from rnd.consumers import NotificationConsumer
from django.conf.urls import url
from channels.auth import AuthMiddlewareStack
from channels.security.websocket import AllowedHostsOriginValidator, OriginValidator
application = ProtocolTypeRouter({
"websocket": AllowedHostsOriginValidator(
AuthMiddlewareStack(
URLRouter(
[
path('',NotificationConsumer),
] ) )
) })
I would greatly appreciate any form of feed back!
When you proxy pass to open a Web-socket connection you need to proxy pass all of the web-socket http headers. https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism#WebSocket-specific_headers you will also need to pass the Origin header since you are using AllowedHostsOriginValidator.
--
also to debug these I suggest using a api client directly rather than the browser. Paw or Insomnia can be used to hit api to check that your Nginx config is routing to channels and not to Gunicorn.
I'm trying to reverse proxy an api with Nginx. I have the following configuration:
worker_processes 4;
events { worker_connections 1024; }
http {
upstream some_upstream {
server 1.something.com;
server 2.something.com;
}
server {
listen 80;
location ~/proxyNow/(?<zvar>(\w+))/(?<xvar>(\w+))/(?<yvar>(\w+))/ {
proxy_pass http://some_upstream/hello/something/$zvar/$xvar/$yvar/somethingelese;
proxy_set_header Host $http_host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_cache maps_cache;
proxy_cache_valid 200 302 365d;
proxy_cache_valid 404 1m;
proxy_redirect off;
}
}
}
When I try to call the following url http://localhost:82/proxyNow/1/2/3/?app_id=someAppId&app_code=someCode
I get the following error message:
Invalid URL
The requested URL
"http://%5bNo%20Host%5d/hello/something/1/2/3/somethingelese", is
invalid. Reference #9.be35dd58.1489086561.5c9bd3c
It seems that the host cannot be retrieved by nginx. But if I execute the call directly:
http://1.something.com/hello/something/$zvar/$xvar/$yvar/somethingelese?app_id=someAppId&app_code=someCode
http://2.something.com/hello/something/$zvar/$xvar/$yvar/somethingelese?app_id=someAppId&app_code=someCode
It seems that Nginx for some reason is not able to resolve the host
You should take a look into the doc.
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
When location is specified using a regular expression.
In this case, the directive should be specified without a URI.
I suggest the following solution without expensive regex location.
http://nginx.org/en/docs/http/ngx_http_core_module.html#location
location /proxyNow/ {
rewrite /proxyNow/(?<zvar>(\w+))/(?<xvar>(\w+))/(?<yvar>(\w+))/.* /hello/something/$zvar/$xvar/$yvar/somethingelese$is_args?$args break;
proxy_pass http://some_upstream;
... other nginx diretives;
}
I went to similar questions but without any success.
Let say I have two node.js app turning on a server:
// App GoodMorning
var express = require('express');
app.post('/breakfast', function (req, res) {
console.log("Eating breakfast");
res.sendStatus(200);
});
app.get('/', function (req, res) {
res.send('GoodMorning');
});
app.listen(3000, function () {
console.log('GoodMorning app listening on port 3000!');
});
and
// App GoodEvening
var express = require('express');
app.post('/diner', function (req, res) {
console.log("Eating diner");
res.sendStatus(200);
});
app.get('/', function (req, res) {
res.send('GoodEvening');
});
app.listen(4000, function () {
console.log('GoodEvening app listening on port 4000!');
});
And let's say Nginx is used as a reverse proxy server. So it has to send requests to the correct port, right ? So the "magic" file is like that:
# HTTP - redirect all requests to HTTPS:
server {
listen 80;
listen [::]:80 default_server ipv6only=on;
return 301 https://$host$request_uri;
}
# HTTPS - proxy requests on to local Node.js app:
server {
listen 443;
server_name iamhungry.com;
ssl on;
# Use certificate and key provided by Let's Encrypt:
ssl_certificate /etc/letsencrypt/live/iamhungry.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/iamhungry.com/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
# Pass requests for / to localhost:3000:
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:3000/;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_cache_bypass $http_upgrade;
proxy_redirect off;
}
# Pass requests for /homepageevening to localhost:4000:
location /homepageevening {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:4000/;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_cache_bypass $http_upgrade;
proxy_redirect off;
}
# Pass requests for /diner to localhost/diner:4000:
location /diner {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost/diner:4000/;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_cache_bypass $http_upgrade;
proxy_redirect off;
}
}
Then the following requests do the following results:
$ curl iamhungry.com
$ GoodMorning // OK
$ curl -X POST iamhungry.com/breakfast
--> I see "Eating brakfast" in the log file of breakfast.js // OK
$ curl iamhungry.com/homepageevening
$ GoodEvening // OK
$ curl -X POST iamhungry.com/diner -I
HTTP/1.1 301 Moved Permanently // why ?!
--> And I see nothing in the log file of evening.js // why ?!
I am not at ease with these proxy concepts. I went through the documentation of nginx and I found no help about that. I wonder if my way of understanding the way it works is correct.
OK thank you #RichardSmith. I had to fix the configuration file:
location /diner {
...
proxy_pass http://localhost:4000/diner;
And do my tests with curl http://iamhungry.com -I -L instead of curl http://iamhungry.com -I to actually follow the rerouting.
Here is what I was missing:
A 301 is not an error. I was doing my test in the terminal using curl http://iamhungry.com -I but using the option -L curl http://iamhungry.com -I -L allowed me to follow the redirection and then get the end of the line ! So 301 is in fact normal using nginx because redirecting is its role.
Port number is attached to the domain name. Thanks #RichardSmith.
from #RichardSmith link: [...]the part of a normalized request URI matching the location is replaced by a URI specified in the directive.
I'm using Go (Golang) 1.4.2 with Gorilla WebSockets behind an nginx 1.4.6 reverse proxy. My WebSockets are disconnecting after about a minute of having the page open. Same behavior occurs on Chrome and Firefox.
At first, I had problems connecting the server and client with WebSockets. Then, I read that I needed to tweak my nginx configuration. This is what I have.
server {
listen 80;
server_name example.com;
proxy_pass_header Server;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forward-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://127.0.0.1:1234;
}
}
My Go code is basically echoing back the client's message. (Errors omitted for brevity). This is my HandleFunc.
var up = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
ws, _ := up.Upgrade(resp, req, nil)
defer ws.Close()
var s struct {
Foo string
Bar string
}
for {
ws.ReadJSON(&s)
ws.WriteJSON(s)
}
The JavaScript is pretty simple as well.
var ws = new WebSocket("ws://example.com/ws/");
ws.addEventListener("message", function(evnt) {
console.log(JSON.parse(evnt.data));
});
var s = {
Foo: "hello",
Bar: "world"
};
ws.send(JSON.stringify(s));
Go is reporting websocket: close 1006 unexpected EOF. I know that when I leave or refresh the page ReadJSON returns EOF, but this appears to be a different error. Also, the unexpected EOF happens by itself after about a minute of having the page open.
I have an onerror function in JavaScript. That event doesn't fire, but onclose fires instead.
I had the same issue, the problem is the nginx configuration. It defaults to a 1 minute read timeout for proxy_pass:
Syntax: proxy_read_timeout time;
Default: proxy_read_timeout 60s;
Context: http, server, location
See http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout
In my case I've increased the timeout to 10 hours:
proxy_read_timeout 36000s;