We are trying to run a Meteor application on a Debian server behind Nginx. The application is running but GET requests at http://url/sockjs?info?cb=[random_string] returns 502 Bad Gateway.
The Nginx configuration is set up as thus:
# this section is needed to proxy web-socket connections
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream app_server {
server 127.0.0.1:3000; # for a web port socket (we'll use this first)
# server unix:/var/run/app/app.sock;
}
server {
listen 80;
server_name app_server.com;
charset utf-8;
client_max_body_size 75M;
access_log /var/log/nginx/app.access.log;
error_log /var/log/nginx/app.error.log;
location / {
proxy_pass http://app_server;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Forwarded-For $remote_addr; # preserve client IP
proxy_read_timeout 60s;
keepalive_timeout 65;
proxy_redirect off;
# the root path (/) MUST NOT be cached
if ($uri != '/') {
expires 30d;
}
}
}
We have tried various configurations and could not figure out where the fault lies. Solution at Meteor WebSocket handshake error 400 with nginx didn't work either.
Edit: Tried following configuration found at recommended nginx configuration for meteor and it was still returning 502.
Edit: The application works fine when not obtaining images from Meteor CFS, which is used to store uploaded images via an admin dashboard. When loading images, a POST to domain/sockjs/img_location/cb/xhr_send causes a 502 error.
Are you sure the issue is really coming from NGINX and websocket?
First you can try wcat as a websocket CLI to ensure if the websockets are working
You can also try to run the app in a console or look at the log (debug / verbose at max level) to see if there is not an underlying error
As per your question edit, CFS uses an HTTP transport as the underlying data transfer layer.
Unfortunately, depending on how you get CFS into your stack, you might end up with an old and buggy version of its dependencies, namely cfs:http-methods, which sometimes tries to end an already ended response, which then translates itself as a cryptic error message.
Fortunately, the bug has been resolved version 0.0.30 onwards and to ensure that Meteor loads that version as the minimum dependency, all you need to do is edit you .meteor/packages file and add the following:
cfs:http-methods#0.0.30
Which will ensure any version that is equal or greater than 0.0.30, which as of this moment, is the latest on Atmosphere (meteor's package server).
Related
I'm working on web sockets in an angular app. I have it connect to a python back-end through nginx. I'm find that I'm getting 502 "Bad Gateway" errors about 90% of the time. I'll do this:
Load page in browser and connect web socket
Python back-end sends data to angular front-end
Disconnect web socket
Attempt to re-connect web socket <-- 502 Bad Gateway error
Hard-reload in Chrome
Load page in browser and connect web socket <-- No 502 error
I can't figure out why this is happening. I can't tell why I'm getting a 502 error. Nor can I figure out why doing a hard-reload fixes the problem. Things I've tried:
Increase nginx log-level to debug. Still the logs don't have any useful information.
I don't keep any web socket objects in state. I do this in case something is being cached somewhere.
I always close the web socket with close code 1000
I manually run the python service on the server so that I can watch it. When the 502 error happens, the service doesn't show anything unusual.
Setting the nginx max_fails to 0. Setting the fail_timeout to 0. Neither of these changes seems to have any effect. (I found this suggestion in other SO answers)
What should I be looking for that will help me fix this problem?
EDIT: Here's my nginx conf.d file:
server {
listen 80;
index index.html;
root /var/www/mysite;
location / {
access_log /var/log/nginx/mysite/ui.access.log;
error_log /var/log/nginx/mysite/ui.error.log;
try_files $uri $uri/ /index.html;
}
location /ws/ {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Host $proxy_host;
proxy_pass http://WEBSOCKET/;
access_log /var/log/nginx/mysite/ws_services.access.log;
error_log /var/log/nginx/mysite/ws_services.error.log;
proxy_read_timeout 300s;
}
}
upstream WEBSOCKET {
ip_hash;
server 127.0.0.1:8765;
}
Not the same problem the OP had, but just in case anyone comes across this and has the same setup as I had:
I was using WebSockets over SSL (so wss:// protocol) and had 502 popping up, even though the config had worked before. The config was as follows:
...
proxy_pass http://127.0.0.1:8080;
...
In the backend I was using a Node with the ws package to create a websocket server
As I said: It was working before, but suddenly stopped working. Additionally nginx wrote upstream prematurely closed connection while reading response header from upstream errors into the error log. I suppose that either nginx or node closed some kind of security issue, that lead to the setup not working anymore.
What I had to do in order to make it work, was to use https instead of http for the proxy_pass config
...
proxy_pass https://127.0.0.1:8080;
...
I'm trying to deploy a basic Phoenix app to a DigitalOcean server running ubuntu 14.04. I'm using exrm to generate the release. The release works when I test it on my local machine and on the server. I'm following the Phoenix guides on deployment. The thing that doesn't seem to work is the last part with the nginx server setup. For some reason I cant get it to load anyting but the default page. When I run the
nginx -t # command. It says everything is fine.
I've tried editing the /etc/nginx/sites-available files. Doesn't seem to do anything. I've tried restarting the nginx server with
sudo service nginx reload
sudo service nginx restart
But that doesn't seem to work either.
And this is the content of my /etc/nginx/sites-available/my_app.conf
upstream my_app {
server 127.0.0.1:4000;
}
server{
listen 80;
server_name www.example.com;
location / {
try_files $uri #proxy;
}
location #proxy {
include proxy_params;
proxy_redirect off;
proxy_pass http://my_app;
# The following two headers need to be set in order
# to keep the websocket connection open. Otherwise you'll see
# HTTP 400's being returned from websocket connections.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
Update: Tried connecting directly via server_ip:port, and it worked. The url still doesnt display anything.
Solved: For some reason deleting this solves the problem.:
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
Solved: For some reason deleting this solves the problem.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
I've installed GitLab 8.0.2 on a VM, and I have an nginx reverse proxy set up to direct HTTP traffic to the VM. I am able to view the main login page for GitLab, but when I try to login using the Google OAuth2 method, the callback fails to log me in after entering my correct credentials. I simply get directed back to the GitLab login page.
Where might the problem be? The reverse proxy settings? GitLab settings (ie. Google OAuth config)?
Below is my nginx conf:
upstream gitlab {
server 192.168.122.134:80;
}
server {
listen 80;
server_name myserver.com;
access_log /var/log/nginx/gitlab.access.log;
error_log /var/log/nginx/gitlab.error.log;
root /dev/null;
## send request back to gitlab ##
location / {
proxy_pass http://gitlab;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Interestingly, the old setup I had used iptables to redirect port 81 on the host machine to port 80 on the GitLab VM, and, in that case, the Google OAuth callback worked. I'd prefer to have people simply use standard port 80 for accessing my GitLab instance, though, so I want this reverse proxy method to work.
GitLab 8.x has quite a few new things. Although I don't see anything specifically wrong with your nginx.conf file, it is pretty short compared to the example in the GitLab repository. Look through https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab-ssl to get an idea of the configuration you should consider adding.
Once your nginx.conf file is updated, read through GitLab OmniAuth documentation and the Google OAuth2 integration documentation under 'Providers' on that OmniAuth page. Make sure you provide the correct callback URL to Google when registering.
I'm having difficulty deploying my meteor app ("myApp" below) into production using meteor-up with https and NGINX as a proxy. In particular, I think I am having trouble configuring the correct ports and/or paths.
The deployment has worked in most respects. It is running on a digital ocean droplet with a mongohq (now compose.io) database. My mup setup, mup reconfig (run now many times on my mup.json file) and mup deploy commands with meteor-up all report no errors. If I ssh into my ubuntu environment on digital ocean and run status myApp it reports myApp start/running, process 10049, and when I check my mongohq database, I can see the expected collections for myApp were created and seeded. I think on this basis that the app is running properly.
My problem is that I cannot locate it visiting the site, and having no experience with NGINX servers, I cannot tell if I am doing something very basic and wrong setting up the ports and forwarding.
I have reproduced the relevant parts of my NGINX config file and mup.json file below.
The behavior I expected with the setup below is that if my meteor app listens on port 3000 in mup.json the app should appear when I visit the site. In fact, if I set mup.json's env.PORT to 3000, when visiting the site my browser tells me there is a redirect loop. If I change mup's env.PORT to 80, or leave the env.PORT out entirely, I receive a 502 Bad Gateway message - this part is to be expected because myApp should be listening on localhost:3000 and I wouldn't expect to find anything anywhere else.
All help is MUCH appreciated.
MUP.JSON (in relevant part, lmk if more needs to be shown)
"env": {
"PORT": 3000,
"NODE_ENV": "production",
"ROOT_URL": "http://myApp.com",
"MONGO_URL": // working ok, not reproduced here,
"MONGO_OPLOG_URL": // working ok I think,
"MAIL_URL": // working ok
}
NGINX
server_tokens off;
# according to a digital ocean guide i followed here, https://www.digitalocean.com/community/tutorials/how-to-deploy-a-meteor-js-application-on-ubuntu-14-04-with-nginx, this section is needed to proxy web-socket connections
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
# HTTP
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
server_name myApp.com;
# redirect non-SSL to SSL
location / {
rewrite ^ https://$server_name$request_uri? permanent;
}
}
# HTTPS
server {
listen 443 ssl spdy;
# this domain must match Common Name (CN) in the SSL certificate
server_name myApp.com;
root html;
index index.html index.htm;
ssl_certificate /etc/nginx/ssl/tempcert.crt;
ssl_certificate_key /etc/nginx/ssl/tempcert.key;
ssl_stapling on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'long string I didn't reproduce here'
add_header Strict-Transport-Security "max-age=31536000;";
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
Also note that the SSL certificates are configured and work fine so I think it is something with how the ports, paths and forwarding is configured. I don't know where the redirect loop is coming from.
For anyone coming across this in the future, I was able to solve things by removing the force-ssl package from my bundled meteor app. Apparently force-ssl and an NGINX proxy are either redundant or if used together can cause too many redirects. This was not well-documented in the materials I was able to locate.
If there is a configuration that supports using force-ssl together with a proxy that serves some purpose and is preferable to removing the package altogether, please post as I would be interested to know. Thanks.
I believe you can keep the force-ssl package as long as you add the X-Forward-Proto header to your Nginx config.
Example:
proxy_set_header X-Forward-Proto https;
Additionally, make sure you have the X-Forward-For set as well, though that's already in the example you posted.
Source
As the documentation of the force-ssl package says , you have to set the x-forwarded-proto header to https :
So your location field in the nginx configuration will be like :
location / {
#your own config...
proxy_set_header X-Forwarded-Proto https;
}
I'm running meteor behind an NGinx proxy. I got the error about too many redirects after installing force-ssl.
What worked to remove force-ssl and then add the following lines to my location in my nginx config:
proxy_set_header X-Forward-Proto https;
proxy_set_header X-Nginx-Proxy true;
Works perfectly now.
I'm using Nginx as a proxy to filter requests to my application. With the help of the "http_geoip_module" I'm creating a country code http-header, and I want to pass it as a request header using "headers-more-nginx-module". This is the location block in the Nginx configuration:
location / {
proxy_pass http://mysite.com;
proxy_set_header Host http://mysite.com;;
proxy_pass_request_headers on;
more_set_headers 'HTTP_Country-Code: $geoip_country_code';
}
But this only sets the header in the response. I tried using "more_set_input_headers" instead of "more_set_headers" but then the header isn't even passed to the response.
What am I missing here?
If you want to pass the variable to your proxy backend, you have to set it with the proxy module.
location / {
proxy_pass http://example.com;
proxy_set_header Host example.com;
proxy_set_header HTTP_Country-Code $geoip_country_code;
proxy_pass_request_headers on;
}
And now it's passed to the proxy backend.
The problem is that '_' underscores are not valid in header attribute. If removing the underscore is not an option you can add to the server block:
underscores_in_headers on;
This is basically a copy and paste from #kishorer747 comment on #Fleshgrinder answer, and solution is from: https://serverfault.com/questions/586970/nginx-is-not-forwarding-a-header-value-when-using-proxy-pass/586997#586997
I added it here as in my case the application behind nginx was working perfectly fine, but as soon ngix was between my flask app and the client, my flask app would not see the headers any longer. It was kind of time consuming to debug.
You may pass ALL headers by adding this:
ignore_invalid_headers off;
But please consider security issues by doing this.