When I upload a 10M file to server, Nginx returns a 501 error. But smaller file is uploaded OK.
<html>
<head><title>501 Not Implemented</title></head>
<body bgcolor="white">
<center><h1>501 Not Implemented</h1></center>
<hr><center>nginx/1.8.1</center>
</body>
</html>
access.log
[01/Mar/2017:10:13:29 +0800] "POST /boss/cgi/importemoji HTTP/1.1" 501
582
The Nginx config file is
http {
include mime.types;
#default_type application/octet-stream;
default_type text/plain;
access_log logs/access.log main;
#access_log off;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 30;
keepalive_requests 100;
gzip on;
#gzip_disable msie6;
proxy_max_temp_file_size 0;
proxy_buffer_size 20M;
proxy_buffers 4 20M;
#mail_spam add url, host will be mod
#server_name_in_redirect off;
proxy_connect_timeout 60;
proxy_read_timeout 120;
proxy_send_timeout 120;
client_header_buffer_size 20M;
client_max_body_size 80M;
client_body_buffer_size 60M;
client_body_temp_path /usr/local/qspace/nginx/client_body_temp;
client_header_timeout 1m;
client_body_timeout 1m;
server_names_hash_max_size 1024;
server_names_hash_bucket_size 1024;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-Port $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header PROXY_FORWARDED_FOR "disabled";
server {
listen 80;
listen 443 ssl;
keepalive_timeout 70;
...
}
}
error.log
2017/03/01 10:41:10 [debug] 6167#6167: *132144 __mydebug. menshen cleanup r: 00000000011DD720
2017/03/01 10:41:11 [debug] 6171#6171: *132871 __mydebug_menshen. ngx_http_dummy_payload_handler wait_for_body: yes
2017/03/01 10:41:11 [debug] 6171#6171: *132871 status: unkown. uri: /boss/cgi/importemoji args: r: 00000000011DD720 r->main: 00000000011DD720 r->count: 1
2017/03/01 10:41:11 [debug] 6171#6171: *132871 __mydebug_menshen. ngx_http_menshen_handler is called r: 00000000011DD720 nginx_version: 1008001
2017/03/01 10:41:11 [debug] 6171#6171: *132871 status: send. uri: /boss/cgi/importemoji. args: r: 00000000011DD720 r->main: 00000000011DD720 r->count: 1
2017/03/01 10:41:11 [debug] 6171#6171: *132871 __mydebug_menshen. len: 3078 header:
POST /boss/cgi/importemoji HTTP/1.1
Proxy-Connection:keep-alive
Content-Length:9465320
Pragma:no-cache
Cache-Control:no-cache
Accept:application/json, text/javascript, */*; q=0.01
I try to upload the 10M file using CURL directly to the server, It is Ok. So the problem probably arises from Nginx.
How can I fix the bug?
I have solved the problem. Generally, there exist these points when encountering uploading large file error on Nginx
file size limit. revise client_max_body_size.
keep-alive connections timeout. revise keepalive_timeout
reverse proxy timeout. revise proxy_connect_timeout
and at last you must make sure that Nginx is Native. In this case, My company compile a module called Menshen acted as firewall to Nginx. It only lets pass the file upload request less than 8M.
Related
I am trying to upload files from a client through an nginx ingress. I have set the following annotations on the ingress after receiving a 413 response;
Annotations: nginx.ingress.kubernetes.io/body-size: 1024m
nginx.ingress.kubernetes.io/client-body-buffer-size: 50m
nginx.ingress.kubernetes.io/client-max-body-size: 50m
nginx.ingress.kubernetes.io/proxy-body-size: 1024m
nginx.ingress.kubernetes.io/proxy-buffer-size: 32k
nginx.ingress.kubernetes.io/proxy-buffers-number: 8
The client is an Angular application. It sends a base64 string of the file in the request body. I have tried uploading images of a few KB, so I definitely ain't hitting any of these limits. I'm new to Kubernetes. Do I need to restart the ingress for these annotations to take effect?
I have also tried creating a ConfigMap;
apiVersion: v1
kind: ConfigMap
metadata:
name: app-ingress-configuration
namespace: development
labels:
app.kubernetes.io/name: [name of ingress]
app.kubernetes.io/part-of: [name of ingress]
data:
proxy-connect-timeout: "50"
proxy-read-timeout: "120"
proxy-send-timeout: "120"
body-size: "1024m"
client-body-buffer-size: "50m"
client-max-body-size: "50m"
proxy-body-size: "1024m"
proxy-buffers: "8 32k"
proxy-buffer-size: "32k"
Still getting a 502.
Not sure how to access the nginx.conf through kubectl, it seems from the docs if I update this ConfigMap the settings are changed in nginx anyway.
Any help appreciated.
UPDATE
nginx.conf
# Configuration checksum: 1961171210939107273
# setup custom paths that do not require root access
pid /tmp/nginx.pid;
daemon off;
worker_processes 2;
worker_rlimit_nofile 523264;
worker_shutdown_timeout 240s ;
events {
multi_accept on;
worker_connections 16384;
use epoll;
}
http {
client_max_body_size 100M;
lua_package_path "/etc/nginx/lua/?.lua;;";
lua_shared_dict balancer_ewma 10M;
lua_shared_dict balancer_ewma_last_touched_at 10M;
lua_shared_dict balancer_ewma_locks 1M;
lua_shared_dict certificate_data 20M;
lua_shared_dict certificate_servers 5M;
lua_shared_dict configuration_data 20M;
init_by_lua_block {
collectgarbage("collect")
-- init modules
local ok, res
ok, res = pcall(require, "lua_ingress")
if not ok then
error("require failed: " .. tostring(res))
else
lua_ingress = res
lua_ingress.set_config({
use_forwarded_headers = false,
use_proxy_protocol = false,
is_ssl_passthrough_enabled = false,
http_redirect_code = 308,
listen_ports = { ssl_proxy = "442", https = "443" },
hsts = true,
hsts_max_age = 15724800,
hsts_include_subdomains = true,
hsts_preload = false,
})
end
ok, res = pcall(require, "configuration")
if not ok then
error("require failed: " .. tostring(res))
else
configuration = res
end
ok, res = pcall(require, "balancer")
if not ok then
error("require failed: " .. tostring(res))
else
balancer = res
end
ok, res = pcall(require, "monitor")
if not ok then
error("require failed: " .. tostring(res))
else
monitor = res
end
ok, res = pcall(require, "certificate")
if not ok then
error("require failed: " .. tostring(res))
else
certificate = res
end
ok, res = pcall(require, "plugins")
if not ok then
error("require failed: " .. tostring(res))
else
plugins = res
end
-- load all plugins that'll be used here
plugins.init({})
}
init_worker_by_lua_block {
lua_ingress.init_worker()
balancer.init_worker()
monitor.init_worker()
plugins.run()
}
geoip_country /etc/nginx/geoip/GeoIP.dat;
geoip_city /etc/nginx/geoip/GeoLiteCity.dat;
geoip_org /etc/nginx/geoip/GeoIPASNum.dat;
geoip_proxy_recursive on;
aio threads;
aio_write on;
tcp_nopush on;
tcp_nodelay on;
log_subrequest on;
reset_timedout_connection on;
keepalive_timeout 75s;
keepalive_requests 100;
client_body_temp_path /tmp/client-body;
fastcgi_temp_path /tmp/fastcgi-temp;
proxy_temp_path /tmp/proxy-temp;
ajp_temp_path /tmp/ajp-temp;
client_header_buffer_size 1M;
client_header_timeout 60s;
large_client_header_buffers 4 5M;
client_body_buffer_size 1M;
client_body_timeout 60s;
http2_max_field_size 1M;
http2_max_header_size 5M;
http2_max_requests 1000;
http2_max_concurrent_streams 128;
types_hash_max_size 2048;
server_names_hash_max_size 1024;
server_names_hash_bucket_size 64;
map_hash_bucket_size 64;
proxy_headers_hash_max_size 512;
proxy_headers_hash_bucket_size 64;
variables_hash_bucket_size 256;
variables_hash_max_size 2048;
underscores_in_headers off;
ignore_invalid_headers on;
limit_req_status 503;
limit_conn_status 503;
include /etc/nginx/mime.types;
default_type text/html;
gzip on;
gzip_comp_level 5;
gzip_http_version 1.1;
gzip_min_length 256;
gzip_types application/atom+xml application/javascript application/x-javascript application/json application/rss+xml application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/svg+xml image/x-icon text/css text/javascript text/plain text/x-component;
gzip_proxied any;
gzip_vary on;
# Custom headers for response
server_tokens on;
# disable warnings
uninitialized_variable_warn off;
# Additional available variables:
# $namespace
# $ingress_name
# $service_name
# $service_port
log_format upstreaminfo '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id';
map $request_uri $loggable {
default 1;
}
access_log /var/log/nginx/access.log upstreaminfo if=$loggable;
error_log /var/log/nginx/error.log notice;
resolver 10.245.0.10 valid=30s;
# See https://www.nginx.com/blog/websocket-nginx
map $http_upgrade $connection_upgrade {
default upgrade;
# See http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
'' '';
}
# Reverse proxies can detect if a client provides a X-Request-ID header, and pass it on to the backend server.
# If no such header is provided, it can provide a random value.
map $http_x_request_id $req_id {
default $http_x_request_id;
"" $request_id;
}
# Create a variable that contains the literal $ character.
# This works because the geo module will not resolve variables.
geo $literal_dollar {
default "$";
}
server_name_in_redirect off;
port_in_redirect off;
ssl_protocols TLSv1.2;
ssl_early_data off;
# turn on session caching to drastically improve performance
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_session_timeout 10m;
# allow configuring ssl session tickets
ssl_session_tickets on;
# slightly reduce the time-to-first-byte
ssl_buffer_size 4k;
# allow configuring custom ssl ciphers
ssl_ciphers '';
ssl_prefer_server_ciphers on;
ssl_ecdh_curve auto;
# PEM sha: ---
ssl_certificate /etc/ingress-controller/ssl/default-fake-certificate.pem;
ssl_certificate_key /etc/ingress-controller/ssl/default-fake-certificate.pem;
proxy_ssl_session_reuse on;
upstream upstream_balancer {
### Attention!!!
#
# We no longer create "upstream" section for every backend.
# Backends are handled dynamically using Lua. If you would like to debug
# and see what backends ingress-nginx has in its memory you can
# install our kubectl plugin https://kubernetes.github.io/ingress-nginx/kubectl-plugin.
# Once you have the plugin you can use "kubectl ingress-nginx backends" command to
# inspect current backends.
#
###
server 0.0.0.1; # placeholder
balancer_by_lua_block {
balancer.balance()
}
keepalive 32;
keepalive_timeout 60s;
keepalive_requests 100;
}
# Cache for internal auth checks
proxy_cache_path /tmp/nginx-cache-auth levels=1:2 keys_zone=auth_cache:10m max_size=128m inactive=30m use_temp_path=off;
# Global filters
## start server _
server {
server_name _ ;
listen 80 default_server reuseport backlog=511 ;
listen [::]:80 default_server reuseport backlog=511 ;
listen 443 default_server reuseport backlog=511 ssl http2 ;
listen [::]:443 default_server reuseport backlog=511 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "";
set $ingress_name "";
set $service_name "";
set $service_port "";
set $location_path "/";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = false,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
access_log off;
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "upstream-default-backend";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1m;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
proxy_buffer_size 5M;
proxy_buffers 4 5M;
proxy_max_temp_file_size 1024M;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
# health checks in cloud providers require the use of port 80
location /healthz {
access_log off;
return 200;
}
# this is required to avoid error if nginx is being monitored
# with an external software (like sysdig)
location /nginx_status {
allow 127.0.0.1;
allow ::1;
deny all;
access_log off;
stub_status on;
}
}
## end server _
## start server dev-api
server {
server_name dev-api ;
listen 80 ;
listen [::]:80 ;
listen 443 ssl http2 ;
listen [::]:443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "development";
set $ingress_name "app-ingress";
set $service_name "app-api-svc";
set $service_port "80";
set $location_path "/";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
}
log_by_lua_block {
balancer.log()
monitor.call()
plugins.run()
}
port_in_redirect off;
set $balancer_ewma_score -1;
set $proxy_upstream_name "development-app-api-svc-80";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
set $proxy_alternative_upstream_name "";
client_max_body_size 1024M;
client_body_buffer_size 50M;
proxy_set_header Host $best_http_host;
# Pass the extracted client certificate to the backend
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;
proxy_set_header X-Scheme $pass_access_scheme;
# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
proxy_connect_timeout 50s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
proxy_buffer_size 5M;
proxy_buffers 8 5M;
proxy_max_temp_file_size 1024M;
proxy_request_buffering on;
proxy_http_version 1.1;
proxy_cookie_domain off;
proxy_cookie_path off;
# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
}
## end server dev-api
.......
UPDATE 2
Log from kubectl logs -n nginx-ingress-controller-XXX command
127.0.0.1 - - [16/Jul/2020:10:11:14 +0000] "POST [Ingress/Service endpoint] HTTP/2.0" 502 4 "https://[client-host-name]/[client-path]" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 Edg/83.0.478.58" 9351 0.659 [service-name-80] [] 10.244.1.72:80 14 0.652 502 7b7bdf8a9319e88c80ba3444372daf2d
You need to make sure you have the file size set up on the ingress controller. Nginx will catch up the settings. try this. For more information about the annotations follow this. https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: service-api-tls-ingress
namespace: production
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-body-size: 8m
My original issue was nginx, but after I changed the limits it was forwarding the request to the service but I wasn't checking the correct logs. So #mWatney was correct to double check directly to the service/pod.
For anyone else the issue I was seeing was to do with running a .NET core 3.1 application in the Linux alpine container. Within the app I was using a version of System.Drawing.Common which causes exceptions running under Linux, whihc look like this;
System.TypeInitializationException: The type initializer for 'Gdip' threw an exception.
---> System.DllNotFoundException: Unable to load shared library 'libgdiplus'
The solution was to add to the dockerfile;
RUN apk add libgdiplus-dev fontconfig ttf-dejavu --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
This allows the use of System.Drawing.Common under Linux by adding the ability to load shared library libgdiplus.
Credit here: https://github.com/dotnet/dotnet-docker/issues/618#issuecomment-467619498
The more permanent solution is to remove the dependency on System.Drawing.Common entirely from the application. Thanks #mWatney for help, you got me on the right track.
I am using Nginx as reverse proxy to my backend (Java app with Spring boot). In overall (avg, p50, p90, p95, p99 latencies) it performs well. But time to time, I see latency spikes around 100-200 milliseconds. When I enabled the access logs, I see that upstream response time (upstream_response_time) is very low even though request time (request_time) is high. For example,
[25/Apr/2020:18:28:17 +0000] "XXX" XXX - request="POST /v1/composite-monitoring-data HTTP/1.1" status=429 request_time=0.081 trace_id="Root=1-5ea48141-2f8e07a4c7c71a1360d9c5f5" request_length=9864 bytes_sent=979 body_bytes_sent=623 upstream_addr=127.0.0.1:5000 upstream_status=429 upstream_response_time=0.004 upstream_connect_time=0.000 upstream_header_time=0.004 user_agent="okhttp/3.10.0" current_time_msec=1587839297.256
...
[25/Apr/2020:18:28:17 +0000] "XXX" XXX - request="POST /v1/composite-monitoring-data HTTP/1.1" status=429 request_time=0.084 trace_id="Root=1-5ea48141-51f0d12a6f7c4b0651f6ef42" request_length=20534 bytes_sent=979 body_bytes_sent=623 upstream_addr=127.0.0.1:5000 upstream_status=429 upstream_response_time=0.000 upstream_connect_time=0.000 upstream_header_time=0.000 user_agent="okhttp/3.10.0" current_time_msec=1587839297.278
Also here is my nginx.conf file:
user nginx;
pid /var/run/nginx.pid;
error_log /var/log/nginx/error.log;
worker_processes auto;
worker_rlimit_nofile 32768;
events {
worker_connections 4096;
use epoll;
multi_accept on;
}
http {
include /etc/nginx/mime.types;
include /etc/nginx/conf.d/*.conf;
default_type application/json;
sendfile on;
tcp_nopush off;
tcp_nodelay on;
keepalive_timeout 300;
keepalive_requests 10000;
client_body_timeout 15;
client_header_timeout 15;
client_body_buffer_size 4m;
client_max_body_size 4m;
log_format main '[$time_local] "$http_x_forwarded_for" $remote_addr - '
'request="$request" status=$status request_time=$request_time trace_id="$http_x_amzn_trace_id" '
'request_length=$request_length bytes_sent=$bytes_sent body_bytes_sent=$body_bytes_sent '
'upstream_addr=$upstream_addr '
'upstream_status=$upstream_status '
'upstream_response_time=$upstream_response_time '
'upstream_connect_time=$upstream_connect_time '
'upstream_header_time=$upstream_header_time '
'user_agent="$http_user_agent" '
'current_time_msec=$msec';
access_log /var/log/nginx/access.log main;
upstream http_backend {
server 127.0.0.1:5000;
keepalive 1024;
}
server {
listen 80;
listen [::]:80;
server_name _ localhost;
location /v1 {
proxy_pass http://http_backend/v1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Request-Start $msec;
proxy_set_header Connection "";
proxy_http_version 1.1;
keepalive_timeout 300;
keepalive_requests 10000;
}
location /ping {
proxy_pass http://http_backend/ping;
}
}
}
What might cause this big difference between the request time and upstream response time? Is there anything I need to configure and not configured properly?
I have an Odoo instance on a VPS behind a nginx reverse proxy. When I access the Odoo instance directly at the VPS IP:8069 or domain:8069 then the page is displayed correctly and there are no asset errors. If I try to access through the proxy, using ip or domain name (default port 80), some javascript resources don't load and/or images are truncated.
nginx.conf
user nobody;
worker_processes 1;
error_log logs/error.log;
worker_rlimit_nofile 8192;
events {
worker_connections 4096; ## Default: 1024
}
http{
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
server {
listen 80;
server_name www.mysite.com;
rewrite ^(.*) https://$host$1 permanent;
}
server {
listen 443 ssl http2;
server_name www.mysite.com;
proxy_read_timeout 720s;
proxy_connect_timeout 720s;
proxy_send_timeout 720s;
# Add Headers for odoo proxy mode
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
#SSL parameters
ssl on;
ssl_certificate /etc/mycert/location/fullcert.pem;
ssl_certificate_key /etc/mycert/location/pkey.pem;
ssl_session_timeout 30m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_tickets off;
#log
access_log /var/log/nginx/odoo.access.log;
error_log /var/log/nginx/odoo.error.log;
location ~* / {
proxy_redirect off;
proxy_pass http://odoo;
}
location /longpolling {
proxy_pass http://odoo-chat;
}
# gzip
gzip_types text/css text/less text/plain text/xml application/xml application/json application/javascript;
gzip on;
}
upstream odoo {
server 127.0.0.1:8069;
}
upstream odoo-chat {
server 127.0.0.1:8072;
}
}
I've tried Odoo 12 and 13 with Python 3.6 based on some Python errors that were mentioned in a forum, no change. I have proxy = true in my odoo.conf file.
2019/10/21 11:56:46 [crit] 2803#0: *1 open() "/var/lib/nginx/tmp/proxy/1/00/0000000001" failed (13: Permission denied) while reading upstream, client: 99.99.254.254, server: www.example.com, request: "GET /web/image/366?access_token=ff2c51a8-716b-448f-98b7-85b97349d39a HTTP/1.1", upstream: "http://127.0.0.1:8069/web/image/366?access_token=ff2c51a8-716b-448f-98b7-85b97349d39a", host: "example.com", referrer: "https://example.com/"
I experienced this problem just now. This is because nginx is dynamically creating temporary files in /var/lib/nginx/tmp/proxy and chown-ing will not solve the issue. What I learned was to set proxy_temp_path:
create a directory wherein user nobody has permissions: e.g. mkdir /home/nobody/tmp
Set proxy_temp_path in nginx.config: proxy_temp_path /home/nobody/tmp 1 2;
An excerpt of config is shown below:
location ~* / {
proxy_temp_path /home/nobody/tmp 1 2;
proxy_redirect off;
proxy_pass http://odoo;
}
This is one step ahead of this article,
Below is the current flow and configuration and it is working. I can access UI from /16 machine. This was done by someone else. I guess by following above article.
Client------------------nginx----------------------nginx
192.168.13.90/16 eth0 - 192.168.13.2/16 eth0 - 192.168.13.9/16
eth0.1 - 182.28.129.202/24
|
|
uWSGI
I am beginner to nginx and python. I am trying to configure nginx as reverse proxy on eth0.1 interface in the following scenario,
Client------------------nginx---------------------nginx
182.28.129.201/24 eth0 - 192.168.13.9/16 eth0 - 192.168.13.2/16
eth0.1 - 182.28.129.202/24
|
|
uWSGI
Below are the current nginx config files with my current trial & error comments,
Edit2 - Comment out the location block for special login.html page from upstream nginx reverse proxy server
# /etc/nginx/nginx.conf
# 192.168.13.2/16 - this upstream server
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
access_log /var/log/access.log;
error_log /var/log/error.log;
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
Below is the ui.conf file included by above nginx.conf,
# /etc/nginx/sites-enabled/ui.conf
# 192.168.13.2/16 - this upstream server
server {
listen 192.168.13.2:80;
server_name 192.168.13.2:80;
access_log /var/log/access.log;
error_log /var/log/error.log;
# No physical login.html,
# it is passing it to root which is login screen
#location = /login.html { <--------- Edit2
# return 301 /;
#}
location / {
include uwsgi_params;
uwsgi_pass unix:/var/run/ui.sock;
}
}
Following is the nginx.conf on eth0.1 on different subnet,
Edit1 - The problem I am facing with this config is the request is proxied to the upstream nginx server but the response is gzip and chunked. I don't see page loading on the client.
Edit2 - I added location block for special page login.html and it loaded the page but it stuck at the redirect page. See wireshark stream below,
# /etc/nginx/nginx.conf
# eth0 - 192.168.13.9/16
# eth0.1 - 182.28.129.202/24 - Reverse proxy
user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log debug;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 0; #Disable
gzip on;
gzip_disable "msie6"; # <----------- Edit2
server {
listen 182.28.129.202:8080;
server_name 182.28.129.202:8080;
#rewrite ^(.*) $scheme://$server_name$1 permanent;
location = /login.html { # <---------- Edit2
proxy_pass http://192.168.13.2:80/;
}
location / {
proxy_pass http://192.168.13.2:80;
# trial and error
#proxy_http_version 1.1;
#proxy_set_header Host $host;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header X-Forwarded-Proto http;
#proxy_bind 192.168.13.9;
#proxy_buffering off;
}
}
}
Edit2 - Wireshark stream, stuck at redirect page, /home
POST /login.html HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Referer: http://182.28.129.202:8080/login.html
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Host: 182.28.129.202:8080
Content-Length: 33
DNT: 1
Connection: Keep-Alive
Cache-Control: no-cache
password=xxxxxxxxxx&login-submit=Server: nginx/1.6.2
Date: Thu, 07 Dec 2017 20:10:50 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 239
Connection: close
Location: http://192.168.13.2/home
Set-Cookie: remember_token=admin|c8aa43aab8b27724a207eb28ac7d1034d1e274fc4f528002a2d6106bb7c36a41756f6951d518f632d426a7d8c8257ad00dbab78e3daf7a5bbbc723ba33107e5e; Expires=Fri, 07-Dec-2018 20:10:50 GMT; Path=/
Set-Cookie: session=.eJw1zksKwzAMRdG9eNyBLUeSnc0E2ZJoBgkln1Hp3msonT4uvPMOix92PsN8Hbc9wrJqmAMgdgOPWhJx06jSM-lEuUKN5CzelFJJJY-pRs5QDLkgRWsdPZZkTZETkRikws1gUufeRL0a9zhlHAlXFEDPYlLHW1RI3sAYwoC87Nhkt_360-7Tjh9PdFv38PkC4lY3Tg.DQsxyg.lZrpSNvXnwE-JHT5t6qlYLAQP4Y; Expires=Sun, 07-Jan-2018 20:10:50 GMT; HttpOnly; Path=/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: /home. If not click the link.
First I am trying to be able to access UI over HTTP and then with HTTPS. I am not able to move ahead due to my knowledge in this area so greatly appreciate any guidance.
Thank you,
I was doing a lot of trial and error and thats where was observing weird errors. Following are the config files which worked for me. Hope it will help some-one.
# 192.168.13.2/16 - this upstream server
server {
listen 192.168.13.2:80;
server_name 192.168.13.2:80;
access_log /var/log/access.log;
error_log /var/log/error.log;
# No physical login.html,
# it is passing it to root which is login screen
location = /login.html {
rewrite ^ http://192.168.13.2/ last;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/var/run/ui.sock;
}
}
# /etc/nginx/nginx.conf
# eth0 - 192.168.13.9/16
# eth0.1 - 182.28.129.202/24 - Reverse proxy
user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log debug;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 0; #Disable
gzip on;
gzip_disable "msie6";
server {
listen 182.28.129.202:8080;
server_name 182.28.129.202:8080;
location / {
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_pass http://192.168.13.2:80;
}
}
}
$upstream_cache_status is HIT, but the $request_time sometimes last for 5s, What's the problem?
My nginx.conf
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
gzip on;
client_max_body_size 30M;
proxy_temp_path /tmp/proxy_temp_dir;
proxy_cache_path /tmp/proxy_cache_dir levels=1:2 keys_zone=cache:500m inactive=1d max_size=500m;
log_format cache_log '$remote_addr - [$request_time] $status $upstream_cache_status "$request"';
server {
access_log logs/access.log cache_log;
error_log logs/error.log error;
proxy_cache cache;
proxy_cache_valid 10m;
location / {
proxy_next_upstream http_502 error timeout;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://xxxxxx;
}
}
include /usr/local/openresty/nginx/conf/vhosts/*.conf;
}
And access.log:
x.x.x.x - [5.076] 200 HIT "GET /xxx"
x.x.x.x - [0.092] 200 HIT "GET /xxx"
Same request url, both are hit the cache, why $request_time last for 5s or more?
Thanks.
It's a disk IO problem, I moved the proxy_cache_path to another SSD and the problem is solved.