How can I use nginx brotli_static with proxy_pass? - nginx

nginx is compiled with Brotli enabled. In my nginx.conf
http {
...
brotli_static on;
}
My .br files are located on a server with proxy_pass.
location / {
...
proxy_pass http://app;
}
And .br files have been generated on that app server:
$ ls -lh public/js/dist/index.js*
-rw-r--r-- 1 mike wheel 1.2M Apr 4 09:07 public/js/dist/index.js
-rw-r--r-- 1 mike wheel 201K Apr 4 09:07 public/js/dist/index.js.br
Pulling down the uncompressed file works:
wget https://example.com/js/dist/index.js
Pulls down 1,157,704 size uncompressed file.
wget -S --header="accept-encoding: gzip" https://example.com/js/dist/index.js
Pulls down a 309,360 size gzipped file.
But:
wget -S --header="accept-encoding: br" https://example.com/js/dist/index.js
Still gets the full 1,157,704 size uncompressed file.
I had hoped brotli_static would proxy the .br file requests too - sending something a GET request to the backend for the .br equivalent resource - but this doesn't seem to work.
Can brotli_static work through proxy_pass?

Based on Maxim Dounin (an nginx core engineer)'s comment on gzip_static - which I imagine brotli_static behaves similarly to - brotli_static only handles files, not HTTP resources:
That is, gzip_static is only expected to work when nginx is about to return regular files.
So it looks like brotli_static and proxy_pass isn't possible.

Your nginx config file needs a section to tell it to serve the static content folder. You don't want your app server to do that.
I believe you'll need to place it before the location / so that it takes precedence.

Related

How to config nginx to tail log on web browser

My goal is to see the newest log messages on web page.
I know I could use tail -f <file> to trace the latest 10 line log messages on terminal.
But today, I want to config nginx, so that I could see the same result on web.
For example, when I access to http://192.168.1.200/nginx (My Nginx Host)
I could see the files under /var/log/nginx
Index of /nginx/
-------------------------------------------------
../
access.log 08-Aug-2019 16:43 20651
error.log 08-Aug-2019 16:43 17810
And when I access to http://192.168.1.200/nginx/access.log
I could see the same result as tail -f /var/log/nginx/access.log in terminal (and it is dynamic).
/etc/nginx/conf.d/log.conf
server {
listen 80;
root /var/log/nginx;
location /nginx {
root /var/log/;
default_type text/plain;
autoindex on;
}
}
This is my config, but there are 2 points that doesn't meet my requirements:
I want to access to log page by accessing /log/access.log not by /nginx/access.log
When I access to /log/access.log, this page is static.

Upload file to Nginx with cURL

All,
I'm trying to upload a local file to my remote Nginx server via cURL. I have built Nginx from source with the upload module and the DAV module. At the bottom of the Nginx page, there is an example form to upload a file. I'm not sure how I would implement the form, and (several) Google searches have returned little helpful information about uploading directly to Nginx via cURL.
Current tech stack:
Nginx
Green Unicorn
Flask
Of all the different avenues I've tried, the following is the one that seems the most appropriate for the task.
curl -X POST -F "image=#example.gif" http://54.226.64.199/upload
However, the response is underwhelming.
I've tried --uploade-file as well, the response is a 405. From what I've read, upload only accepts a POST command, not PUT, hence why I get a 405.
I don't need a full solution (would be great!), only pointing in the right direction.
Any help is appreciated. Thanks
EDIT: sorry wanted to include part of my .conf
location /upload {
upload_store /tmp;
#upload_pass #none;
upload_store_access all:rw;
upload_cleanup 400 404 499 500-505;
}
You can do this by specifying filename into URL, without using any external module :
location ~ "/upload/([0-9a-zA-Z-.]*)$" {
alias /storage/www/upload/$1;
client_body_temp_path /tmp/upload_tmp;
dav_methods PUT DELETE MKCOL COPY MOVE;
create_full_put_path on;
dav_access group:rw all:r;
}
And use : curl -T example.gif http://54.226.64.199/upload/example.gif

Adaptive bit rate streaming not working in nginx-vod-module in NGINX server

I have installed Nginx and configured VOD for adaprive streaming using nginx-vod-module. While requesting the master.m3u8 file I'm getting same ts files served for different network bandwidth.
The master.m3u8 file has the following content:
#EXTM3U
#EXT-X-STREAM-INF:PROGRAMID=1,BANDWIDTH=1914317,RESOLUTION=1280x544,CODECS="avc1.64001f,mp4a.40.2"
http://localhost/content/Input.mp4/index-v1-a1.m3u8
The Nginx configuration is:
location /content {
vod hls;
vod_mode local;
root /usr/share/nginx/html;
gzip on;
gzip_types application/vnd.apple.mpegurl;
expires 100d;
add_header Last-Modified "Sun, 19 Nov 2000 08:52:00 GMT";
}
How can I get adaptive bitrate enabled using nginx-vod-module and what's the best way to verify it ?
You encode multiple versions of your Input.mp4 with different resolutions/bitrates. The aspect ratio should be the same. Eg: Input_high.mp4, Input_low.mp4
You edit the master m3u8 playlist and add each rendition with its specific bitrate and resolution:
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=...,RESOLUTION=...,CODECS="..."
/content/Input_low.mp4.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=...,RESOLUTION=...,CODECS="..."
/content/Input_high.mp4.m3u8
When the nginx-vod-module receives a request for a filename.mp4.m3u8 it automatically segments filename.mp4 for HLS and creates the playlist for you. Eg: /content/Input_low.mp4.m3u8 for /content/Input_low.mp4

nginx hooks ? or any other approach

I have couple files on my web project. They are all sending to client with nginx help.
But i want to know (and to show for end users) how many times file was downloaded.
For example file has url like: example.com/file.ppt
and nginx settings is next:
location ~* \.(ppt|swf)$ {
root /path/to/static/content;
expires 30d;
}
so my question: can nginx sent some request by some url ? or something else, when someone tries to download certain file ?
You could count successed file downloads in nginx access.log for example with grep:
grep -c file.ppt access.log

nginx - client_max_body_size has no effect

nginx keeps saying client intended to send too large body. Googling and RTM pointed me to client_max_body_size. I set it to 200m in the nginx.conf as well as in the vhost conf, restarted Nginx a couple of times but I'm still getting the error message.
Did I overlook something? The backend is php-fpm (max_post_size and max_upload_file_size are set accordingly).
Following nginx documentation, you can set client_max_body_size 20m ( or any value you need ) in the following context:
context: http, server, location
NGINX large uploads are successfully working on hosted WordPress sites, finally (as per suggestions from nembleton & rjha94)
I thought it might be helpful for someone, if I added a little clarification to their suggestions. For starters, please be certain you have included your increased upload directive in ALL THREE separate definition blocks (server, location & http). Each should have a separate line entry. The result will like something like this (where the ... reflects other lines in the definition block):
http {
...
client_max_body_size 200M;
}
(in my ISPconfig 3 setup, this block is in the /etc/nginx/nginx.conf file)
server {
...
client_max_body_size 200M;
}
location / {
...
client_max_body_size 200M;
}
(in my ISPconfig 3 setup, these blocks are in the /etc/nginx/conf.d/default.conf file)
Also, make certain that your server's php.ini file is consistent with these NGINX settings. In my case, I changed the setting in php.ini's File_Uploads section to read:
upload_max_filesize = 200M
Note: if you are managing an ISPconfig 3 setup (my setup is on CentOS 6.3, as per The Perfect Server), you will need to manage these entries in several separate files. If your configuration is similar to one in the step-by-step setup, the NGINX conf files you need to modify are located here:
/etc/nginx/nginx.conf
/etc/nginx/conf.d/default.conf
My php.ini file was located here:
/etc/php.ini
I continued to overlook the http {} block in the nginx.conf file. Apparently, overlooking this had the effect of limiting uploading to the 1M default limit. After making the associated changes, you will also want to be sure to restart your NGINX and PHP FastCGI Process Manager (PHP-FPM) services. On the above configuration, I use the following commands:
/etc/init.d/nginx restart
/etc/init.d/php-fpm restart
As of March 2016, I ran into this issue trying to POST json over https (from python requests, not that it matters).
The trick is to put "client_max_body_size 200M;" in at least two places http {} and server {}:
1. the http directory
Typically in /etc/nginx/nginx.conf
2. the server directory in your vhost.
For Debian/Ubuntu users who installed via apt-get (and other distro package managers which install nginx with vhosts by default), thats /etc/nginx/sites-available/mysite.com, for those who do not have vhosts, it's probably your nginx.conf or in the same directory as it.
3. the location / directory in the same place as 2.
You can be more specific than /, but if its not working at all, i'd recommend applying this to / and then once its working be more specific.
Remember - if you have SSL, that will require you to set the above for the SSL server and location too, wherever that may be (ideally the same as 2.). I found that if your client tries to upload on http, and you expect them to get 301'd to https, nginx will actually drop the connection before the redirect due to the file being too large for the http server, so it has to be in both.
Recent comments suggest that there is an issue with this on SSL with newer nginx versions, but i'm on 1.4.6 and everything is good :)
You need to apply following changes:
Update php.ini (Find right ini file from phpinfo();) and increase post_max_size and upload_max_filesize to size you want:
sed -i "s/post_max_size =.*/post_max_size = 200M/g" /etc/php5/fpm/php.ini
sed -i "s/upload_max_filesize =.*/upload_max_filesize = 200M/g" /etc/php5/fpm/php.ini```
Update NginX settings for your website and add client_max_body_size value in your location, http, or server context.
location / {
client_max_body_size 200m;
...
}
Restart NginX and PHP-FPM:
service nginx restart
service php5-fpm restart
NOTE: Sometime (In my case almost every time) you need to kill php-fpm process if it didn't refresh by service command properly. To do that you can get list of processes (ps -elf | grep php-fpm) and kill one by one (kill -9 12345) or use following command to do it for you:
ps -elf | grep php-fpm | grep -v grep | awk '{ print $4 }' | xargs kill -9
Please see if you are setting client_max_body_size directive inside http {} block and not inside location {} block. I have set it inside http{} block and it works
Someone correct me if this is bad, but I like to lock everything down as much as possible, and if you've only got one target for uploads (as it usually the case), then just target your changes to that one file. This works for me on the Ubuntu nginx-extras mainline 1.7+ package:
location = /upload.php {
client_max_body_size 102M;
fastcgi_param PHP_VALUE "upload_max_filesize=102M \n post_max_size=102M";
(...)
}
I had a similar problem recently and found out, that client_max_body_size 0; can solve such an issue. This will set client_max_body_size to no limit. But the best practice is to improve your code, so there is no need to increase this limit.
I meet the same problem, but I found it nothing to do with nginx. I am using nodejs as backend server, use nginx as a reverse proxy, 413 code is triggered by node server. node use koa parse the body. koa limit the urlencoded length.
formLimit: limit of the urlencoded body. If the body ends up being larger than this limit, a 413 error code is returned. Default is 56kb.
set formLimit to bigger can solve this problem.
Assuming you have already set the client_max_body_size and various PHP settings (upload_max_filesize / post_max_size , etc) in the other answers, then restarted or reloaded NGINX and PHP without any result, run this...
nginx -T
This will give you any unresolved errors in your NGINX configs. In my case, I struggled with the 413 error for a whole day before I realized there were some other unresolved SSL errors in the NGINX config (wrong pathing for certs) that needed to be corrected. Once I fixed the unresolved issues I got from 'nginx -T', reloaded NGINX, and EUREKA!! That fixed it.
I'm setting up a dev server to play with that mirrors our outdated live one, I used The Perfect Server - Ubuntu 14.04 (nginx, BIND, MySQL, PHP, Postfix, Dovecot and ISPConfig 3)
After experiencing the same issue, I came across this post and nothing was working. I changed the value in every recommended file (nginx.conf, ispconfig.vhost, /sites-available/default, etc.)
Finally, changing client_max_body_size in my /etc/nginx/sites-available/apps.vhost and restarting nginx is what did the trick. Hopefully it helps someone else.
In case you are using Kubernetes, add the following annotations to your Ingress:
annotations:
nginx.ingress.kubernetes.io/client-max-body-size: "5m"
nginx.ingress.kubernetes.io/client-body-buffer-size: "8k"
nginx.ingress.kubernetes.io/proxy-body-size: "5m"
nginx.ingress.kubernetes.io/proxy-buffer-size: "8k"
Confirm the changes were applied:
kubectl -n <namespace> describe ingress <ingress-name>
References:
Client Body Buffer Size
Custom max body size
Had the same issue that the client_max_body_size directive was ignored.
My silly error was, that I put a file inside /etc/nginx/conf.d which did not end with .conf. Nginx will not load these by default.
If you tried the above options and no success, also you're using IIS (iisnode) to host your node app, putting this code on web.config resolved the problem for me:
Here is the reference: https://www.inflectra.com/support/knowledgebase/kb306.aspx
Also, you can chagne the length allowed because now I think its 2GB. Modify it by your needs.
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
The following config worked for me. Notice I only set client_max_body_size 50M; once, contrary to what others are saying...
File: /etc/nginx/conf.d/sites.conf
server {
listen 80 default_server;
server_name portal.myserver.com;
return 301 https://$host$request_uri;
}
server {
resolver 127.0.0.11 valid=30s;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl_certificate /secret/portal.myserver.com.crt;
ssl_certificate_key /secret/portal.myserver.com.pem;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
server_name portal.myserver.com;
client_max_body_size 50M;
location /fileserver/ {
set $upstream http://fileserver:6976;
proxy_pass $upstream;
}
}
If you are using windows version nginx, you can try to kill all nginx process and restart it to see.
I encountered same issue In my environment, but resolved it with this solution.

Resources