An explanation of the nginx/starman/dancer web stack - nginx

I've been doing web-programming for a while now and am quite familiar with the LAMP stack. I've decided to try playing around with the nginx/starman/dancer stack and I'm a bit confused about how to understand, from a high-level, how all the pieces relate to each other. Setting up the stack doesn't seem as straight forward as setting up the LAMP stack, but that's probably because I don't really understand how the pieces relate.
I understand the role nginx is playing - a lightweight webserver/proxy - but I'm confused about how starman relates to pgsi, plack and dancer.
I would appreciate a high-level breakdown of how these pieces relate to each other and why each is necessary (or not necessary) to get the stack setup. Thanks!

I've spent the last day reading about the various components and I think I have enough of an understanding to answer my own question. Most of my answer can be found in various places on the web, but hopefully there will be some value to putting all the pieces in one place:
Nginx: The first and most obvious piece of the stack to understand is nginx. Nginx is a lightweight webserver that can act as a replacement for the ubiquitous Apache webserver. Nginx can also act as a proxy server. It has been growing rapidly in its use and currently serves about 10% of all web domains. One crucial advantage of nginx is that it is asynchronous and event-driven instead of creating a process thread to handle each connection. In theory this means that nginx is able to handle a large number of connections without using a lot of system resources.
PSGI: PSGI is a protocol (to distinguish it from a particular implementation of the protocol, such as Plack). The main motivation for creating PSGI, as far as I can gather, is that when Apache was first created there was no native support for handling requests with scripts written in e.g., Perl. The ability to do this was tacked on to Apache using mod_cgi. To test your Perl application, you would have to run the entire webserver, as the application ran within the webserver. In contrast, PSGI provides a protocol with which a webserver can communicate with a server written in e.g. Perl. One of the benefits of this is that it's much easier to test the Perl server independently of the webserver. Another benefit is that once an application server is built, it's very easy to switch in different PSGI-compatible webservers to test which provides the best performance.
Plack: This is a particular implementation of the PSGI protocol that provides the glue between a PSGI-compatible webserver and a perl application server. Plack is Perl's equivalent of Ruby's Rack.
Starman: A perl based webserver that is compatible with the PSGI protocol. One confusion I had was why I would want to use both Starman and Nginx at the same time, but thankfully that question was answered quite well here on Stackoverflow. The essence is that it might be better to let nginx serve static files without requiring a perl process to do that, while also allowing the perl application server to run on a higher port.
Dancer: A web application framework for Perl. Kind of an equivalent of Ruby on Rails. Or to be more precise, an equivalent of Sinatra for Ruby (the difference is that Sinatra is a minimalist framework, whereas Ruby on Rails is a more comprehensive web framework). As someone who dealt with PHP and hadn't really used a web framework before, I was a bit confused about how this related to the serving stack. The point of web frameworks is they abstract away common tasks that are very frequently performed in web applications, such as converting database queries into objects/data structures in the web application.
Installation (on ubuntu):
sudo apt-get install nginx
sudo apt-get install build-essential curl
sudo cpan App::cpanminus
sudo cpanm Starman
sudo cpanm Task::Plack
sudo apt-get install libdancer-perl
Getting it running:
cd
dancer -a mywebapp
sudo plackup -s Starman -p 5001 -E deployment --workers=10 -a mywebapp/bin/app.pl
Now you will have a starman server running your Dancer application on port 5001. To make nginx send traffic to the server you have to modify /etc/nginx/nginx.conf and add a rule something like this to the http section:
server {
server_name permanentinvesting.com
listen 80;
location /css/ {
alias /home/ubuntu/mywebapp/public/css/;
expires 30d;
access_log off;
}
location / {
proxy_pass http://localhost:5001;
proxy_set_header X-Real-IP $remote_addr;
}
}
The first location rule specifies that nginx should handle static content in the /css directory by getting it from /home/ubuntu/mywebapp/public/css/. The second location rule says that traffic to the webserver on port 80 should be sent to the Starman server to handle. Now we just need to start nginx:
sudo service nginx start

Your Answer is so far correct, but it would be better to set up nginx the following way:
server {
listen 80;
server_name foo.example.com;
location / {
# Serve static files directly:
if (-f $request_filename) {
expires 30d;
break;
}
# Pass on other requests to Dancer app
proxy_pass_header Server;
proxy_pass http://localhost:5001/;
}
}
This make nginx serve all static files (JavaScript and images) and not just the css.
This example is taken from the 2011 Perl Dancer Advent :)

From nginx wiki:
"IfIsEvil ... Directive if has problems when used in location context, in some cases it doesn't do what you expect but something completely different instead. In some cases it even segfaults. It's generally a good idea to avoid it if possible...."
A better set up is:
server {
listen 80;
server_name foo.example.com;
location / {
# Checks the existence of files and uses the first match
try_files $uri $uri/ #dancer;
}
location #dancer {
# Pass on other requests to Dancer app
proxy_pass_header Server;
proxy_pass http://localhost:5001/;
}
}

Correction for the answer from s.magri:
location #dancer {
# Pass on other requests to Dancer app
proxy_pass_header Server;
proxy_pass http://localhost:5001;
}
I had to remove the trailing slash in the last proxy_pass directive. My version of nginx (1.10.3) won't start up with the trailing slash.

Related

Nginx does not re-resolve DNS names in Docker

I am running nginx as part of the docker-compose template.
In nginx config I am referring to other services by their docker hostnames (e.g. backend, ui).
That works fine until I do that trick:
docker stop backend
docker stop ui
docker start ui
docker start backend
which makes backend and ui containers to exchange IP addresses (docker provides private network IPs on a basis of giving the next IP available in CIDR to each new requester). This 4 commands executed imitate some rare cases when both upstream containers got restarted at the same time but the nginx container did not. Also, I believe, this should be a very common situation when running pods on Kubernetes-based clusters.
Now nginx resolves backend host to ui's IP and ui to backend's IP.
Reloading nginx' configuration does help (nginx -s reload).
Also, if I do nslookup from within the nginx container - the IPs are always resolved correctly.
So this isolates the problem to be a pure nginx issue around the DNS caching.
The things I tried:
I have the resolver set under the http {} block in nginx config:
resolver 127.0.0.11 ipv6=off valid=10s;
Most common solution proposed by the folks on the internet to use variables in proxy-pass (this helps to prevent nginx to resolve and cache DNS records on start) - that did not make ANY difference at all:
server {
<...>
set $mybackend "backend:3000";
location /backend/ {
proxy_pass http://$mybackend;
}
}
Tried adding resolver line into the location itself
Tried setting the variable on the http{} block level, using map:
http {
map "" $mybackend {
default backend:3000;
}
server {
...
}
}
Tried to use openresty fork of nginx (https://hub.docker.com/r/openresty/openresty/) with resolver local=true
None of the solutions gave any effect at all. The DNS caches are only wiped if I reload nginx configuration inside of the container OR restart the container manually.
My current workaround is to use static docker network declared in docker-compose.yml. But this has its cons too.
Nginx version used: 1.20.0 (latest as of now)
Openresty versions used: 1.13.6.1 and 1.19.3.1 (latest as of now)
Would appreciate any thoughts
UPDATE 2021-09-08: Few months later I am back to solving this same issue and still no luck. Really looks like the bug in nginx - I can not make nginx to re-resolve the dns names. There seems to be no timeout to nginx' dns cache and none of the options listed above to introduce timeouts or trigger dns flush work.
UPDATE 2022-01-11: I think the problem is really in the nginx. I tested my config in many ways a couple months ago and it looks like something else in my nginx.conf prevents the valid parameter of the resolver directive from working properly. It is either the limit_req_zone or the proxy_cache_path directives used for request rate limiting and caching respectively. These just don't play nicely with the valid param for some reason. And I could not find any information about this anywhere in nginx docs.
I will get back to this later to confirm my hypothesis.
Maybe it's because nginx's DNS resolver for upstream servers only works in the commercial version, nginx plus?
https://www.nginx.com/products/nginx/load-balancing/#service-discovery
TLDR: Your Internet Provider may be caching dnses with no respect to tiny TTL values (like 1 second).
I've been trying to retest locally the same thing.
Your docker might be using local resolver (127.0.0.11)
Then Dns might be cached by your OS (which you may clean - that's OS specific)
Then you might have it cached on your WIFI/router (yes!)
Later it goes to your ISP and is beyond your control.
But nslookup is your friend, you can query each dns server between nginx and root DNS server.
Something very easy to reproduce (without setting up local dns server)
Create route 53 'A' entry with TTL of 1 second and try to query AWS dns server in your hosted zone (it will be sth. like ns-239.awsdns-29.com)
Play around with dig / nslookup command
nslookup
set type=a
server ns-239.awsdns-29.com
your.domain.com
It will return IP you have set
Change the Route53 'A' entry to some other ip.
use dig / nslookup and make sure you see changes immediately
Then set resolver in nginx to AWS dns name (for testing purposes only).
If that works it means that DNS is cached elsewere and this is no longer nginx issue!
In my case it was sunrise WIFI router which began to see new IP only after I restarted it (I assume things would resolve after some longer value).
Great help when debugging this is when your nginx is compiled with
--with-debug
Then in nginx logs you see whether given dns was resolved and to what IP.
My whole config looks like this (here with standard docker resolver which has to be set if you are using variables in proxy_pass!)
server {
listen 0.0.0.0:8888;
server_name nginx.my.custom.domain.in.aws;
resolver 127.0.0.11 valid=1s;
location / {
proxy_ssl_server_name on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $host;
set $backend_servers my.custom.domain.in.aws;
proxy_pass https://$backend_servers$request_uri;
}
}
Then you can try to test it with
curl -L http://nginx.my.custom.domain.in.aws --resolve nginx.my.custom.domain.in.aws 0.0.0.0:8888
Was struggling on the same thing exactly for the same thing (Docker Swarm) and actually to make it work I required to let the upstream away from my configuration.
Something that works well (tested 5' ago on NGINX 2.22) :
location ~* /api/parameters/(.*)$ {
resolver 127.0.0.11 ipv6=off valid = 1s;
set $bck_parameters parameters:8000;
proxy_pass http://$bck_parameters/api/$1$is_args$args;
}
where $bck_parameters is NOT an upstream but the real server behind.
Doing same thing with upstream will fail.

Accessing GraphDB Workbench throught the internet (not localhost) in a Nginx server

I have GraphDb (standlone server version) in Ubuntu Server 16 running in localhost (with the command ./graphdb -d in /etc/graphdb/bin). But I only have ssh access to the server in the terminal, can't open Worbench in localhost:7200 locally.
I have many websites running on this machine with Ningx. If I try to access the machine's main IP with the port 7200 through the external web it doesn't work (e.g. http://193.133.16.72:7200/ = "connection timmed out").
I have tried to make a reverse proxy with Nginx with this code ("xxx" = domain):
listen 7200;
listen [::]:7200;
server_name sparql.xxx.com;
location / {
proxy_pass http://127.0.0.1:7200;
}
}
But all this fails. I checked and port 7200 is open in firewall (ufw). In the logs I get info that GraphDB is working locally in some testes. But I need Workbench access to import and create repositories (not sure how to do it or if it is possible without the Workbench GUI).
Is there a way to connect through the external web to the Workbench using a domain/IP and/or Nginx?
Read all the documentation and searched all day, but could not find a way to deal with this sorry. I only used GraphDB locally (the simple installer version), never used the standalone server in production before, sorry.
PS: two extra questions related:
a) For creating an URI endpoint it is the same procedure?
b) For GraphDB daemon to start automattically at boot time (with the command ./graphdb -d in graph/bin folder), what is the recommended way and configuration? (tryed the line "/etc/graphdb/bin ./graphdb -d" in rc.local but it didn't worked).
In case it is usefull for someone, I was able to make it work with this Nginx configuration:
server {
listen 80;
server_name sparql.xxxxxx.com;
location / {
proxy_pass http://localhost:7200;
proxy_set_header Host $host;
}
}
I think it was the "proxy_set_header Host $host;" that solved it (tried the rest without it before and didn't worked). I think GraphDB uses some headers to set configurations, and they were not passing.
I wounder if I am forgetting something else important to forward in the proxy, but in this moment the Worbench seams to work and opens in the domain used "sparql.xxxxxx.com".

Can I use Clojure with nginx?

This is a follow up to my question here. I've set up a home server (just my other laptop running ubuntu and nginx) and I want to serve clojure files.
I am asking help for understanding how this process works. I am sorry at this point I am confused and I think I need to start over. I am asking a new question because I want to use nginx not lein ring server, as suggested in the answer for that question.
First I started a project guestbook with leiningen and I ran lein ring server and I see "Hello World" at localhost:3000. As far as I understand this has nothing to do with nginx!
How does nginx enter in this process? At first I was trying to create a proxy server with nginx and that worked too, but I did not know how serve clojure files with that setup.
This is what I have in my nginx.conf file adapted from this answer:
upstream ring {
server 127.0.0.1:3000 fail_timeout=0;
}
server {
root /home/a/guestbook/resources/public;
# make site accessible from http://localhost
server_name localhost;
location / {
# first attempt to serve request as file
try_files $uri $uri/ #ring;
}
location #ring {
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_pass http://ring;
}
location ~ ^(assets|images|javascript|stylesheets|system)/ {
expires max;
add_header Cache-Control public;
}
}
So I want to use my domain example.com (not localhost); how do I go about doing this?
EDIT
As per #noisesmith's comment I will opt to go with lein uberjar option. As explained here, it appears very easy to create one:
$ lein uberjar
Unpacking clojure-1.1.0-alpha-20091113.120145-2.jar
Unpacking clojure-contrib-1.0-20091114.050149-13.jar
Compiling helloworld
[jar] Building jar: helloworld.jar
$ java -jar helloworld.jar
Hello world!
Can you also direct me to the right documentation about how I can use this uberjar with nginx?
Please try Nginx-Clojure module. You can run clojure Ring handlers with Nginx without any Java Web Server, eg. Jetty.
For starters, don't use lein to run things in production. You can use lein uberjar to create a jar file with all your deps ready to run, and java -jar to run the app from the resulting jar. There is also the option of running lein ring uberwar to create a war archive to be run inside tomcat, which provides some other conveniences (like log rotation and integration with /etc/init.d as a service etc. on most Linux systems).
nginx sits in front of your app, on port 80. It will serve up the content by proxying your app. This is useful because nginx has many capabilities (especially regarding security) that you then don't need to implement in your own app, including optional integration with https and selinux integration. Using nginx in front of your app also prevents you from needing to run java as root (typically only the root user can use port 80). Furthermore you can let nginx serve static assets directly, rather than having to serve them from your app.

Running CKAN behind Nginx

I would like to run CKAN behind Nginx. Does it make sense to run the CKAN site through the paster process:
paster serve production.ini
... and then just point Nginx at it through a reverse proxy?
server {
listen 80;
location / {
proxy_pass http://127.0.0.1:5000;
}
}
Or is there a way to run as a FastCGI process? Perhaps a FastCGI daemon process similar to Django?
The default installation of CKAN (the one done via package install) sets up CKAN running on Apache with mod_wsgi with Nginx on top. Have at look at the Apache and Nginx configuration files for details on how it is done.
Regarding using paster on a production environment, I'd imagine that sooner or later you will hit performance problems, so I would avoid it. You may find this answer useful:
Can I use paster on production site?

nginx - client_max_body_size has no effect

nginx keeps saying client intended to send too large body. Googling and RTM pointed me to client_max_body_size. I set it to 200m in the nginx.conf as well as in the vhost conf, restarted Nginx a couple of times but I'm still getting the error message.
Did I overlook something? The backend is php-fpm (max_post_size and max_upload_file_size are set accordingly).
Following nginx documentation, you can set client_max_body_size 20m ( or any value you need ) in the following context:
context: http, server, location
NGINX large uploads are successfully working on hosted WordPress sites, finally (as per suggestions from nembleton & rjha94)
I thought it might be helpful for someone, if I added a little clarification to their suggestions. For starters, please be certain you have included your increased upload directive in ALL THREE separate definition blocks (server, location & http). Each should have a separate line entry. The result will like something like this (where the ... reflects other lines in the definition block):
http {
...
client_max_body_size 200M;
}
(in my ISPconfig 3 setup, this block is in the /etc/nginx/nginx.conf file)
server {
...
client_max_body_size 200M;
}
location / {
...
client_max_body_size 200M;
}
(in my ISPconfig 3 setup, these blocks are in the /etc/nginx/conf.d/default.conf file)
Also, make certain that your server's php.ini file is consistent with these NGINX settings. In my case, I changed the setting in php.ini's File_Uploads section to read:
upload_max_filesize = 200M
Note: if you are managing an ISPconfig 3 setup (my setup is on CentOS 6.3, as per The Perfect Server), you will need to manage these entries in several separate files. If your configuration is similar to one in the step-by-step setup, the NGINX conf files you need to modify are located here:
/etc/nginx/nginx.conf
/etc/nginx/conf.d/default.conf
My php.ini file was located here:
/etc/php.ini
I continued to overlook the http {} block in the nginx.conf file. Apparently, overlooking this had the effect of limiting uploading to the 1M default limit. After making the associated changes, you will also want to be sure to restart your NGINX and PHP FastCGI Process Manager (PHP-FPM) services. On the above configuration, I use the following commands:
/etc/init.d/nginx restart
/etc/init.d/php-fpm restart
As of March 2016, I ran into this issue trying to POST json over https (from python requests, not that it matters).
The trick is to put "client_max_body_size 200M;" in at least two places http {} and server {}:
1. the http directory
Typically in /etc/nginx/nginx.conf
2. the server directory in your vhost.
For Debian/Ubuntu users who installed via apt-get (and other distro package managers which install nginx with vhosts by default), thats /etc/nginx/sites-available/mysite.com, for those who do not have vhosts, it's probably your nginx.conf or in the same directory as it.
3. the location / directory in the same place as 2.
You can be more specific than /, but if its not working at all, i'd recommend applying this to / and then once its working be more specific.
Remember - if you have SSL, that will require you to set the above for the SSL server and location too, wherever that may be (ideally the same as 2.). I found that if your client tries to upload on http, and you expect them to get 301'd to https, nginx will actually drop the connection before the redirect due to the file being too large for the http server, so it has to be in both.
Recent comments suggest that there is an issue with this on SSL with newer nginx versions, but i'm on 1.4.6 and everything is good :)
You need to apply following changes:
Update php.ini (Find right ini file from phpinfo();) and increase post_max_size and upload_max_filesize to size you want:
sed -i "s/post_max_size =.*/post_max_size = 200M/g" /etc/php5/fpm/php.ini
sed -i "s/upload_max_filesize =.*/upload_max_filesize = 200M/g" /etc/php5/fpm/php.ini```
Update NginX settings for your website and add client_max_body_size value in your location, http, or server context.
location / {
client_max_body_size 200m;
...
}
Restart NginX and PHP-FPM:
service nginx restart
service php5-fpm restart
NOTE: Sometime (In my case almost every time) you need to kill php-fpm process if it didn't refresh by service command properly. To do that you can get list of processes (ps -elf | grep php-fpm) and kill one by one (kill -9 12345) or use following command to do it for you:
ps -elf | grep php-fpm | grep -v grep | awk '{ print $4 }' | xargs kill -9
Please see if you are setting client_max_body_size directive inside http {} block and not inside location {} block. I have set it inside http{} block and it works
Someone correct me if this is bad, but I like to lock everything down as much as possible, and if you've only got one target for uploads (as it usually the case), then just target your changes to that one file. This works for me on the Ubuntu nginx-extras mainline 1.7+ package:
location = /upload.php {
client_max_body_size 102M;
fastcgi_param PHP_VALUE "upload_max_filesize=102M \n post_max_size=102M";
(...)
}
I had a similar problem recently and found out, that client_max_body_size 0; can solve such an issue. This will set client_max_body_size to no limit. But the best practice is to improve your code, so there is no need to increase this limit.
I meet the same problem, but I found it nothing to do with nginx. I am using nodejs as backend server, use nginx as a reverse proxy, 413 code is triggered by node server. node use koa parse the body. koa limit the urlencoded length.
formLimit: limit of the urlencoded body. If the body ends up being larger than this limit, a 413 error code is returned. Default is 56kb.
set formLimit to bigger can solve this problem.
Assuming you have already set the client_max_body_size and various PHP settings (upload_max_filesize / post_max_size , etc) in the other answers, then restarted or reloaded NGINX and PHP without any result, run this...
nginx -T
This will give you any unresolved errors in your NGINX configs. In my case, I struggled with the 413 error for a whole day before I realized there were some other unresolved SSL errors in the NGINX config (wrong pathing for certs) that needed to be corrected. Once I fixed the unresolved issues I got from 'nginx -T', reloaded NGINX, and EUREKA!! That fixed it.
I'm setting up a dev server to play with that mirrors our outdated live one, I used The Perfect Server - Ubuntu 14.04 (nginx, BIND, MySQL, PHP, Postfix, Dovecot and ISPConfig 3)
After experiencing the same issue, I came across this post and nothing was working. I changed the value in every recommended file (nginx.conf, ispconfig.vhost, /sites-available/default, etc.)
Finally, changing client_max_body_size in my /etc/nginx/sites-available/apps.vhost and restarting nginx is what did the trick. Hopefully it helps someone else.
In case you are using Kubernetes, add the following annotations to your Ingress:
annotations:
nginx.ingress.kubernetes.io/client-max-body-size: "5m"
nginx.ingress.kubernetes.io/client-body-buffer-size: "8k"
nginx.ingress.kubernetes.io/proxy-body-size: "5m"
nginx.ingress.kubernetes.io/proxy-buffer-size: "8k"
Confirm the changes were applied:
kubectl -n <namespace> describe ingress <ingress-name>
References:
Client Body Buffer Size
Custom max body size
Had the same issue that the client_max_body_size directive was ignored.
My silly error was, that I put a file inside /etc/nginx/conf.d which did not end with .conf. Nginx will not load these by default.
If you tried the above options and no success, also you're using IIS (iisnode) to host your node app, putting this code on web.config resolved the problem for me:
Here is the reference: https://www.inflectra.com/support/knowledgebase/kb306.aspx
Also, you can chagne the length allowed because now I think its 2GB. Modify it by your needs.
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
The following config worked for me. Notice I only set client_max_body_size 50M; once, contrary to what others are saying...
File: /etc/nginx/conf.d/sites.conf
server {
listen 80 default_server;
server_name portal.myserver.com;
return 301 https://$host$request_uri;
}
server {
resolver 127.0.0.11 valid=30s;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl_certificate /secret/portal.myserver.com.crt;
ssl_certificate_key /secret/portal.myserver.com.pem;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
server_name portal.myserver.com;
client_max_body_size 50M;
location /fileserver/ {
set $upstream http://fileserver:6976;
proxy_pass $upstream;
}
}
If you are using windows version nginx, you can try to kill all nginx process and restart it to see.
I encountered same issue In my environment, but resolved it with this solution.

Resources