Go web server with nginx server in web application [duplicate] - http

This question already has answers here:
What are the benefits of using Nginx in front of a webserver for Go?
(4 answers)
Closed 7 years ago.
Sorry, I cannot find this answer from Google search
and nobody seems to explain clearly the difference between pure Go webserver
and nginx reverse proxy. Everybody seems to use nginx in front
for web applications.
My question is, while Go has all http serving functions,
what is the benefit of using nginx over pure Go web servers?
And in most cases, we set up the Go webserver for all routes here
and have the nginx configurations in front.
Something like:
limit_req_zone $binary_remote_addr zone=limit:10m rate=2r/s;
server {
listen 80;
log_format lf '[$time_local] $remote_addr ;
access_log /var/log/nginx/access.log lf;
error_log /var/log/nginx/error.log;
set_real_ip_from 0.0.0.0/0;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
server_name 1.2.3.4 mywebsite.com;
}
When we have this Go:
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
Are the traffic to nginx and Go web server different?
If not, why do we have two layers of web server?
Please help me understand this.
Thanks,

There's nothing stopping you from serving requests from Go directly.
On the other hand, there are some features that nginx provides out-of-the box that may be useful, for example:
handle many virtual servers (e.g. have go respond on app.example.com and a different app on www.example.com)
http basic auth in some paths, say www.example.com/secure
access logs
etc
All of this can be done in go but would require programming, while in nginx it's just a matter of editing a .conf file and reloading the configuration. Nginx doesn't even need a restart for this changes to take place.
(From a "process" point of view, nginx could be managed by an ops employee, with root permissions, running in a well known port, while developers deploy their apps on higher ones.)

The general idea of using nginx in this scenario is to serve up static resources via nginx and allow Go to handle everything else.
Search for "try_files" in nginx. It allows for checking the disk for the existence of a file and serving it directly instead of having to handle static assets in the Go app.

This has been asked a few times before[1] but for posterity:
It depends.
Out of the box, putting nginx in front as a reverse proxy is going to
give you:
Access logs
Error logs
Easy SSL termination
SPDY support
gzip support
Easy ways to set HTTP headers for certain routes in a couple of lines
Very fast static asset serving (if you're serving off S3/etc. though, this isn't that relevant)
The Go HTTP server is very good, but you will need to reinvent the
wheel to do some of these things (which is fine: it's not meant to be
everything to everyone).
I've always found it easier to put nginx in front—which is what it is
good at—and let it do the "web server" stuff. My Go application does
the application stuff, and only the bare minimum of headers/etc. that
it needs to. Don't look at putting nginx in front as a "bad" thing.
Further, to extend on my answer there, there's also the question of crash resilience: your Go application isn't restricted by a configuration language and can do a lot of things.
Some of these things may crash your program. Having nginx (or HAProxy, or Varnish, etc.) as a reverse proxy can give you a some request buffering (to allow your program to restart) and/or serve stale content from its local cache (i.e. your static home page), which may be better than having the browser time out and serve a "cannot connect to server error".
On the other hand, if you're building small internal services, 'naked' Go web servers with your own logging library can be easier to manage (in terms of ops).
If you do want to keep everything in your Go program, look at gorilla/handlers for gzip, logging and proxy header middleware, and lumberjack for log rotation (else you can use your system's logging tools).

Related

Basic NGINX proxy_pass example is serving 404's

I have an very basic out-of-the-box NGINX server running on a Windows machine at http://10.0.15.19:80. I'm attempting to use it as a proxy to get around a CORS restriction. I have the following nginx.conf, which from every example I've seen sounds like it should redirect any traffic from http://10.0.15.19 to http://10.0.1.2:3000.
events {
worker_connections 1024;
}
http {
server {
listen 80;
server_name 10.0.15.19;
location / {
proxy_pass http://10.0.1.2:3000/;
}
}
}
Instead of serving the content from http://10.0.1.2:3000 I get the default index.html page inside the included html folder. Similarly, if I were to go to http://10.0.15.19/any/path I get a 404 even though http://10.0.1.2:3000/any/path works correctly.
EDIT: I've noticed that even after commenting out the entire server block of my configuration, it's still serving content from the included html folder. This makes me think there is another server configuration running that I'm not aware of, but I'm not sure where. I downloaded NGINX from here, and I assume all configuration files exist within this folder.
It turns out this was because simply closing the window that pops up when you open nginx.exe doesn't actually kill the process. And in Windows you can apparently have multiple services bound to the same port, so I had many servers serving on port 80. Killing all of these processes and relaunching with the originally posted config fixed my problem.

Nginx: capture post requests when upstream is offline

I'm using Nginx as a reverse proxy for a Ruby on Rails application.
The application has 2 critical endpoints that are responsible for capturing data from customers who're registering their details with our service. These endpoints take POST data from a form that may or may not be hosted on our website.
When our application goes down for maintenance (rare, but we have a couple of SPOF services), I would like to ensure the POST data is captured so we don't lose it forever.
Nginx seems like a good place to do this given that it's already responsible for serving requests to the upstream Rails application, and has a custom vhost configuration in place that serves a static page for when we enable maintenance mode. I figured this might be a good place for additional logic to store these incoming POST requests.
The issue I'm having is that Nginx doesn't parse POST data unless you're pointing it at an upstream server. In the case of our maintenance configuration, we're not; we're just rendering a maintenance page. This means that $request_body¹ is empty. We could perhaps get around this by faking a proxy server, or maybe even pointing Nginx at itself and enabling the logger on a particular location. This seems hacky though.
Am I going about this the wrong way? I've done some research and haven't found a canonical way to solve this use-case. Should I be using a third-party tool and not Nginx?
1: from ngx_http_core_module: "The variable’s value is made available in locations processed by the proxy_pass, fastcgi_pass, uwsgi_pass, and scgi_pass directives when the request body was read to a memory buffer."

Server static files from FTP server using NGINX

I have a local network, on which there are some old insecure services. I use nginx reverse proxy with client certificates authentication as safe entrypoint to this local network from the Internet. Till now I used it only to proxy HTTP servers using
location / {
proxy_pass http://192.168.123.45:80/;
}
and everything works fine.
But now I would like to serve static files, that are accessible through FTP on a local server, I tried simply:
location /foo {
proxy_pass ftp://user:password#192.168.100.200:5000/;
}
but that doesn't work, and I could not find anything that would simply proxy HTTP request to FTP request.
Is there any way to do this?
Nginx doesn't support proxying to FTP servers. At best, you can proxy the socket... and this is a real hassle with regular old FTP due to it opening new connections on random ports every time a file is requested.
What you can probably do instead is create a FUSE mount to that FTP server on some local path, and serve that path with Nginx like normal. To that end, CurlFtpFS is one tool for this. Tutorial: https://linuxconfig.org/mount-remote-ftp-directory-host-locally-into-linux-filesystem
(Note: For security and reliability, it's strongly recommended you migrate away from FTP when possible. Consider SSH/SFTP instead.)

HTTP relay / proxy / mapping server

I'm trying to set up a simple relay / mapping server locally and feel there has to be some off the shelf solution, but I can't seem to find it.
I'm debugging an application of mine that needs to connect to host_A. Instead of connecting to host_A I want to configure it to connect to local_proxy. I don't want to use proxying protocols, but instead want to configure it to connect to http://localhost:80 and then have local_proxy connect to host_A and have local_proxy simply relay all messages back and forth.
I would expect to have to configure local_proxy to tell it what server it is supposed to relaying
Then there is 1 particular endpoint I want to be able to intercept and change the return info so I can better debug my application.
I thought I should be able to do this with Charles Proxy, but I couldn't figure out how.
At the moment, this doesn't need to support SSL (though that would always be nice).
I think what you are trying to build here is known as a reverse-proxy. There is a variety of solutions available for that, but you may produce results fastest with nginx, which can not only be configured for reverse proxy duties but is also sporting some powerfull SSL capabilities. A minimal solution to your problem would look like this:
server {
listen 80;
# Adjust for expected hostname. Space-separated list of hostnames possible.
server_name host_a;
location / {
# Forward all incoming requests to host_a
proxy_pass http://host_a;
}
}

Nginx Reverse Proxy to Remote Server

Okay, so I know there is a lot of questions about Nginx Reverse Proxy but I don't understand any of the answers. I read the documentation on Nginx's website and I kind of get it but need some help.
So here is what I want.
Visitor ---> Nginx Reverse Proxy ---> Nginx Server (Website)
I know that you can get the reverse proxy to listen to a remote server, but I can't find the configuration files that I want. I also want to show a static html page when passing through the Nginx Reverse Proxy. So like a page that says "Hosted by this company", like Cloudflare does. I asked someone and they told me if I put the html file on the reverse proxy server then it'll show up when the visitor goes through the server, but I don't understand that concept, how do I get that static html to show up? And what will be the correct configuration be for this? I imagine it'll be different from a normal remote server configuration.
Thanks in advance!
In Nginx reverse proxy configuration it is possible to make some urls to be served from reverse proxy itself without going to source server.
Take a look at the following article: https://www.cevapsepeti.com/how-to-setup-nginx-as-reverse-proxy-for-wordpress/
In this article you can see in virtual.conf lines 51-55 that error_pages are served directly from reverse proxy, while other assets are fetched from source server first. Cloudflare employs the same concept. I hope this helps.

Resources