Why are these Swagger endpoints behind a reverse-proxy not found? - nginx

I have a small number of ASP.NET Core services, all running in Docker (via Docker Compose). All services are currently using a prefixed route (their own service name). And they're all setup in Docker Compose to use their own service name as their hostname (connectivity between service containers is OK).
The /api-docs endpoint is provided by Swashbuckle; we setup the prefixed route here, too.
app.UseSwagger(options =>
{
options.RouteTemplate = "scheduler/api-docs/{documentName}/swagger.json";
});
app.UseSwaggerUI(options =>
{
options.RoutePrefix = "scheduler/api-docs";
options.SwaggerEndpoint("/scheduler/api-docs/v1/swagger.json", "Scheduler API v1");
});
I am trying to configure an Nginx reverse-proxy in the container network so that I can go to, say...
http://localhost/<service-name>/api-docs
and it will redirect, inside the container network, to...
http://<service-name>:5000/<service-name>/api-docs
So, here's the Nginx configuration I've come up... basically, match the first part of the request URI, which should be the service name, and proxy to a host called the same and Nginx should add the $request_uri on automatically.
server {
listen 80;
location ~* ^/(?<target>.+)/ {
proxy_pass http://$target:5000;
proxy_redirect off;
resolver 127.0.0.11;
}
}
Here's what I get for a /scheduler/healthcheck endpoint. All good!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:50:24 +0000] "GET /scheduler/healthcheck HTTP/1.1" 200 491 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
But, when I try to go to /scheduler/api-docs, I have problems. We get to the service container and Swashbuckle does a 301 Redirect from /scheduler/api-docs to /scheduler/api-docs/.
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
And, then, things go bad... we "lose" the /scheduler part of the route!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 17:51:18 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000", host: "localhost:4000"
Why does the Swashbuckle redirect send the request back through Nginx, I thought this would all be handled by the local service, and why is Nginx stripping the necessary route prefix from this request?
How do I get this to behave?!
I tried to reconstruct the "whole" URI, just to see what happens...
- proxy_pass http://$target:5000;
+ proxy_pass http://$target:5000$request_uri;
And that got even worse!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 18:03:48 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000/scheduler/api-docs/", host: "localhost:4000"
FYI, everything works fine in my browser if I visit the sites directly (after publishing the ports via Docker, etc.)

Ahem... the problem is a "greedy" regex. The regex capture group .+, as defined, will consume everything up to the last forward-slash.
You should use a "lazy" regex capture group .+? to capture just a single URI segment between two forward-slashes, but not everything to the last forward-slash!

Related

How to forward requests between docker containers from nginx reverse proxy to react routes in nginx?

I used the GitHub repo in the source below for setting up Dockerfiles and docker-compose and built on it.
How it works is that there is an Nginx reverse proxy that sends requests to the client(react) or backend(node js) depending on the URL.
This works fine for single-page React pages. I went and added multiple pages in a single react via react-routes-dom. I set it up like below and it works when I npm start the react code and access at localhost:3000/path.
function Main() {
return (
<Switch>
<Route path='/' exact component={ComponentA} />
<Route path='/path' exact component={ComponentB} />
</Switch>
);
}
The problem happens when I try to access it via the reverse proxy. The configuration is almost identical to the one here from the repo default.conf
The problem happens when I try to access the other routes.
If I try to access the base path localhost. It works.
If I try to access the path localhost/path, it does not work.
Logs for accessing base /
client | 172.18.0.5 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
nginx | 172.18.0.1 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
Logs for accessing custom /path
nginx | 172.18.0.1 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
client | 2021/04/06 11:52:43 [error] 31#31: *7 open() "/usr/share/nginx/html/path " failed (2: No such file or directory), client: 172.18.0.5, server: , request: "GET /path HTTP/1.0", host: "client"
client | 172.18.0.5 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.0" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
I tried to modify the conf file following React-router and nginx , https://gkedge.gitbooks.io/react-router-in-the-real/content/nginx.html , Nginx proxy_pass then try_file . They all don't work.
I tried to redirect all traffic to / to maybe help with the path, but I get an empty page.
location / {
rewrite /(.*) / break;
proxy_pass http://client;
}
Source: https://github.com/LukeMwila/multi-container-nginx-react-node-mongo
After re-thinking it through and reading this solution here https://stackoverflow.com/a/36623117/8293176, I realized that I misunderstood the concept of routing in React.
What I did before was that I tried to apply the static re-routing within the reverse proxy conf file to the Nginx hosting react which was incorrect! It just brought me to another page.
I had to apply it to the Nginx hosting the React build itself. This way, the redirects are client-side.
I applied the Catch-all method proposed in the link referenced above and the links in the Question, and it worked nicely!
I hope this post can provide clarity to future readers.

Nginx changing access logformat in nginx.conf has no effect

I am trying to change the log format /etc/nginx/nginx.conf to:
http {
log_format custom '3,$time_iso8601,$cookie_binuDid,,IPS,,0,$remote_addr,??,Unknown,N,N,$content_type,content,fetch,N'
'$status,Y,$upstream_response_time,$status,$upstream_response_time,$upstream_response_length,$upstream_response_length'
'"$upstream_http_cache-control",$request_uri,,,,,000000,0,Unknown'
access_log /var/log/nginx/access.log custom;
Yet my access log prints:
220.233.181.158 - - [08/Dec/2019:14:26:08 +0000] "GET /static/js/5.32912c95.chunk.js HTTP/1.1" 304 0 "http://example-3.com/auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
I did service nginx restart after the change, with no effect.
I guess there is another configuration file that needs to be changed, but I couldn't figure. Can someone please help? Am I doing something wrong?

Request to URL defined with ingress.kubernetes.io/auth-url annotation is done with HTTP/1.0

I am using auth-url and auth-signin annotation for authenticating access to app. Problem is that request to URL defined with auth-url is always done with HTTP/1.0 and not with HTTP/1.1 as expected. From logs you can see that all other requests are done with HTTP/1.1.
Version used: nginx-ingress-controller:0.9.0-beta.19
Logs from ELB:
2017-11-30T14:28:30.606436Z dev-sandbox-2cb4 201.137.96.59:58692 10.10.0.101:80 0.000044 0.031215 0.000039 302 302 0 154 "GET https://example.net:443/testing/ HTTP/1.1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
2017-11-30T14:28:30.623944Z dev-sandbox-2cb4 24.134.104.23:40704 10.10.7.144:80 0.000029 0.01263 0.000068 401 401 0 21 "GET https://example.net:443/oauth2/auth HTTP/1.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
2017-11-30T14:28:30.699239Z dev-sandbox-2cb4 201.137.96.59:58692 10.10.3.6:80 0.000028 0.001223 0.000046 302 302 0 395 "GET https://example.net:443/oauth2/start?rd=https://example.net/testing/ HTTP/1.1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
Annotation:
annotations:
ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start"
Problem is that in the environment I need to use only 1.1 is allowed.
Is this something to be expected or am I doing something wrong?
Issue can be solved by adding
proxy_http_version 1.1;
under location = {{ $authPath }} block in nginx ingress template.
See https://github.com/kubernetes/ingress-nginx/pull/1787.

fail2ban nginx ignoreregex not working

I have a rails app, and the requests related to assets are to be ignored. If I put the following regex in the ignoreregex setting, it doesnt match any of the lines, whereas, if I put it in the failregex, it correctly identifies the lines.
ignoreregex = (?i)^<HOST> - .* "GET .*/(assets|site_images|site_scripts)/.*
The example of the log line that I would like to count as to be "ignored":
XX.XX.XX.XX - - [30/Aug/2017:02:01:40 +0000] "GET /assets/logo-1a29bc0c23e29be7ca1f27d9fd90d735adb61e94562db7478d9f6c445205da5c.jpg HTTP/1.1" 200 32279 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) wkhtmltopdf_linux_amd64 Safari/534.34" "-"
Fail2Ban version: v0.9.3
Ubuntu 16.04.2 LTS

http proxy behind nginx redirecting to wrong url

I have nginx listening on a server to serve a website www.example.com
I'm trying to redirect www.example.com/iPython to an http-server which serves a temporary iPython notebook and is sitting on the server at port 8000 https://github.com/jupyterhub/configurable-http-proxy
server {
listen 80;
server_name http://xxx.xxx.xxx.xxx;
charset utf-8;
location /iPython {
proxy_pass http://xxx.xxx.xxx.xxx:8000/;
}
}
Here is the log:
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:09 +0000] "GET /iPythontest HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
web_1 | Not Found: /spawn/iPythontest
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:10 +0000] "GET /spawn/iPythontest HTTP/1.1" 404 9298 "http://xxx.xxx.xxx.xxx/iPythontest" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
The server then gets redirected to http://www.example.com/spawn/iPython rather than http://www.example.com/iPython/spawn which returns a 404 error on my site. I would like to make sure all traffic gets redirected in this way, but I'm not sure which directive can accomplish that

Resources