I am trying to run a search in kibana and capture specific http response codes (503).
As you can see, the below example has both a 302 and a 503 code.
XXX.XXX.XXX.XXX - - [08/Apr/2022:13:19:49 +0000] "GET /oauth/authorize/?client_id=XXX&code_challenge_method=S256&response_mode=query HTTP/2.0" 302 0 "https://XXXXXXXXXX/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36" 503 0.038 [XXXXXXXXXXXXX-service-http] [] 192.168.XXX.XXX:8080 0 0.040 302 XXXXXXXXXXXXX
I would like to run a search only for results matching a 503 as the first occurence.
Can I do that from a search query in the lucene query text field in kibana?
or do I need to go through the hassle of creating an index pattern etc... just to match that 503 error?
Thank you
I used the GitHub repo in the source below for setting up Dockerfiles and docker-compose and built on it.
How it works is that there is an Nginx reverse proxy that sends requests to the client(react) or backend(node js) depending on the URL.
This works fine for single-page React pages. I went and added multiple pages in a single react via react-routes-dom. I set it up like below and it works when I npm start the react code and access at localhost:3000/path.
function Main() {
return (
<Switch>
<Route path='/' exact component={ComponentA} />
<Route path='/path' exact component={ComponentB} />
</Switch>
);
}
The problem happens when I try to access it via the reverse proxy. The configuration is almost identical to the one here from the repo default.conf
The problem happens when I try to access the other routes.
If I try to access the base path localhost. It works.
If I try to access the path localhost/path, it does not work.
Logs for accessing base /
client | 172.18.0.5 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
nginx | 172.18.0.1 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
Logs for accessing custom /path
nginx | 172.18.0.1 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
client | 2021/04/06 11:52:43 [error] 31#31: *7 open() "/usr/share/nginx/html/path " failed (2: No such file or directory), client: 172.18.0.5, server: , request: "GET /path HTTP/1.0", host: "client"
client | 172.18.0.5 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.0" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
I tried to modify the conf file following React-router and nginx , https://gkedge.gitbooks.io/react-router-in-the-real/content/nginx.html , Nginx proxy_pass then try_file . They all don't work.
I tried to redirect all traffic to / to maybe help with the path, but I get an empty page.
location / {
rewrite /(.*) / break;
proxy_pass http://client;
}
Source: https://github.com/LukeMwila/multi-container-nginx-react-node-mongo
After re-thinking it through and reading this solution here https://stackoverflow.com/a/36623117/8293176, I realized that I misunderstood the concept of routing in React.
What I did before was that I tried to apply the static re-routing within the reverse proxy conf file to the Nginx hosting react which was incorrect! It just brought me to another page.
I had to apply it to the Nginx hosting the React build itself. This way, the redirects are client-side.
I applied the Catch-all method proposed in the link referenced above and the links in the Question, and it worked nicely!
I hope this post can provide clarity to future readers.
I've used Nginx docs and set access log format.
https://docs.nginx.com/nginx/admin-guide/monitoring/logging/
access_log /var/log/nginx/access.log;
log_format compression '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" "$gzip_ratio"';
When I access the site using Firefox I get the following log:
192.168.xx.xxx - - [13/Apr/2021:14:52:16 +0200] "GET /home HTTP/1.1" 200 21408 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0"
But when I access the site using Chrome or Opera I get following logs:
Opera:
192.168.xx.xxx - - [13/Apr/2021:14:53:19 +0200] "GET /me HTTP/1.1" 200 24062 "https://my-site.com/me" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 OPR/64.0.3417.92"
Chrome:
192.168.xx.xxx - - [13/Apr/2021:15:06:56 +0200] "GET /me HTTP/1.1" 200 115426 "https://my-site.com/me" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"
Did I properly set my Nginx logging configuration?
I have a small number of ASP.NET Core services, all running in Docker (via Docker Compose). All services are currently using a prefixed route (their own service name). And they're all setup in Docker Compose to use their own service name as their hostname (connectivity between service containers is OK).
The /api-docs endpoint is provided by Swashbuckle; we setup the prefixed route here, too.
app.UseSwagger(options =>
{
options.RouteTemplate = "scheduler/api-docs/{documentName}/swagger.json";
});
app.UseSwaggerUI(options =>
{
options.RoutePrefix = "scheduler/api-docs";
options.SwaggerEndpoint("/scheduler/api-docs/v1/swagger.json", "Scheduler API v1");
});
I am trying to configure an Nginx reverse-proxy in the container network so that I can go to, say...
http://localhost/<service-name>/api-docs
and it will redirect, inside the container network, to...
http://<service-name>:5000/<service-name>/api-docs
So, here's the Nginx configuration I've come up... basically, match the first part of the request URI, which should be the service name, and proxy to a host called the same and Nginx should add the $request_uri on automatically.
server {
listen 80;
location ~* ^/(?<target>.+)/ {
proxy_pass http://$target:5000;
proxy_redirect off;
resolver 127.0.0.11;
}
}
Here's what I get for a /scheduler/healthcheck endpoint. All good!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:50:24 +0000] "GET /scheduler/healthcheck HTTP/1.1" 200 491 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
But, when I try to go to /scheduler/api-docs, I have problems. We get to the service container and Swashbuckle does a 301 Redirect from /scheduler/api-docs to /scheduler/api-docs/.
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
And, then, things go bad... we "lose" the /scheduler part of the route!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 17:51:18 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000", host: "localhost:4000"
Why does the Swashbuckle redirect send the request back through Nginx, I thought this would all be handled by the local service, and why is Nginx stripping the necessary route prefix from this request?
How do I get this to behave?!
I tried to reconstruct the "whole" URI, just to see what happens...
- proxy_pass http://$target:5000;
+ proxy_pass http://$target:5000$request_uri;
And that got even worse!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 18:03:48 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000/scheduler/api-docs/", host: "localhost:4000"
FYI, everything works fine in my browser if I visit the sites directly (after publishing the ports via Docker, etc.)
Ahem... the problem is a "greedy" regex. The regex capture group .+, as defined, will consume everything up to the last forward-slash.
You should use a "lazy" regex capture group .+? to capture just a single URI segment between two forward-slashes, but not everything to the last forward-slash!
I have nginx listening on a server to serve a website www.example.com
I'm trying to redirect www.example.com/iPython to an http-server which serves a temporary iPython notebook and is sitting on the server at port 8000 https://github.com/jupyterhub/configurable-http-proxy
server {
listen 80;
server_name http://xxx.xxx.xxx.xxx;
charset utf-8;
location /iPython {
proxy_pass http://xxx.xxx.xxx.xxx:8000/;
}
}
Here is the log:
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:09 +0000] "GET /iPythontest HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
web_1 | Not Found: /spawn/iPythontest
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:10 +0000] "GET /spawn/iPythontest HTTP/1.1" 404 9298 "http://xxx.xxx.xxx.xxx/iPythontest" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
The server then gets redirected to http://www.example.com/spawn/iPython rather than http://www.example.com/iPython/spawn which returns a 404 error on my site. I would like to make sure all traffic gets redirected in this way, but I'm not sure which directive can accomplish that