I am trying to run a search in kibana and capture specific http response codes (503).
As you can see, the below example has both a 302 and a 503 code.
XXX.XXX.XXX.XXX - - [08/Apr/2022:13:19:49 +0000] "GET /oauth/authorize/?client_id=XXX&code_challenge_method=S256&response_mode=query HTTP/2.0" 302 0 "https://XXXXXXXXXX/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36" 503 0.038 [XXXXXXXXXXXXX-service-http] [] 192.168.XXX.XXX:8080 0 0.040 302 XXXXXXXXXXXXX
I would like to run a search only for results matching a 503 as the first occurence.
Can I do that from a search query in the lucene query text field in kibana?
or do I need to go through the hassle of creating an index pattern etc... just to match that 503 error?
Thank you
Related
I used the GitHub repo in the source below for setting up Dockerfiles and docker-compose and built on it.
How it works is that there is an Nginx reverse proxy that sends requests to the client(react) or backend(node js) depending on the URL.
This works fine for single-page React pages. I went and added multiple pages in a single react via react-routes-dom. I set it up like below and it works when I npm start the react code and access at localhost:3000/path.
function Main() {
return (
<Switch>
<Route path='/' exact component={ComponentA} />
<Route path='/path' exact component={ComponentB} />
</Switch>
);
}
The problem happens when I try to access it via the reverse proxy. The configuration is almost identical to the one here from the repo default.conf
The problem happens when I try to access the other routes.
If I try to access the base path localhost. It works.
If I try to access the path localhost/path, it does not work.
Logs for accessing base /
client | 172.18.0.5 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
nginx | 172.18.0.1 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
Logs for accessing custom /path
nginx | 172.18.0.1 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
client | 2021/04/06 11:52:43 [error] 31#31: *7 open() "/usr/share/nginx/html/path " failed (2: No such file or directory), client: 172.18.0.5, server: , request: "GET /path HTTP/1.0", host: "client"
client | 172.18.0.5 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.0" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
I tried to modify the conf file following React-router and nginx , https://gkedge.gitbooks.io/react-router-in-the-real/content/nginx.html , Nginx proxy_pass then try_file . They all don't work.
I tried to redirect all traffic to / to maybe help with the path, but I get an empty page.
location / {
rewrite /(.*) / break;
proxy_pass http://client;
}
Source: https://github.com/LukeMwila/multi-container-nginx-react-node-mongo
After re-thinking it through and reading this solution here https://stackoverflow.com/a/36623117/8293176, I realized that I misunderstood the concept of routing in React.
What I did before was that I tried to apply the static re-routing within the reverse proxy conf file to the Nginx hosting react which was incorrect! It just brought me to another page.
I had to apply it to the Nginx hosting the React build itself. This way, the redirects are client-side.
I applied the Catch-all method proposed in the link referenced above and the links in the Question, and it worked nicely!
I hope this post can provide clarity to future readers.
I am trying to change the log format /etc/nginx/nginx.conf to:
http {
log_format custom '3,$time_iso8601,$cookie_binuDid,,IPS,,0,$remote_addr,??,Unknown,N,N,$content_type,content,fetch,N'
'$status,Y,$upstream_response_time,$status,$upstream_response_time,$upstream_response_length,$upstream_response_length'
'"$upstream_http_cache-control",$request_uri,,,,,000000,0,Unknown'
access_log /var/log/nginx/access.log custom;
Yet my access log prints:
220.233.181.158 - - [08/Dec/2019:14:26:08 +0000] "GET /static/js/5.32912c95.chunk.js HTTP/1.1" 304 0 "http://example-3.com/auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
I did service nginx restart after the change, with no effect.
I guess there is another configuration file that needs to be changed, but I couldn't figure. Can someone please help? Am I doing something wrong?
I am trying to strip password from my nginx/openresty POST data logs.
Post data logs looks like this
"POST /myapi/login/ HTTP/1.1" 200 37 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/75.0.3770.100 Safari/537.36" 0.000
req_body:"----password\x22\x0D\x0A\x0D\x0Atestuser\x0D\x0A------9\x0D\x0AContent-Disposition: form-data;
resp_body:"{success: \x22login succesful\x22}\x0A" upstream_response_time=- msec=1562036599.391 upstream_connect_time=- request_time=0.000 status=200
Here I want to replace "testuser" with "XXXXX" so that the user password is not visible to anyone
Whenever I'm trying to open web application my Nginx access log shows "GET /false HTTP/1.1" 404 206 "https://www.example.com/FeedifySW.js" "Mozilla/5.0 (Linux; Android 6.0; Micromax Q4260 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.85 Mobile Safari/537.36" - -
I have searched through all way possible and still couldn't find the solution. Can anyone help?
This is for an nginx server where i have hosted my web application. due to high request counts, sometimes the application crashes
"GET /false HTTP/1.1" 404 206 "https://www.example.com/FeedifySW.js" "Mozilla/5.0 (Linux; Android 6.0; Micromax Q4260 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.85 Mobile Safari/537.36" - -
I have a small number of ASP.NET Core services, all running in Docker (via Docker Compose). All services are currently using a prefixed route (their own service name). And they're all setup in Docker Compose to use their own service name as their hostname (connectivity between service containers is OK).
The /api-docs endpoint is provided by Swashbuckle; we setup the prefixed route here, too.
app.UseSwagger(options =>
{
options.RouteTemplate = "scheduler/api-docs/{documentName}/swagger.json";
});
app.UseSwaggerUI(options =>
{
options.RoutePrefix = "scheduler/api-docs";
options.SwaggerEndpoint("/scheduler/api-docs/v1/swagger.json", "Scheduler API v1");
});
I am trying to configure an Nginx reverse-proxy in the container network so that I can go to, say...
http://localhost/<service-name>/api-docs
and it will redirect, inside the container network, to...
http://<service-name>:5000/<service-name>/api-docs
So, here's the Nginx configuration I've come up... basically, match the first part of the request URI, which should be the service name, and proxy to a host called the same and Nginx should add the $request_uri on automatically.
server {
listen 80;
location ~* ^/(?<target>.+)/ {
proxy_pass http://$target:5000;
proxy_redirect off;
resolver 127.0.0.11;
}
}
Here's what I get for a /scheduler/healthcheck endpoint. All good!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:50:24 +0000] "GET /scheduler/healthcheck HTTP/1.1" 200 491 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
But, when I try to go to /scheduler/api-docs, I have problems. We get to the service container and Swashbuckle does a 301 Redirect from /scheduler/api-docs to /scheduler/api-docs/.
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
And, then, things go bad... we "lose" the /scheduler part of the route!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 17:51:18 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000", host: "localhost:4000"
Why does the Swashbuckle redirect send the request back through Nginx, I thought this would all be handled by the local service, and why is Nginx stripping the necessary route prefix from this request?
How do I get this to behave?!
I tried to reconstruct the "whole" URI, just to see what happens...
- proxy_pass http://$target:5000;
+ proxy_pass http://$target:5000$request_uri;
And that got even worse!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 18:03:48 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000/scheduler/api-docs/", host: "localhost:4000"
FYI, everything works fine in my browser if I visit the sites directly (after publishing the ports via Docker, etc.)
Ahem... the problem is a "greedy" regex. The regex capture group .+, as defined, will consume everything up to the last forward-slash.
You should use a "lazy" regex capture group .+? to capture just a single URI segment between two forward-slashes, but not everything to the last forward-slash!