I have a rails app, and the requests related to assets are to be ignored. If I put the following regex in the ignoreregex setting, it doesnt match any of the lines, whereas, if I put it in the failregex, it correctly identifies the lines.
ignoreregex = (?i)^<HOST> - .* "GET .*/(assets|site_images|site_scripts)/.*
The example of the log line that I would like to count as to be "ignored":
XX.XX.XX.XX - - [30/Aug/2017:02:01:40 +0000] "GET /assets/logo-1a29bc0c23e29be7ca1f27d9fd90d735adb61e94562db7478d9f6c445205da5c.jpg HTTP/1.1" 200 32279 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) wkhtmltopdf_linux_amd64 Safari/534.34" "-"
Fail2Ban version: v0.9.3
Ubuntu 16.04.2 LTS
Related
I do Web app on NextJS with Socket.IO.
When I run my app, everything is fine. But after 2 minutes errors appear.
NGINX logs:
172.19.0.1 - - [12/Sep/2022:16:27:39 +0000] "POST /api/ HTTP/1.1" 200 147 "http://localhost/en" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
172.19.0.1 - - [12/Sep/2022:16:27:59 +0000] "GET /_next/webpack-hmr HTTP/1.1" 101 269 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
...
172.19.0.1 - - [12/Sep/2022:16:29:21 +0000] "GET /_next/webpack-hmr HTTP/1.1" 101 71 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
172.19.0.1 - - [12/Sep/2022:16:29:26 +0000] "GET /_next/webpack-hmr HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
101 -- fine. 499 -- errors. But nothing happens, where are the errors coming from?
In console I see this error on each 499:
WebSocket connection to 'ws://localhost/_next/webpack-hmr' failed: WebSocket is closed before the connection is established
And this problem only on Safari (MacOS)!
On Chrome everything is okay.
From Next.js 12, HMR (Hot Module Replacement) in development uses a WebSocket connection.
When using Nginx with Next.js you'll have to configure it to pass through the WebSocket request properly.
In some cases when proxying requests to the Next.js dev server, you
will need to ensure the upgrade request is handled correctly. For
example, in nginx you would need to add the following configuration:
location /_next/webpack-hmr {
proxy_pass http://localhost:3000/_next/webpack-hmr;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
— Next.js, Upgrade Guide, Next.js' HMR connection now uses a WebSocket
I used the GitHub repo in the source below for setting up Dockerfiles and docker-compose and built on it.
How it works is that there is an Nginx reverse proxy that sends requests to the client(react) or backend(node js) depending on the URL.
This works fine for single-page React pages. I went and added multiple pages in a single react via react-routes-dom. I set it up like below and it works when I npm start the react code and access at localhost:3000/path.
function Main() {
return (
<Switch>
<Route path='/' exact component={ComponentA} />
<Route path='/path' exact component={ComponentB} />
</Switch>
);
}
The problem happens when I try to access it via the reverse proxy. The configuration is almost identical to the one here from the repo default.conf
The problem happens when I try to access the other routes.
If I try to access the base path localhost. It works.
If I try to access the path localhost/path, it does not work.
Logs for accessing base /
client | 172.18.0.5 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
nginx | 172.18.0.1 - - [06/Apr/2021:11:51:15 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
Logs for accessing custom /path
nginx | 172.18.0.1 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
client | 2021/04/06 11:52:43 [error] 31#31: *7 open() "/usr/share/nginx/html/path " failed (2: No such file or directory), client: 172.18.0.5, server: , request: "GET /path HTTP/1.0", host: "client"
client | 172.18.0.5 - - [06/Apr/2021:11:52:43 +0000] "GET /path HTTP/1.0" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36" "-"
I tried to modify the conf file following React-router and nginx , https://gkedge.gitbooks.io/react-router-in-the-real/content/nginx.html , Nginx proxy_pass then try_file . They all don't work.
I tried to redirect all traffic to / to maybe help with the path, but I get an empty page.
location / {
rewrite /(.*) / break;
proxy_pass http://client;
}
Source: https://github.com/LukeMwila/multi-container-nginx-react-node-mongo
After re-thinking it through and reading this solution here https://stackoverflow.com/a/36623117/8293176, I realized that I misunderstood the concept of routing in React.
What I did before was that I tried to apply the static re-routing within the reverse proxy conf file to the Nginx hosting react which was incorrect! It just brought me to another page.
I had to apply it to the Nginx hosting the React build itself. This way, the redirects are client-side.
I applied the Catch-all method proposed in the link referenced above and the links in the Question, and it worked nicely!
I hope this post can provide clarity to future readers.
When I access a page in the browser I get a proper 200 from the server:
xx.xxx.xxx.xxx - - [02/May/2019:19:53:50 +0200] "GET /retourneren HTTP/1.1" 200 2889 "https://mysite.nl/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15" "-"
However when I add the url in prerender I get a 400:
3.90.111.223 - - [02/May/2019:19:50:39 +0200] "GET /retourneren HTTP/1.1" 404 10050 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3163.59 Safari/537.36 Prerender (+https://github.com/prerender/prerender)" "-"
therefor the page is not getting cached. Does anyone have an idea?
That seems like you might be setting the prerender-status-code meta tag on the page and setting it to "404", which would make Prerender.io return a 404 response code directly.
Can you confirm whether or not that meta tag is being set in the HTML of the page?
I have a small number of ASP.NET Core services, all running in Docker (via Docker Compose). All services are currently using a prefixed route (their own service name). And they're all setup in Docker Compose to use their own service name as their hostname (connectivity between service containers is OK).
The /api-docs endpoint is provided by Swashbuckle; we setup the prefixed route here, too.
app.UseSwagger(options =>
{
options.RouteTemplate = "scheduler/api-docs/{documentName}/swagger.json";
});
app.UseSwaggerUI(options =>
{
options.RoutePrefix = "scheduler/api-docs";
options.SwaggerEndpoint("/scheduler/api-docs/v1/swagger.json", "Scheduler API v1");
});
I am trying to configure an Nginx reverse-proxy in the container network so that I can go to, say...
http://localhost/<service-name>/api-docs
and it will redirect, inside the container network, to...
http://<service-name>:5000/<service-name>/api-docs
So, here's the Nginx configuration I've come up... basically, match the first part of the request URI, which should be the service name, and proxy to a host called the same and Nginx should add the $request_uri on automatically.
server {
listen 80;
location ~* ^/(?<target>.+)/ {
proxy_pass http://$target:5000;
proxy_redirect off;
resolver 127.0.0.11;
}
}
Here's what I get for a /scheduler/healthcheck endpoint. All good!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:50:24 +0000] "GET /scheduler/healthcheck HTTP/1.1" 200 491 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
But, when I try to go to /scheduler/api-docs, I have problems. We get to the service container and Swashbuckle does a 301 Redirect from /scheduler/api-docs to /scheduler/api-docs/.
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
And, then, things go bad... we "lose" the /scheduler part of the route!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:17:51:18 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 17:51:18 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000", host: "localhost:4000"
Why does the Swashbuckle redirect send the request back through Nginx, I thought this would all be handled by the local service, and why is Nginx stripping the necessary route prefix from this request?
How do I get this to behave?!
I tried to reconstruct the "whole" URI, just to see what happens...
- proxy_pass http://$target:5000;
+ proxy_pass http://$target:5000$request_uri;
And that got even worse!
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 172.19.0.1 - - [27/Mar/2018:18:03:48 +0000] "GET /scheduler/api-docs/ HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" "-"
api-gateway_1 | 2018/03/27 18:03:48 [error] 5#5: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /scheduler/api-docs/ HTTP/1.1", upstream: "http://172.19.0.5:80/api-docs:5000/scheduler/api-docs/", host: "localhost:4000"
FYI, everything works fine in my browser if I visit the sites directly (after publishing the ports via Docker, etc.)
Ahem... the problem is a "greedy" regex. The regex capture group .+, as defined, will consume everything up to the last forward-slash.
You should use a "lazy" regex capture group .+? to capture just a single URI segment between two forward-slashes, but not everything to the last forward-slash!
when i try to add route with path /config, it shows 404 not found. Strange thing is that its not regular 404 symfony error that shows when i enter non existing route Here is apache access log:
127.0.0.1 - - [03/Feb/2015:11:32:26 +0100] "GET /config HTTP/1.1" 404 499 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
127.0.0.1 - - [03/Feb/2015:11:35:00 +0100] "GET /configasd HTTP/1.1" 404 743 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
127.0.0.1 - - [03/Feb/2015:11:36:42 +0100] "GET /configasdasd HTTP/1.1" 404 743 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
You can see that accessing /config generates 404 499 error code while accessing another non existing route generates 404 743 error code.
My question is: is "config" an reserved word for using in routes ? Is there an complete list of such words in symfony ?
EDIT: Route configuration:
in app/config/routing.yml:
myapp_config:
resource: "#MyappConfigBundle/Resources/config/routing.yml"
prefix: /config
in MyappConfigBundle/Resources/config/routing.yml:
myapp_config:
path: /
defaults: { _controller: MyappConfigBundle:Config:index }
The status code for response is only 404, 499 and 743 - are sizes of response.
Your server is configured to have at /config path some other resource. It may be configured with some global alias or you can just have file/folder or symlink/hardlink with name config in your web directory.
Check all the cases and you will solve your problem.