Update
After digging through my Apache access logs I believe the issue is related to hosting, not Google Analytics. Clicking a link to my site sometimes results in the Referer information getting dropped.
Here are a few example entries. Some contain a valid referer (Twitter, for example) while others just contain a -. For some of these cases, I was tailing the access log while clicking a link from another site so I know it should have a valid referer.
X.X.X.X - - [10/Jun/2019:03:06:10 +0000] "GET / HTTP/1.0" 200 12153 "https://twitter.com/PxJVBrEB7T" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:67.0) Gecko/20100101 Firefox/67.0"
X.X.X.X - - [10/Jun/2019:03:12:37 +0000] "GET / HTTP/1.0" 200 13535 "https://twitter.com/PTVIpLWqE9" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:67.0) Gecko/20100101 Firefox/67.0"
X.X.X.X - - [10/Jun/2019:03:50:39 +0000] "GET / HTTP/1.0" 200 12308 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:67.0) Gecko/20100101 Firefox/67.0"
This is a Plesk managed hosting account using Apache + Nginx cache. Is it possible Apache is misconfigured and dropping the referer information? Or is more likely something to do with the referring website?
There are multiple referring websites where links aren't showing up properly in the access log, so I suspect it's on the server side but I'm not sure what more to check.
Related
I do Web app on NextJS with Socket.IO.
When I run my app, everything is fine. But after 2 minutes errors appear.
NGINX logs:
172.19.0.1 - - [12/Sep/2022:16:27:39 +0000] "POST /api/ HTTP/1.1" 200 147 "http://localhost/en" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
172.19.0.1 - - [12/Sep/2022:16:27:59 +0000] "GET /_next/webpack-hmr HTTP/1.1" 101 269 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
...
172.19.0.1 - - [12/Sep/2022:16:29:21 +0000] "GET /_next/webpack-hmr HTTP/1.1" 101 71 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
172.19.0.1 - - [12/Sep/2022:16:29:26 +0000] "GET /_next/webpack-hmr HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15"
101 -- fine. 499 -- errors. But nothing happens, where are the errors coming from?
In console I see this error on each 499:
WebSocket connection to 'ws://localhost/_next/webpack-hmr' failed: WebSocket is closed before the connection is established
And this problem only on Safari (MacOS)!
On Chrome everything is okay.
From Next.js 12, HMR (Hot Module Replacement) in development uses a WebSocket connection.
When using Nginx with Next.js you'll have to configure it to pass through the WebSocket request properly.
In some cases when proxying requests to the Next.js dev server, you
will need to ensure the upgrade request is handled correctly. For
example, in nginx you would need to add the following configuration:
location /_next/webpack-hmr {
proxy_pass http://localhost:3000/_next/webpack-hmr;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
— Next.js, Upgrade Guide, Next.js' HMR connection now uses a WebSocket
I was checking my nginx access.log and I found bellow info.
From there is showing my redirection task: redirect to my site webcovid19.live from another portal sme.sk
How is this possible ? Is it hidden somewhere in HTML protocol ?
How webserver knows about redirection ?
Use Google Analytics the same logic ? direct vs referral
85.216.x.x - - [24/May/2020:08:50:52 +0000] "GET / HTTP/1.1" 200 1358 "https://domov.sme.sk/diskusie/3671287/2/koronavirus-
slovensko-minuta-po-minute-23-maj-2020.html" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0"
85.216.x.x - - [24/May/2020:08:50:52 +0000] "GET /styles.css HTTP/1.1" 200 725 "https://webcovid19.live/" "Mozilla/5.0 (X11
; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0"
Many Thanks Incognito :)
Issue solved:
en.wikipedia.org/wiki/HTTP_referer
I have a VPS running Nginx, which currently serves only static content.
Once I was looking at the log and noticed some strange requests:
216.244.66.239 - - [03/Jan/2019:15:04:26 +0100] "GET /en/profile/Souxy HTTP/1.1" 200 4650 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help#moz.com)"
216.244.66.239 - - [03/Jan/2019:15:04:28 +0100] "GET /en/view/8gIi2vad8Y HTTP/1.1" 200 4650 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help#moz.com)"
this is crawler. On this link is descriptin https://moz.com/help/moz-procedures/crawlers/dotbot. Maybe it is indexing your website.
You can block this requests on firewall or add file robots.txt with content
User-agent: dotbot
Disallow: /
I have a rails app, and the requests related to assets are to be ignored. If I put the following regex in the ignoreregex setting, it doesnt match any of the lines, whereas, if I put it in the failregex, it correctly identifies the lines.
ignoreregex = (?i)^<HOST> - .* "GET .*/(assets|site_images|site_scripts)/.*
The example of the log line that I would like to count as to be "ignored":
XX.XX.XX.XX - - [30/Aug/2017:02:01:40 +0000] "GET /assets/logo-1a29bc0c23e29be7ca1f27d9fd90d735adb61e94562db7478d9f6c445205da5c.jpg HTTP/1.1" 200 32279 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) wkhtmltopdf_linux_amd64 Safari/534.34" "-"
Fail2Ban version: v0.9.3
Ubuntu 16.04.2 LTS
when i try to add route with path /config, it shows 404 not found. Strange thing is that its not regular 404 symfony error that shows when i enter non existing route Here is apache access log:
127.0.0.1 - - [03/Feb/2015:11:32:26 +0100] "GET /config HTTP/1.1" 404 499 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
127.0.0.1 - - [03/Feb/2015:11:35:00 +0100] "GET /configasd HTTP/1.1" 404 743 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
127.0.0.1 - - [03/Feb/2015:11:36:42 +0100] "GET /configasdasd HTTP/1.1" 404 743 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0"
You can see that accessing /config generates 404 499 error code while accessing another non existing route generates 404 743 error code.
My question is: is "config" an reserved word for using in routes ? Is there an complete list of such words in symfony ?
EDIT: Route configuration:
in app/config/routing.yml:
myapp_config:
resource: "#MyappConfigBundle/Resources/config/routing.yml"
prefix: /config
in MyappConfigBundle/Resources/config/routing.yml:
myapp_config:
path: /
defaults: { _controller: MyappConfigBundle:Config:index }
The status code for response is only 404, 499 and 743 - are sizes of response.
Your server is configured to have at /config path some other resource. It may be configured with some global alias or you can just have file/folder or symlink/hardlink with name config in your web directory.
Check all the cases and you will solve your problem.