I'm running a Solaris server to serve PHP through Apache. What tools can I use to measure the bandwidth my server is currently using? I use Google analytics to measure traffic, but as far as I know, it ignores file size. I have a rough idea of the average size of the pages I serve, and can do a back-of-the-envelope calculation of my bandwidth usage by multiplying page views (from Google) by average page size, but I'm looking for a solution that is more rigorous and exact.
Also, I'm not trying to throttle anything, or implement usage caps or anything like that. I'd just like to measure the bandwidth usage, so I know what it is.
An example of what I'm after is the usage meter that Slicehost provides in their admin website for their users. They tell me (for another site I run) how much bandwidth I've used each month and also divide the usage for uploading and downloading. So, it seems like this data can be measured, and I'd like to be able to do it myself.
To put it simply, what is the conventional method for measuring the bandwidth usage of my server?
This depends on your setup. If you have a (near-)dedicated physical interface for your web server you could gather stats straight from the interface.
Methods to do this could include SNMP (try net-snmp) or "ifconfig", combined with RRDTool or simple logging to flat files.
An alternative is using the Apache log, which could look like this:
192.168.101.155 - - [17/Apr/2005:20:39:19 -0700] "GET / HTTP/1.1" 200 1456
192.168.101.155 - - [17/Apr/2005:20:39:19 -0700] "GET /apache_pb.gif HTTP/1.1" 200 2326
192.168.101.155 - - [17/Apr/2005:20:39:19 -0700] "GET /favicon.ico HTTP/1.1" 404 303
192.168.101.155 - - [17/Apr/2005:20:39:42 -0700] "GET /index.html.ca HTTP/1.1" 200 1663
192.168.101.155 - - [17/Apr/2005:20:39:42 -0700] "GET /apache_pb.gif HTTP/1.1" 304 -
192.168.101.155 - - [17/Apr/2005:20:39:43 -0700] "GET /favicon.ico HTTP/1.1" 404 303
192.168.101.155 - - [17/Apr/2005:20:40:01 -0700] "GET /apache_pb.gif HTTP/1.1" 304 -
192.168.101.155 - - [17/Apr/2005:20:40:09 -0700] "GET /apache_pb.gift HTTP/1.1" 404 306
192.168.101.155 - - [17/Apr/2005:20:40:09 -0700] "GET /favicon.ico HTTP/1.1" 404 303
The last number is the amount of bytes transferred, excluding the header(!). See Apache Log Docs.
I am just guessing, but I think the usual approach is to use the same tools and services that are used to deliver QoS features. QoS == Quality of Service. Somewhere on the server itself, or on the network routers around the server, there will be services enabled that measure the size of the packets flowing out of your server. These same services can be used to limit the amount of bandwidth for customers that need to have such limitations enforced. I have not heard of an application that can be run on your server that measures bandwidth. I think it should be possible to create such an app, but that's not the usual way that such measurements are collected. I suspect this answer will end up not being solaris-specific.
Related
How do I configure the visualizer so it does not hang after sending a picture to the browser?
DisplaCy sends the picture to the browser, it is drawn in the browser, but then they do not respond to keystrokes and buttons. In the browser, it is only manages to close the page with the picture, but this is not sent to spaCy. The program waits for a response but does not get one, I have to crash execution to continue. After kernel`s interrupting execution continues normally. Execution in Spyder environment.
My actions: start the browser, Spyder, start the visualization program, after appearing in the console "Serving on http://0.0.0.0:5000 ..." enter the address in the browser - a picture appears and in the console lines: "127.0.0.0.1 - - [09/Feb/2023 19:06:28] "GET / HTTP/1.1" 200 3400". Further - only close.
import spacy
from spacy import displacy
print('\n begin') #
nlp = spacy.load("en_core_web_sm")
doc = nlp("This is a sentence.")
displacy.serve(doc, style="dep", options={"compact": True})
print('\n end') #`
console content:
begin
Using the 'dep' visualizer
Serving on http://0.0.0.0:5000 ...
127.0.0.1 - - [09/Feb/2023 19:06:28] "GET / HTTP/1.1" 200 3400
127.0.0.1 - - [09/Feb/2023 19:06:29] "GET /favicon.ico HTTP/1.1" 200 3400
127.0.0.1 - - [09/Feb/2023 19:06:29] "GET /favicon.ico HTTP/1.1" 200 3400
Shutting down server on port 5000.
end`
I'm confused by the different logs, one reporting http2, the other http 1.0.
I'm not sure which config file to cite. Or if it's a normal occurrence for puma's stdout redirect to show 1.0 for http version? Thank you.
nginx
==> /var/log/nginx/access.log <==
[10/Oct/2021:05:45:15 +0000] "GET /users/Ovbzv/quickrates/o5l05/payment/YabQ0/pending HTTP/2.0" 200
puma
==> app/log/stdout.log <==
[5626] 2604:a880:800:10::637:b005 - - [10/Oct/2021:05:45:15 +0000] "GET /users/Ovbzv/quickrates/o5l05/payment/YabQ0/pending HTTP/1.0" 200 - 0.0826
You are looking at two different connections:
the connection between the client and nginx (the reverse proxy); and
the connection between nginx and Puma;
In this specific case, each of these connections is using a different HTTP version, as indicated by the logs.
This is easily possible because HTTP/2 was specifically designed with some backwards compatibility in mind, allowing HTTP/2 to be converted to HTTP/1 when in need (and the same goes to converting HTTP/1 to HTTP/2).
I have a brand new Azure Load Balancer configured in private mode and VMSS (Single Server) configured with nginx and the default site. Any time I try to use the load balancer nginx returns a 400 response but if I use the server directly I get a 200 response.
Further looking at the access logs I see this ->
xxx.xxx.xxx.xxx - - [30/Jun/2021:17:51:48 +0000] "\x00" 400 166 "-" "-"
xxx.xxx.xxx.xxx - - [30/Jun/2021:17:51:51 +0000] "GET / HTTP/1.1" 304 0 "-" "{Browser Info ...}"
When using the load balancer, the path is \x00 instead of / - I'm not sure what is going on here or where to look.
This was caused by a private link service configured for TCP proxy V2 that was configured on the Load Balancer
A word press site i maintain, gets infected with .ico extension PHP scripts and their invocation links. I periodically remove them. Now i have written a cron job to find and remove them every minute. I am trying to find the source of this hack. I have closed all the back doors as far as i know ( FTP, DB users etc..).
After reading similar questions and looking at https://perishablepress.com/protect-post-requests/, now i think this could be because of malware POST requests. Monitoring the access log i see plenty of POST requests that fail with 40X response. But i also see requests that succeed which should not. Example one below, first request fails, similar POST Requests succeeds with 200 response few hours later.
I tried duplicating a similar request from https://www.askapache.com/online-tools/http-headers-tool/, but that fails with 40X response. Help me understand this behavior. Thanks.
POST Fails as expected
146.185.253.165 - - [08/Dec/2019:04:49:13 -0700] "POST / HTTP/1.1" 403 134 "http://website.com/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.24 (KHTML, like Gecko) RockMelt/0.9.58.494 Chrome/11.0.696.71 Safari/534.24" website.com
Few hours later same post succeeds
146.185.253.165 - - [08/Dec/2019:08:55:39 -0700] "POST / HTTP/1.1" 200 33827 "http://website.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.861.0 Safari/535.2" website.com
146.185.253.167 - - [08/Dec/2019:08:55:42 -0700] "POST / HTTP/1.1" 200 33827 "http://website.com/" "Mozilla/5.0 (Windows NT 5.1)
I checked this link but could understand as they have provided with a different format.
This is an entry in my access log.
127.0.0.1 - - [13/Aug/2012:13:39:53 +0530] "GET /cgi-bin/test.cgi HTTP/1.1" 200 48
I want to know the meaning of the last code.
I know that 200 id OK status code but what is 48 or any other number.... ?
Please help!
i think it means the size in bytes ..
Regards
mimiz