I'm trying to write down an Azure Log analytics query that would show me the average CPU usage in the past 30 days for my Azure virtual machines.
With the following query I have some results, but not really what I'm looking for:
Perf
| where ObjectName == 'Processor' and CounterName == '% Processor Time' and InstanceName == '_Total'
| summarize CPUAvarage = avg(CounterValue) by Computer, bin(TimeGenerated, 1h)
Result:
The result are are in the correct form, but the TimeGenerated should be last 30 days and "CPUAvarage" should display 1 number for past 30 days (30 days average cpu). If I'm correct query should add all values together past 30 days and then divide it by count but my with my current Kusto skills I'm not able to do this.
I don't fully understand what you're trying to achieve, so here are your options:
If you want the result to be per day, and not per hour, then you should replace bin(TimeGenerated, 1h) with bin(TimeGenerated, 1d), because 1h is an hour, and 1d is a day.
If you want the average CPU per computer over the whole month, then replace your summarize line with | summarize CPUAvarage = avg(CounterValue) (note that I removed the bin part).
If you want the average CPU per day for all your computers (rather than per computer), then replace your summarize line with | summarize CPUAvarage = avg(CounterValue) by bin(TimeGenerated, 1d).
Related
I am running KQL (Kusto query language) queries against Azure Application Insights. I have certain measurements that I want to aggregate weekly. I am trying to figure out how to split my data into weeks.
To illustrate what I seek, here is a query that computes daily averages of the duration column.
requests
| where timestamp > ago(7d)
| summarize
avg(duration)
by
Date = format_datetime(timestamp, "yyyy-MM-dd")
This produces something similar to this:
In the above I have converted datetimes to string and thus effectively "rounded them down" to the precision of one day. This may be ugly, but it's the easiest way I could think of in order to group all results from a given day. It would be trivial to round down to months or years with the same technique.
But what if I want to group datetimes by week? Is there a nice way to do that?
I do not care whether my "weeks" start on Monday or Sunday or January 1st or whatever. I just want to group a collection of KQL datetimes into 7-day chunks. How can I do that?
Thanks in advance!
Looks like you are looking for the "bin()" function:
requests
| where timestamp > ago(7d)
| summarize
avg(duration)
by
bin(timestamp, 1d) // one day, for 7 days change it to 7d
I found out that I can use the week_of_year function to split datetimes by week number:
requests
| where timestamp > ago(30d)
| summarize
avg(duration)
by
Week = week_of_year(timestamp)
| sort by Week
I am having a hard time creating a SQLite command that will return an evenly spaced out data points based on time if the number of data points is beyond 50.
Basically, I am having data stored every 30 seconds. However, if I want to see the data in the last hour, that will be a very large amount of data and will end up freezing my RPI as it attempts to visualize this data. So, my solution is to create a SQLite command that will limit the number of return data points to 50 that are spread evenly across a time range.
I have separate commands for if I want last min, 5 mins, 10 mins, etc. Once it goes beyond 1 hour, I need to limit the data so I can hard code this into the command (no need for IF statements)
Here is my currently attempt at the command which is not working:
select Voltage from Battery2 where Timestamp >= Datetime('now', '-1 hour') % (SELECT COUNT(*)/50 FROM Battery2)=0;
This is based on this stack overflow post: How to select fixed number of evenly spread rows in timeseries sqlite database
EDIT:
Here is some sample data from the output of the function:
Voltage: 54
Timestamp: "2022-01-13 16:47:47"
Voltage: 54
Timestamp: "2022-01-13 16:48:18"
Voltage: 54
Timestamp: "2022-01-13 16:48:49"
You can use NTILE() window function to divide the resultset in 50 groups, based on the column Timestamp and then with aggregation pick 1 row from each group with MAX() or MIN() aggregate function:
WITH cte AS (
SELECT *, NTILE(50) OVER (ORDER BY Timestamp) nt
FROM Battery2
WHERE Timestamp >= datetime('now', '-1 hour')
)
SELECT MAX(Timestamp) AS Timestamp, Voltage
FROM cte
GROUP BY nt;
I have a requirement where I need to regularize/aggregate data which is polled every 1 sec into 1 min intervals. And I have two columns which need to be aggregated as well, say SensorName, SensorValue. I am able to bin the timestamp to 1 minute, but I am not able to get the corresponding two colums. How do I do that? Below is the query I used and the output I get.
Table
| where TimeStamp between (datetime(2020-09-01)..datetime(2020-09-30))
| summarize by bin(TimeStamp , 1min)
based on my understanding of the question (could be wrong, as there's no clear specification of sample input/schema and matching output), you could try following this example - it calculates the average sensor value for each sensor name, using an aggregation span of 1 minute:
Table
| where TimeStamp between (datetime(2020-09-01)..datetime(2020-09-30))
| summarize avg(SensorValue) by SensorName, bin(TimeStamp, 1min)
Here are my table's columns :
Time | Close | High | Low | Open | pairVolume | Trades | Volume
I would love to have my data group by range of time.
Now the tricky part is that this range is custom (it's a user input which could very well be grouping by 10 minutes, 2 hours, or even 5 days)
My time field is stored in millisecond since epoch.
Solution I found for now which I'm uncertain about :
SELECT time + (21600000 - (time%21600000)) as gap, count(time)
FROM price_chart
WHERE time >= 1517418000000 and time <= 1518195600000
GROUP BY gap
21600000 is 6 hours in milliseconds
time is time since epoch
Yes, it works.
Putting some numbers into excel with your formula below, it works for me. Your gap value will be returned as the top end of each time range grouping.
SELECT time + (21600000 - (time%21600000)) as gap ...
Using the below:
SELECT time - (time%21600000) as gap_bottom ...
Would return you the bottom end of each time range grouping. You could add this as an additional calculated column and have both returned.
EDIT / PS:
You can also use the SQLite date formatting functions after dividing 1,000 milliseconds out of your epoch time and converting it to the SQLite unixepoch:
strftime('%Y-%m-%d %H:%M:%S', datetime(1517418000000 / 1000, 'unixepoch') )
... for ...
SELECT strftime('%Y-%m-%d %H:%M:%S', datetime( (time + (21600000 - (time%21600000))) / 1000, 'unixepoch') ) as gap ...
How can I render a chart for the query
performanceCounters
| where name == "% Processor Time"
| summarize avg(value) by bin(timestamp, 5s),cloud_RoleInstance
where i get a point for every 5 seconds and not every 1 min?
Perf counters are collected at a regular interval (about 1 min) the effect of the bin function will move the time stamp to the nearest 5 second interval. What you're seeing is because of the counter collection interval and you won't get that granularity. You would need to implement your own module to do that.
https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/v2.3.0/Src/PerformanceCollector/Shared/PerformanceCollectorModule.cs#L49