I am putting together a dashboard for our production services and I am trying to create a unified chart of response performance across multiple services.
Now, I can get the the query to run but it created a STACKED column chart but I cannot seem to get it to UNSTACK unless I manually go to the chart properties and change it, but this is not a persistent change as it will reset when you reload / refresh the page.
Here's my query
// Create a combined dataset
let mainTable = union pageViews, customEvents, requests
| where isempty(operation_SyntheticSource)
| extend name =replace("\n", "", name)
| extend name =replace("\r", "", name);
// Generate chart data
mainTable
| where duration > 1000 // Not interested in any responses under 1 second
| parse (duration / 1000) // Take the whole number of seconds
with P1: int
"."
notInterested: string
| extend peformanceBucket = iff(P1 > 4, ">4", tostring(P1))
| project name, duration, peformanceBucket
| summarize Count = count() by peformanceBucket, name
| order by peformanceBucket
| render columnchart
And here's the output
But this is what I want (by default)
I can get the the query to run but it created a STACKED column chart but I cannot seem to get it to UNSTACK unless I manually go to the chart properties and change it, but this is not a persistent change as it will reset when you reload / refresh the page.
If you want to show the Unstacked Chart Type use the Kind property has Unstacked.
Workaround follows:
The query I have used in my Application Insights
exceptions
| summarize count = sum(itemCount) by bin(timestamp, 3h), problemId
| order by timestamp asc, problemId
| render columnchart kind=unstacked
Result
After Adding in Dashboard
Related
In Azure Monitor for Application Insights how do you make a bar chart using a search term where the results are grouped by day?
If you have large logs in Application Insights but want to print a bar chart where they are grouped by day using the Kusto language.
// replace "my search term" with yours
union isfuzzy=true
availabilityResults,
requests,
exceptions,
pageViews,
traces,
customEvents,
dependencies
| where * has "my search term"
| summarize Requests = count() by day = toint(format_datetime(timestamp, 'd'))
| order by day asc
| render barchart
union
This pulls together all the different types of Azure Monitor log records, you can comment some out if you are only looking for traces.
where
the * operator looks in all the searchable properties inside each element in the union
summarize
this is the equivalent of a group by statement where we build the x-axis of the bar as the Requests property and then group them by day for the y-axis. The timestamp is converted to a date of the month value (1,2,3...31) and then converted to integer
order by
order the elements by the day ascending
render
render by barchart, remember there are Chart Formatting settings next to the Chart in Azure Monitor where you can fine tune the output
date range
the date range is handled by the Time range control above the Azure Monitor log editor window.
Visualizations charts
Kusto reference
I have a .net application uploading customEvents to application insights. I'd like to create an alert to trigger when a scanning (like a heartbeat) message hasn't been sent.
If I run the following log query I get back what I think looks like something I can alert on:
app('dev-insights').customEvents
| where name == "Scanning"
| project-rename TimeGenerated = timestamp
| make-series kind=nonempty counts=count() default=0
on TimeGenerated in range(ago(1h), now()-5m, 5m)
| mv-expand TimeGenerated, counts
| project todatetime(TimeGenerated), toint(counts)
| summarize AggregatedValue=avg(counts) by bin(TimeGenerated, 5m)
And I get results like this:
But with the following in azure alerts it never seems to fire:
Is this something that should be possible? I notice that the graph in the alert preview never looks right which I suspect is something to do with the issue, but seeing as the raw log query looks ok I'm a bit stuck.
Ideally I guess I'd prefer this to be a metric alert as I think if even I can get this to work I'll get an alert sent every time the condition is true rather than just one when it triggers and one when it resolves. Is there any way to achieve that?
We have an web application. Part of web the application is profiles. Profiles have following urls:
/Profile/1
/Profile/2
/Profile/3
We also have Adds. Add's urls are as following:
/Add/1
/Add/2
/Add/3
I need a view of top 20 most viewed profiles and top 20 most viewed add's and how many views they have. It must also be possible to extract those two lists with c#. Can this be achieved with log analytics? Do you need to extract everything your self from application insights and do analytics your self, or how would you achieve this?
I made a little POC. I have added following to AI. The https://gac/url/1 has thousands of id's. Maybe there us 244 views on https://gac/url/1 and 1128 views on https://gac/url/2and so forth.
I noticed that Id is not added to the log. See code below:
public void WritePageView(string name, string id, Uri url)
{
var pageView = new PageViewTelemetry
{
Name = name,
Id = id,
Url = url
};
pageView.Properties.Add("Id", "1");
telemetry.TrackPageView(pageView);
}
What I need is a top 20 (top x) of which https://gac/url is most visited. Something like:
https://gac/url1: 3434
https://gac/url1: 2432
https://gac/url1: 1298
https://gac/url1: 8211
..
If showed in a graph to would be really great.
So it would be something like group by name and Id. But as Id is only pressent in the url how do I extract that?
It's easy to use analytics query for application insights.
Nav to your azure portal -> application insights -> Logs(Analytics), the related info should be in the pageViews table. When you write your query, you can set custom "Time Range" via UI or use where clause.
You can get all the pageViews info by just write "pageViews", screenshot as below:
And you can use Summarize operator and count operator to get the total views count for each page. Sample query code like below:
pageViews
| summarize totalCount = count(url) by url
| order by totalCount desc
The result as below:
If you want to only get the top 2 most views page, use the following code:
pageViews
| summarize totalCount = count(url) by url
| order by totalCount desc
| top 2 by totalCount
in my BigQuery project I store event data integrated from Firebase. The granularity and dimension is such that trying to present raw data in Data Studio quickly makes the report become VERY slow (1-2 min per page/interaction).
I then started to think how I could create pre-aggregated tables in BigQuery to speed everything up, but quickly realised COUNT DISTINCT metrics would be a problem with this approach.
Let me explain:
SELECT user, date
FROM UNNEST([
STRUCT("Adam" AS user, "20190923" AS date),
("Bob", "20190923"),
("Carl", "20190923"),
("Adam", "20190924"),
("Bob", "20190924"),
("Adam", "20190925"),
("Carl", "20190925"),
("Bob", "20190926")
]) AS website_visits;
+------+----------+
| User | Date |
+------+----------+
| Adam | 20190923 |
| Bob | 20190923 |
| Carl | 20190923 |
| Adam | 20190924 |
| Bob | 20190924 |
| Adam | 20190925 |
| Carl | 20190925 |
| Bob | 20190926 |
+------+----------+
The above is a table of website visits.
Clearly, creating a pre-aggregated table like
SELECT date, COUNT(DISTINCT user) FROM website_visits GROUP BY date
has the limitation that the count cannot be aggregated further (or even less, dinamically) to get a total, as doing a SUM would return 8 unique users which is not correct, there are only 3 unique users.
In BigQuery, this is fixed by using HLL_COUNT, which despite the approximation works ok for me.
Now to the big question:
How to do the same so that the result is displayable in Data Studio????
HLL_COUNT.EXTRACT is not available as function in there, and in the reporting I always have to keep in mind that the date range is set by the user however (s)he likes so it's not possible to store a pre-aggregated result for ALL cases...
EDIT 1: APPROX_COUNT_DISTINCT
As per answer from Bobbylank, I tried to use APPROX_COUNT_DISTINCT.
However I found that this just seems to move the issue down the line. My fault for not explaining what's over there.
Despite being performances acceptable it does not seem possible to me to blend a data source with this calculated metric.
Example: After displaying the amount of unique users in the selected period (which now works), I'm also trying to display Average Revenue Per User (ARPU) in Data Studio like Firebase does.
To do this, I have to SUM(REVENUE) / APPROX_COUNT_DISTINCT(USER)
Clearly, REVENUE works ok with pre-aggregation and is available in the raw data. I tried then to blend the raw data with a table containing just user visits. However APPROX_COUNT_DISTINCT can't be used in the blended data definition as calculated metrics are not allowed.
Even trying to use the USER field as a metric with Count Distinct aggregation, despite returning the correct figures when showing revenue and user count separately, when I try to divide them the problem becomes aggregation (apply SUM or AVG to the field and basically the result will be AVG(REVENUE/USERS) for each day).
I also then tried to store REVENUE directly in the visits table, but was reminded by Data Studio that I can't create calculated metrics that I can't mix dimensions and metrics in a calculated field.
APPROX_COUNT_DISTINCT might be more performance friendly for you?
https://support.google.com/datastudio/answer/9189108?hl=en
Otherwise the only way I can think would be to pre-calculate several metrics (e.g. unique users on that day, 7-day cumulative, 14-day, etc.) as your customer require for each single day.
Or you could provide a 2 page report with both of these methods with the caveat that the first can be used over a time period but will be much slower?
We have a custom event put in place on page which tracks the link clicks on given page to app insights. And with the REST API we would like to get the frequently accessed links from app insights.
How can we build the Query to get this analytics data, any sample on reading custom events available?
Thanks
if you open the Application Insights Analytics website for any resource, there's some "Common Queries" examples right on the front page. one of them is called "Usage" and if you click it it will show you this one:
//What are the top 10 custom events of your application in the past 24 hours?
customEvents
| where timestamp >= ago(24h)
| summarize dcount(user_Id), count() by name
| top 10 by count_
| render barchart
which:
queries customEvents,
filtering to the last 24 hours (timestamp >= ago(24h)),
does a summary of the distinct count of users (dcount(user_Id)) and the total number of events (count()), grouped by the event name (by name),
then filters to the top 10 by the _count field created from the summarization (top 10 by count_)
and then renders it as a bar chart (render barchart)
there are many other examples on the analytics home page as well.
Edit to add: You can easily query any custom properties or metrics that you send as well. the customDimensions and customMeasurements fields in each event type are json typed fields, and if there's no spaces in the names, you can just use dot notation to grab values. if the field has names/special characters, use brackets and quotes:
customEvents
| where timestamp >= ago(1h)
| extend a = customDimensions.NameOfFieldWithNoSpacesOrSpecialCharacters
| extend b = customDimensions["Field with spaces"]
| extend duration = customMeasurements["Duration (ms)"]
| project a, b, duration
| limit 10
(you don't need to use extend, you can use the fields however you want this way, with extend or project or summarize or any other functions or anything else. i just used extend for the example here.)