We have a custom event put in place on page which tracks the link clicks on given page to app insights. And with the REST API we would like to get the frequently accessed links from app insights.
How can we build the Query to get this analytics data, any sample on reading custom events available?
Thanks
if you open the Application Insights Analytics website for any resource, there's some "Common Queries" examples right on the front page. one of them is called "Usage" and if you click it it will show you this one:
//What are the top 10 custom events of your application in the past 24 hours?
customEvents
| where timestamp >= ago(24h)
| summarize dcount(user_Id), count() by name
| top 10 by count_
| render barchart
which:
queries customEvents,
filtering to the last 24 hours (timestamp >= ago(24h)),
does a summary of the distinct count of users (dcount(user_Id)) and the total number of events (count()), grouped by the event name (by name),
then filters to the top 10 by the _count field created from the summarization (top 10 by count_)
and then renders it as a bar chart (render barchart)
there are many other examples on the analytics home page as well.
Edit to add: You can easily query any custom properties or metrics that you send as well. the customDimensions and customMeasurements fields in each event type are json typed fields, and if there's no spaces in the names, you can just use dot notation to grab values. if the field has names/special characters, use brackets and quotes:
customEvents
| where timestamp >= ago(1h)
| extend a = customDimensions.NameOfFieldWithNoSpacesOrSpecialCharacters
| extend b = customDimensions["Field with spaces"]
| extend duration = customMeasurements["Duration (ms)"]
| project a, b, duration
| limit 10
(you don't need to use extend, you can use the fields however you want this way, with extend or project or summarize or any other functions or anything else. i just used extend for the example here.)
Related
I am putting together a dashboard for our production services and I am trying to create a unified chart of response performance across multiple services.
Now, I can get the the query to run but it created a STACKED column chart but I cannot seem to get it to UNSTACK unless I manually go to the chart properties and change it, but this is not a persistent change as it will reset when you reload / refresh the page.
Here's my query
// Create a combined dataset
let mainTable = union pageViews, customEvents, requests
| where isempty(operation_SyntheticSource)
| extend name =replace("\n", "", name)
| extend name =replace("\r", "", name);
// Generate chart data
mainTable
| where duration > 1000 // Not interested in any responses under 1 second
| parse (duration / 1000) // Take the whole number of seconds
with P1: int
"."
notInterested: string
| extend peformanceBucket = iff(P1 > 4, ">4", tostring(P1))
| project name, duration, peformanceBucket
| summarize Count = count() by peformanceBucket, name
| order by peformanceBucket
| render columnchart
And here's the output
But this is what I want (by default)
I can get the the query to run but it created a STACKED column chart but I cannot seem to get it to UNSTACK unless I manually go to the chart properties and change it, but this is not a persistent change as it will reset when you reload / refresh the page.
If you want to show the Unstacked Chart Type use the Kind property has Unstacked.
Workaround follows:
The query I have used in my Application Insights
exceptions
| summarize count = sum(itemCount) by bin(timestamp, 3h), problemId
| order by timestamp asc, problemId
| render columnchart kind=unstacked
Result
After Adding in Dashboard
We have an web application. Part of web the application is profiles. Profiles have following urls:
/Profile/1
/Profile/2
/Profile/3
We also have Adds. Add's urls are as following:
/Add/1
/Add/2
/Add/3
I need a view of top 20 most viewed profiles and top 20 most viewed add's and how many views they have. It must also be possible to extract those two lists with c#. Can this be achieved with log analytics? Do you need to extract everything your self from application insights and do analytics your self, or how would you achieve this?
I made a little POC. I have added following to AI. The https://gac/url/1 has thousands of id's. Maybe there us 244 views on https://gac/url/1 and 1128 views on https://gac/url/2and so forth.
I noticed that Id is not added to the log. See code below:
public void WritePageView(string name, string id, Uri url)
{
var pageView = new PageViewTelemetry
{
Name = name,
Id = id,
Url = url
};
pageView.Properties.Add("Id", "1");
telemetry.TrackPageView(pageView);
}
What I need is a top 20 (top x) of which https://gac/url is most visited. Something like:
https://gac/url1: 3434
https://gac/url1: 2432
https://gac/url1: 1298
https://gac/url1: 8211
..
If showed in a graph to would be really great.
So it would be something like group by name and Id. But as Id is only pressent in the url how do I extract that?
It's easy to use analytics query for application insights.
Nav to your azure portal -> application insights -> Logs(Analytics), the related info should be in the pageViews table. When you write your query, you can set custom "Time Range" via UI or use where clause.
You can get all the pageViews info by just write "pageViews", screenshot as below:
And you can use Summarize operator and count operator to get the total views count for each page. Sample query code like below:
pageViews
| summarize totalCount = count(url) by url
| order by totalCount desc
The result as below:
If you want to only get the top 2 most views page, use the following code:
pageViews
| summarize totalCount = count(url) by url
| order by totalCount desc
| top 2 by totalCount
in my BigQuery project I store event data integrated from Firebase. The granularity and dimension is such that trying to present raw data in Data Studio quickly makes the report become VERY slow (1-2 min per page/interaction).
I then started to think how I could create pre-aggregated tables in BigQuery to speed everything up, but quickly realised COUNT DISTINCT metrics would be a problem with this approach.
Let me explain:
SELECT user, date
FROM UNNEST([
STRUCT("Adam" AS user, "20190923" AS date),
("Bob", "20190923"),
("Carl", "20190923"),
("Adam", "20190924"),
("Bob", "20190924"),
("Adam", "20190925"),
("Carl", "20190925"),
("Bob", "20190926")
]) AS website_visits;
+------+----------+
| User | Date |
+------+----------+
| Adam | 20190923 |
| Bob | 20190923 |
| Carl | 20190923 |
| Adam | 20190924 |
| Bob | 20190924 |
| Adam | 20190925 |
| Carl | 20190925 |
| Bob | 20190926 |
+------+----------+
The above is a table of website visits.
Clearly, creating a pre-aggregated table like
SELECT date, COUNT(DISTINCT user) FROM website_visits GROUP BY date
has the limitation that the count cannot be aggregated further (or even less, dinamically) to get a total, as doing a SUM would return 8 unique users which is not correct, there are only 3 unique users.
In BigQuery, this is fixed by using HLL_COUNT, which despite the approximation works ok for me.
Now to the big question:
How to do the same so that the result is displayable in Data Studio????
HLL_COUNT.EXTRACT is not available as function in there, and in the reporting I always have to keep in mind that the date range is set by the user however (s)he likes so it's not possible to store a pre-aggregated result for ALL cases...
EDIT 1: APPROX_COUNT_DISTINCT
As per answer from Bobbylank, I tried to use APPROX_COUNT_DISTINCT.
However I found that this just seems to move the issue down the line. My fault for not explaining what's over there.
Despite being performances acceptable it does not seem possible to me to blend a data source with this calculated metric.
Example: After displaying the amount of unique users in the selected period (which now works), I'm also trying to display Average Revenue Per User (ARPU) in Data Studio like Firebase does.
To do this, I have to SUM(REVENUE) / APPROX_COUNT_DISTINCT(USER)
Clearly, REVENUE works ok with pre-aggregation and is available in the raw data. I tried then to blend the raw data with a table containing just user visits. However APPROX_COUNT_DISTINCT can't be used in the blended data definition as calculated metrics are not allowed.
Even trying to use the USER field as a metric with Count Distinct aggregation, despite returning the correct figures when showing revenue and user count separately, when I try to divide them the problem becomes aggregation (apply SUM or AVG to the field and basically the result will be AVG(REVENUE/USERS) for each day).
I also then tried to store REVENUE directly in the visits table, but was reminded by Data Studio that I can't create calculated metrics that I can't mix dimensions and metrics in a calculated field.
APPROX_COUNT_DISTINCT might be more performance friendly for you?
https://support.google.com/datastudio/answer/9189108?hl=en
Otherwise the only way I can think would be to pre-calculate several metrics (e.g. unique users on that day, 7-day cumulative, 14-day, etc.) as your customer require for each single day.
Or you could provide a 2 page report with both of these methods with the caveat that the first can be used over a time period but will be much slower?
I checked that when i try to query on events data the total record decreased automatically. I want to know that when data is deleted from applicationinsights. My query is just simple count query. I also noticed that some events timestamp also updated. I searched for blog etc but nothing found
customEvents | count
Unless you explicitly set a time range in the query directly, like
customEvents | where timestamp >= ago(7d) | count
then the analytics website is going to automatically apply a last 24 hours time restriction automatically. (look in the upper right corner, there is a dropdown that will either say "set in query" or "last 24 hours" or other choices)
I would like to get google analytics on a small set of pages, of the thousands of pages, tracked by the google analytics. Each of these page has a unique id and I was hoping to use these to identify the subset of pages. Following are examples of such ids.
uda502aaf-3a8f-4aa5-9f49-00d2ac4c6545
ud5655462-c5f5-46c4-b807-d1a8527786d7
uceb1a469-0af9-499f-9608-42242a9f0c63
uc4b4d9e3-a558-4282-a009-e984e81364eb
ubaf6e057-4cad-4b9b-a625-7b2ea11fa8c5
ub1c4cb68-bc41-476b-9e31-9b9800113236
uadb6c44d-792b-48ca-969d-ff1b638dd223
uaa17490f-11ff-4947-9841-9f744963fbc2
u9a60d135-64f7-4890-963f-8093b8cf1eaa
I tried to create a filter on pages, using "contains", with the ids separated by | as shown below
A)
uf07f4d74-2239-4b4b-851f-d40e51ad5f9a | 125bd8a07b4c32917dae805c23ead087 | c65da9b1fcb588e64cb126fda78afee9
AND
B)
(uf07f4d74-2239-4b4b-851f-d40e51ad5f9a) | (125bd8a07b4c32917dae805c23ead087) | (c65da9b1fcb588e64cb126fda78afee9)
and everything was "0". I know users have visited these pages. How can I create a correct filter for such IDs?
Thx in advance
You can't use a "contains" filter type if you have multiple IDs that you want to sort through. In your example A, you are using something like a Regex format, so you should select "Matching RegExp" as the filter type.