I have some data in Application Insights Analytics that has a dynamic object as a property of custom dimensions. For example:
| timestamp | name | customDimensions | etc |
|-------------------------|---------|----------------------------------|-----|
| 2017-09-11T19:56:20.000 | Spinner | { | ... |
MyCustomDimension: "hi"
Properties:
context: "ABC"
userMessage: "Some other"
}
Does that make sense? So a key/value pair inside of customDimensions.
I'm trying to bring up the context property to be a proper column in the results. So expected would be :
| timestamp | name | customDimensions | context| etc |
|-------------------------|---------|----------------------------------|--------|-----|
| 2017-09-11T19:56:20.000 | Spinner | { | ABC | ...
MyCustomDimension: "hi"
Properties:
context: "ABC"
userMessage: "Some other"
}
I've tried this:
customEvents | where name == "Spinner" | extend Context = customDimensions.Properties["context"]
and this:
customEvents | where name == "Spinner" | extend Context = customDimensions.Properties.context
but neither seem to work. They give me a column at the end named "Context" but the column is empty - no values.
Any ideas?
EDIT:
Added a picture for clarifying the format of the data:
edited to working answer:
customEvents
| where name == "Spinner"
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend Context = Properties.context
you need an extra tostring and todynamic in here to get what you expect (and what i expected!)
the explanation i was given:
Dynamic field "promises" you the upper/outer level of key / value access (this is how you access customDimensions.Properties).
Accessing internal structure of that json depends on the exact format of customDimensions.Properties content. It doesn’t have to be json by itself. Even if it looks like a well structured json, it still may be just a string that is not exactly well formatted json.
So basically, it by default won't attempt to parse strings inside of a dynamic/json block because they don't want to spend a lot of time possibly trying and failing to convert nested content to json infinitely.
I still think that extra tostring shouldn't be required inside there, since todynamic should already be allowing both string and dynamic in validly, so i'm checking to see if the team that owns the query stuff can make that step better.
Thanks sooo much.. just to expand on the answer from John. We needed to graph duration of end-points using custom events. This query made it so we could specify the duration as our Y-axis in the chart:
customEvents
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend duration = todouble(todecimal(Properties.duration))
| project timestamp, name, duration
Related
I have a custom event with a json (string) property called EventInfo. Sometimes this property will be larger than the 150 character limit set on event properties, so I have to split it into multiple properties, ie EventInfo0, EventInfo1, ect.
For example (shortened for simplicity)
EventInfo0: [{ "label" : "likeButton", "stat],
EventInfo1: [us" : "success" }]
I found out how to look at EventInfo as a json in app insights like:
customEvents
| where name == "people"
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend type=parsejson(Properties.['EventInfo'])
| mvexpand type
| project type, type.label, type.status]
Is there a way I can concatenate EventInfo0 and EventInfo1 to create the full json string, and query that like above?
According to the documentation, the 150 character limit is on the key, and not on the entire payload. So splitting as you're doing it may not actually be required.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-event-telemetry#custom-properties
that said, to answer your questions - while it's not efficient to do this at query time, the following could work:
datatable(ei0:string, ei1:string)
[
'[{ "label" : "likeButton", "stat]', '[us" : "success" }]',
'[{ "lab]', '[el" : "bar", "hello": "world" }]'
]
| project properties = parse_json(strcat(substring(ei0, 1, strlen(ei0) - 2), substring(ei1, 1, strlen(ei1) - 2)))
| project properties.label
properties_label
----------------
likeButton
bar
For my company I need to extract data from Azure Application Insights.
All the relevant data is stored in the customMeasurements. Currently, the table looks something like this:
name | itemType | customMeasurements
-----------------------------------------------------------
AppName | customEvent | {
Feature1:1,
Feature2:0,
Feature3:0
}
-----------------------------------------------------------
AppName | customEvent | {
Feature1:0,
Feature2:1,
Feature3:0
}
I'm trying to find a Kusto query which will aggregate all enabled features (which would have a value of '1'), but I'm unable to do so.
I tried several things to get this resolved like the following:
customEvents
| extend test = tostring(customMeasurements.["Feature2"])
| summarize count() by test
This actually showed me the number rows that have Feature2 set to '1' but I want to be able to extract all features that have been enabled without specifying them in the query (as they can have custom names).
Could somebody point me in the right direction please
perhaps, something like the following could give you a direction:
datatable(name:string, itemType:string, customMeasurements:dynamic)
[
'AppName', 'customEvent', dynamic({"Feature1":1,"Feature2":0,"Feature3":0}),
'AppName', 'customEvent', dynamic({"Feature1":0,"Feature2":1,"Feature3":0}),
]
| mv-apply customMeasurements on
(
extend feature = tostring(bag_keys(customMeasurements)[0])
| where customMeasurements[feature] == 1
)
| summarize enabled_features = make_set(feature) by name
Given the following Kusto query:
range t from bin(now(), 1h)-23h to bin(now(), 1h) step 1h
| summarize t=make_list(t)
| project id='TS', val=dynamic([0,0,0,0,0,0,0,0,0,10,20,40,100,40,20,10,0,0,0,0,0,0,0,0]), t
| extend 5h_MovingAvg=series_fir(val, dynamic([1,1,1,1,1])),
5h_MovingAvg_centered=series_fir(val, dynamic([1,1,1,1,1]), true, true)
| render timechart
I am unable to get application insights to actually draw the moving average lines shown in this document
I have also tried applying the article to one of our actual applications and have not had any luck either. There are no errors or anything that would give a clue as to why the moving averages are not being drawn. I'm assuming there is a setting somewhere that most probably has to be set. Here is my custom query:
let timeGrain=1d;
let ago = ago(7d);
let mAvgParm = repeat(1, 5);
let dataset=requests
// additional filters can be applied here
| where timestamp >= ago and cloud_RoleName == "recalculateordercombination" and resultCode == 500
| where client_Type != "Browser" ;
// calculate failed request count for all requests
dataset
| make-series dailyFailure=sum(itemCount) default=0 on timestamp in range(ago, now(), timeGrain) by resultCode
// render result in a chart
| extend SMA = series_fir(dailyFailure, mAvgParm)
| render timechart
What are these queries missing in order to draw the moving average lines using series_fir?
ref articles used in my research
https://marckean.com/2019/03/25/log-analytics-advanced-queries/
https://learn.microsoft.com/en-us/azure/kusto/query/series-firfunction
https://learn.microsoft.com/en-us/azure/kusto/query/make-seriesoperator
The web clients for both services are different, and that's also true for their rendering logic.
In Azure Data Explorer (Kusto), you can just use render timechart on time-series data (which is typed as dynamic).
In other cases, you may need to first mv-expand the series (link to doc), before rendering it.
Here's an example which matches the first query in your question:
range t from bin(now(), 1h)-23h to bin(now(), 1h) step 1h
| summarize t=make_list(t)
| project id='TS', val=dynamic([0,0,0,0,0,0,0,0,0,10,20,40,100,40,20,10,0,0,0,0,0,0,0,0]), t
| extend 5h_MovingAvg=series_fir(val, dynamic([1,1,1,1,1])),
5h_MovingAvg_centered=series_fir(val, dynamic([1,1,1,1,1]), true, true)
| mv-expand val to typeof(long), t to typeof(datetime), 5h_MovingAvg to typeof(long), 5h_MovingAvg_centered to typeof(long)
| project t, 5h_MovingAvg, 5h_MovingAvg_centered, val
| render timechart
I have a lot of data looking like
{"tuesday":"<30, 60>"}
{"friday":"<0, 5>"}
{"saturday":"<5, 10>"}
{"friday":"<0, 5>"}
{"saturday":"<5, 10>"}
{"sunday":"0"}
{"monday":"<0, 5>"}
All i want is the value regardless of the key.
My query:
customEvents
| where name == "eventName"
| extend d = parse_json(tostring(customDimensions.['Properties']))
| project d
| take 7
d is a dynamic object and I can do d.monday for the value, but I'd like to get the value without the key. Is this possible with Kusto?
Thanks
for the case of a single-property as you've demonstrated above, using the parse operator could work:
datatable(d:dynamic)
[
,dynamic({"tuesday":"<30, 60>"})
,dynamic({"friday":"<0, 5>"})
,dynamic({"saturday":"<5, 10>"})
,dynamic({"friday":"<0, 5>"})
,dynamic({"saturday":"<5, 10>"})
,dynamic({"sunday":"0"})
,dynamic({"monday":"<0, 5>"})
]
| parse d with * ':"' value '"' *
| project value
Notes:
In case your values are not necessarily encapsulated in double quotes (e.g. are numerics), then you should be able to specify kind=regex for the parse operator, and use a conditional expression for the existence of the double quotes.
In case you have potentially more than 1 property per property bag, using extract_all() is an option.
Relevant Docs:
https://learn.microsoft.com/en-us/azure/kusto/query/parseoperator
https://learn.microsoft.com/en-us/azure/kusto/query/extractallfunction
I would to make the field descriptions and label texts in my pages multi-lingual. Originally they are in English and I could let the user translate them through Google Translate. In order to avoid translation errors I would like to implement a translation data model that contains
FieldDisplayName / LabelText
FieldDisplayName_DE
FieldDisplayName_FR
FieldDisplayName_IT
etc.
All the pages contain a page header fragment that contains a menu button, searchbox etc. like in the Starter App template. I am planning on integrating a dropdown widget in the page header that allows to choose between the languages (DE,EN,FR,IT,...). Is it possible to bind the display name to the user's selection? How would I have to implement that?
The easiest way (to implement/use/maintain) that would provide highest possible translation quality will be introducing Translation data model with the following structure:
+----+--------+------------+------------+------------+-----+
| Id | Locale | FirstName | LastName | Age | ... |
+----+--------+------------+------------+------------+-----+
| 1 | EN | First name | Last name | Age | ... |
+----+--------+------------+------------+------------+-----+
| 2 | RU | Имя | Фамилия | Возраст | ... |
+----+--------+------------+------------+------------+-----+
| 3 | DE | Voornaam | Achternaam | Leeftijd | ... |
+----+--------+------------+------------+------------+-----+
| 4 | ... | ... | .... | ... | ... |
+----+--------+------------+------------+------------+-----+
In this model every column represents unique label within your app and every row represents labels's translations for supported languages. This model can be easily used in label bindings:
#datasources.UserTranslations.item.FieldNameToTranslate
Maintaining these translation will be easy as well, just drag and drop editable table on UI.
Here is a query script for the UserTranslations datasource:
// Assuming that you already have robust user settings implementation.
var userSettings = getUserSettings_();
var query = app.models.Translation.newQuery();
query.filters.Locale._equals = userSettings.Locale;
return query.run();
Radically different implementation will be
Introducing Calculated Model with the same set of fields as in the previous approach
Using Model Metadata API to extract display names from the model's fields
Translate fields using Translate API
Populate calculated model record with translated values
Here is super high level server pseudo script for that flow:
var userLocale = getUserLocaleFromUserSettings();
var fieldsDisplayNames = getFieldsDisplayNames(app.models.Translation);
var translations = translate(fieldsDisplayNames, 'en', userLocale);
var record = app.models.Translation.newRecord();
mapRecordFieldsToTranslations(record, translations);
return [record];
After some trials a translation model turned out to be too laggy for my demands. Therefore I have hardcoded the binding expression into the labels I want to translate. The binding expression looks a little bit like this:
(#pages.UserSettings.LanguageDropdown.value == 'EN') ? 'Contact' : 'Kontakt'