I kinda need help extracting a value from a string and dynamically add new row
Below is the string that I have in the column DBInfo.
[{"DBName":"master","TriggerName":"ramp_sqlpreventivetrigger_sqlserver_v01","TriggerStateDB":"Enabled"},{"DBName":"master","TriggerName":"No Trigger","TriggerStateDB":"No Trigger"}]
As you can see, we have two sets of data in above string. master database is coming twice so i need rows instead of one row.
Below is what I have so far.
I am using substring and trim for now to remove "]" and "[" from the string so that I can use parse_json function. I am not having any luck so far.
SQLTriggerViolations
| where TriggerStatus != "Enabled"
| where ServerName == 'ServerName'
| project ServerName, DBInfo
| extend DBInfoModified = substring(DBInfo, 1, strlen(DBInfo))
| extend DBInfoFinal = trim("]", DBInfoModified)
| extend DBName = parse_json(DBInfoFinal).TriggerName
you can use mv-expand: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/mvexpandoperator
for example:
datatable(DBInfo:string)
[
'[{"DBName":"master","TriggerName":"ramp_sqlpreventivetrigger_sqlserver_v01","TriggerStateDB":"Enabled"},{"DBName":"master","TriggerName":"No Trigger","TriggerStateDB":"No Trigger"}]'
]
| mv-expand parse_json(DBInfo)
DBInfo
{ "DBName": "master", "TriggerName": "ramp_sqlpreventivetrigger_sqlserver_v01", "TriggerStateDB": "Enabled"}
{ "DBName": "master", "TriggerName": "No Trigger", "TriggerStateDB": "No Trigger"}
Related
How do I convert the below Splunk query to Kusto Pls
| eval result=if(Match(Status,"Success|Passed"), "succeeded","failed")
I am trying with below in Kusto but it does not work
| extend result = case(Status matches regex ("Success", "Passed"), "succeeded", "failed")
Thanks.
You could try this:
T
| extend result = case(Status contains "Success" or Status contains "Passed", "succeeded", "failed")
If "Success" and "Passed" are known to be terms in the source data, you should replace "contains" with "has"; and id they're known to be the entire string, you can use "in()" or "in~()" instead.
See: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/datatypes-string-operators
I have a custom event with a json (string) property called EventInfo. Sometimes this property will be larger than the 150 character limit set on event properties, so I have to split it into multiple properties, ie EventInfo0, EventInfo1, ect.
For example (shortened for simplicity)
EventInfo0: [{ "label" : "likeButton", "stat],
EventInfo1: [us" : "success" }]
I found out how to look at EventInfo as a json in app insights like:
customEvents
| where name == "people"
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend type=parsejson(Properties.['EventInfo'])
| mvexpand type
| project type, type.label, type.status]
Is there a way I can concatenate EventInfo0 and EventInfo1 to create the full json string, and query that like above?
According to the documentation, the 150 character limit is on the key, and not on the entire payload. So splitting as you're doing it may not actually be required.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-event-telemetry#custom-properties
that said, to answer your questions - while it's not efficient to do this at query time, the following could work:
datatable(ei0:string, ei1:string)
[
'[{ "label" : "likeButton", "stat]', '[us" : "success" }]',
'[{ "lab]', '[el" : "bar", "hello": "world" }]'
]
| project properties = parse_json(strcat(substring(ei0, 1, strlen(ei0) - 2), substring(ei1, 1, strlen(ei1) - 2)))
| project properties.label
properties_label
----------------
likeButton
bar
For my company I need to extract data from Azure Application Insights.
All the relevant data is stored in the customMeasurements. Currently, the table looks something like this:
name | itemType | customMeasurements
-----------------------------------------------------------
AppName | customEvent | {
Feature1:1,
Feature2:0,
Feature3:0
}
-----------------------------------------------------------
AppName | customEvent | {
Feature1:0,
Feature2:1,
Feature3:0
}
I'm trying to find a Kusto query which will aggregate all enabled features (which would have a value of '1'), but I'm unable to do so.
I tried several things to get this resolved like the following:
customEvents
| extend test = tostring(customMeasurements.["Feature2"])
| summarize count() by test
This actually showed me the number rows that have Feature2 set to '1' but I want to be able to extract all features that have been enabled without specifying them in the query (as they can have custom names).
Could somebody point me in the right direction please
perhaps, something like the following could give you a direction:
datatable(name:string, itemType:string, customMeasurements:dynamic)
[
'AppName', 'customEvent', dynamic({"Feature1":1,"Feature2":0,"Feature3":0}),
'AppName', 'customEvent', dynamic({"Feature1":0,"Feature2":1,"Feature3":0}),
]
| mv-apply customMeasurements on
(
extend feature = tostring(bag_keys(customMeasurements)[0])
| where customMeasurements[feature] == 1
)
| summarize enabled_features = make_set(feature) by name
I have a lot of data looking like
{"tuesday":"<30, 60>"}
{"friday":"<0, 5>"}
{"saturday":"<5, 10>"}
{"friday":"<0, 5>"}
{"saturday":"<5, 10>"}
{"sunday":"0"}
{"monday":"<0, 5>"}
All i want is the value regardless of the key.
My query:
customEvents
| where name == "eventName"
| extend d = parse_json(tostring(customDimensions.['Properties']))
| project d
| take 7
d is a dynamic object and I can do d.monday for the value, but I'd like to get the value without the key. Is this possible with Kusto?
Thanks
for the case of a single-property as you've demonstrated above, using the parse operator could work:
datatable(d:dynamic)
[
,dynamic({"tuesday":"<30, 60>"})
,dynamic({"friday":"<0, 5>"})
,dynamic({"saturday":"<5, 10>"})
,dynamic({"friday":"<0, 5>"})
,dynamic({"saturday":"<5, 10>"})
,dynamic({"sunday":"0"})
,dynamic({"monday":"<0, 5>"})
]
| parse d with * ':"' value '"' *
| project value
Notes:
In case your values are not necessarily encapsulated in double quotes (e.g. are numerics), then you should be able to specify kind=regex for the parse operator, and use a conditional expression for the existence of the double quotes.
In case you have potentially more than 1 property per property bag, using extract_all() is an option.
Relevant Docs:
https://learn.microsoft.com/en-us/azure/kusto/query/parseoperator
https://learn.microsoft.com/en-us/azure/kusto/query/extractallfunction
I have some data in Application Insights Analytics that has a dynamic object as a property of custom dimensions. For example:
| timestamp | name | customDimensions | etc |
|-------------------------|---------|----------------------------------|-----|
| 2017-09-11T19:56:20.000 | Spinner | { | ... |
MyCustomDimension: "hi"
Properties:
context: "ABC"
userMessage: "Some other"
}
Does that make sense? So a key/value pair inside of customDimensions.
I'm trying to bring up the context property to be a proper column in the results. So expected would be :
| timestamp | name | customDimensions | context| etc |
|-------------------------|---------|----------------------------------|--------|-----|
| 2017-09-11T19:56:20.000 | Spinner | { | ABC | ...
MyCustomDimension: "hi"
Properties:
context: "ABC"
userMessage: "Some other"
}
I've tried this:
customEvents | where name == "Spinner" | extend Context = customDimensions.Properties["context"]
and this:
customEvents | where name == "Spinner" | extend Context = customDimensions.Properties.context
but neither seem to work. They give me a column at the end named "Context" but the column is empty - no values.
Any ideas?
EDIT:
Added a picture for clarifying the format of the data:
edited to working answer:
customEvents
| where name == "Spinner"
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend Context = Properties.context
you need an extra tostring and todynamic in here to get what you expect (and what i expected!)
the explanation i was given:
Dynamic field "promises" you the upper/outer level of key / value access (this is how you access customDimensions.Properties).
Accessing internal structure of that json depends on the exact format of customDimensions.Properties content. It doesn’t have to be json by itself. Even if it looks like a well structured json, it still may be just a string that is not exactly well formatted json.
So basically, it by default won't attempt to parse strings inside of a dynamic/json block because they don't want to spend a lot of time possibly trying and failing to convert nested content to json infinitely.
I still think that extra tostring shouldn't be required inside there, since todynamic should already be allowing both string and dynamic in validly, so i'm checking to see if the team that owns the query stuff can make that step better.
Thanks sooo much.. just to expand on the answer from John. We needed to graph duration of end-points using custom events. This query made it so we could specify the duration as our Y-axis in the chart:
customEvents
| extend Properties = todynamic(tostring(customDimensions.Properties))
| extend duration = todouble(todecimal(Properties.duration))
| project timestamp, name, duration