I have following query:
traces
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__recordId) or isnotempty(customDimensions.prop__Entity)
| project operation_Id, Entity = customDimensions.prop__Entity, recordName = customDimensions.prop__recordName, recordId = customDimensions.prop__recordId
I get results like these:
I want to merge rows by operation_id, and get results like these:
Please try use join operator, like below:
traces
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__recordId)
| project operation_Id, customDimensions.prop__recordId
| join kind = inner(
traces
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__Entity)
| project operation_Id,customDimensions.prop__Entity,customDimensions.prop__recordName
) on operation_Id
| project-away operation_Id1 //remove the redundant column,note that it's operation_Id1
| project operation_Id, Entity = customDimensions.prop__Entity, recordName = customDimensions.prop__recordName, recordId = customDimensions.prop__recordId
I did not has the same data, but make some similar data, works fine at my side.
Before merge:
After merge:(and note that use project-away to remove the redundant column which is used as joined key, and it always has number suffix 1 by default)
Final query is:
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__recordId)
| project operation_Id, customDimensions.prop__recordId
| join kind = inner(
traces
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__Entity)
| project operation_Id,customDimensions.prop__Entity
) on operation_Id
| join kind = inner(
traces
| where customDimensions.Category == "Function"
| where isnotempty(customDimensions.prop__recordName)
| project operation_Id,customDimensions.prop__recordName
) on operation_Id
| project operation_Id, Entity = customDimensions_prop__Entity, recordName = customDimensions_prop__recordName, recordId = customDimensions_prop__recordId
Related
We're running into Kusto has_any limit of 10K.
Sample code
// Query: Get failed operations for migrated apps
let migrationsTimeDiff = 15d;
let operationsDiffTime = 24h + 1m;
let migratedApps = FirstTable
| where TimeStamp >= ago(migrationsTimeDiff)
| where MetricName == "JobSucceeded"
| project
MigrationTime = PreciseTimeStamp,
appName = tostring(parse_json(Annotations).AppName)
| project appName;
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| where Url has_any (appName)
| where Result == "Fail"
Is there a way to restructure the query via joins?
Alternatively is it possible to loop in batches of 10k?
Thanks for reading!
If Url is an exact match to appName, then you should use:
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| where Url in (appName) // 'in' instead of 'has_any'
| where Result == "Fail"
Otherwise, you'll need to extract the application name from the Url using extend, and then use in like I suggested above, so your query will look like this:
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| extend ExtractedAppNameFromUrl = ...
| where ExtractedAppNameFromUrl in (appName) // 'in' instead of 'has_any'
| where Result == "Fail"
I am trying to parse the below data in Kusto. Need help.
[[ObjectCount][LinkCount][DurationInUs]]
[ChangeEnumeration][[88][9][346194]]
[ModifyTargetInLive][[3][6][595903]]
Need generic implementation without any hardcoding.
ideally - you'd be able to change the component that produces source data in that format to use a standard format (e.g. CSV, Json, etc.) instead.
The following could work, but you should consider it very inefficient
let T = datatable(s:string)
[
'[[ObjectCount][LinkCount][DurationInUs]]',
'[ChangeEnumeration][[88][9][346194]]',
'[ModifyTargetInLive][[3][6][595903]]',
];
let keys = toscalar(
T
| where s startswith "[["
| take 1
| project extract_all(#'\[([^\[\]]+)\]', s)
);
T
| where s !startswith "[["
| project values = extract_all(#'\[([^\[\]]+)\]', s)
| mv-apply with_itemindex = i keys on (
extend Category = tostring(values[0]), p = pack(tostring(keys[i]), values[i + 1])
| summarize b = make_bag(p) by Category
)
| project-away values
| evaluate bag_unpack(b)
--->
| Category | ObjectCount | LinkCount | DurationInUs |
|--------------------|-------------|-----------|--------------|
| ChangeEnumeration | 88 | 9 | 346194 |
| ModifyTargetInLive | 3 | 6 | 595903 |
I need the messages in Azure AppInsights grouped by the existence of particular substrings in the messages and the counts of these messages.
At the end, here is what the grouping would look like
messages count
-------- -------
foomessages <say, 300>
barmessages <say, 450>
:
:
where
foomessages = All messages containing the substring "foo" etc.
How can I construct a query for this ?
datatable(log: string) [
"hello world",
"this is a test",
"this is a world test",
"another test"
]
| summarize
LogsWithWorld = countif(log has "world"),
LogsWithTest = countif(log has "test")
| project Result = pack_all()
| mv-expand Result
| extend Message = tostring(bag_keys(Result)[0])
| extend Count = tolong(Result[Message])
| project Message, Count
The produced result is:
| Message | Count |
|---------------|-------|
| LogsWithWorld | 2 |
| LogsWithTest | 3 |
|---------------|-------|
I'm looking to get the count of query param usage from the query string from page views stored in app insights using KQL. My query currently looks like:
pageViews
| project parsed=parseurl(url)
| project keys=bag_keys(parsed["Query Parameters"])
and the results look like
with each row looking like
I'm looking to get the count of each value in the list when it is contained in the url in order to anwser the question "How many times does page appear in the querystring". So the results might look like:
Page | From | ...
1000 | 67 | ...
Thanks in advance
you could try something along the following lines:
datatable(url:string)
[
"https://a.b.c/d?p1=hello&p2=world",
"https://a.b.c/d?p2=world&p3=foo&p4=bar"
]
| project parsed = parseurl(url)
| project keys = bag_keys(parsed["Query Parameters"])
| mv-expand key = ['keys'] to typeof(string)
| summarize count() by key
which returns:
| key | count_ |
|-----|--------|
| p1 | 1 |
| p2 | 2 |
| p3 | 1 |
| p4 | 1 |
Table is the following:
CREATE TABLE UserLog(uid TEXT, clicks INT, lang TEXT)
Where uid field should be unique.
Here is some sample data:
| uid | clicks | lang |
----------------------------------------
| "898187354" | 4 | "ru" |
| "898187354" | 4 | "ru" |
| "123456789" | 1 | <null> |
| "123456789" | 10 | "en" |
| "140922382" | 13 | <null> |
As you can see, I have multiple rows with where the uid field is now duplicated. I would like for those rows to be merged in a following way:
clicks fields are added, and lang fields are updated if their previous value was null.
For the data shown above, it would look something like this:
| uid | clicks | lang |
---------------------------------------
| "898187354" | 8 | "ru" |
| "123456789" | 11 | "en" |
| "140922382" | 13 | <null> |
It seems that I can find many ways to simply delete duplicate data, which I do not necessarily want to do. I'm unsure how I can introduce logic in SQL statements that does this.
First update:
update userlog
set
clicks = (select sum(u.clicks) from userlog u where u.uid = userlog.uid),
lang = (select max(u.lang) from userlog u where u.uid = userlog.uid)
where not exists (
select 1 from userlog u
where u.uid = userlog.uid and u.rowid < userlog.rowid
);
and then delete the duplicate rows that are not needed:
delete from userlog
where exists (
select 1 from userlog u
where u.uid = userlog.uid and u.rowid < userlog.rowid
);