I am trying to identify shared mailboxes that aren't in use. Checked "Search-MailboxAuditLog" already and some mailboxes do not return any results even tho auditing enabled, but can see activity in Azure sentinel.
Is there a way to improve below Kusto code? (During testing tried mailboxes with activities but sometimes do not get any results from the query)
With Kusto, Is there a way to loop through "mbs" like powershell "foreach ( $item in $mbs)"?
Thanks,
let mbs = datatable (name: string)
[
"xxx1#something.com",
"xxx2#something.com",
"xxx3#something.com",
];
OfficeActivity
| where OfficeWorkload == "Exchange" and TimeGenerated > ago(30d)
| where MailboxOwnerUPN in~ (mbs)
| distinct MailboxOwnerUPN
Update : Need help with the query
Input would be list of shared mailbox UPNs
Output would be list of shared mailboxes with any activity, example MBs with any action in “Operation" filed
"in" doesn't work on datatables (tabular inputs) like that; it is not a "filter", it is an "operator". The "where" is effectively the "foreach" you are referring to.
Given the sample input, the query could probably be written as:
OfficeActivity //tabular input with many records
| TimeGenerated > ago(30d) //Filter records to window of interest first
| where OfficeWorkload == "Exchange" //foreach row
| where MailboxOwnerUPN in~ ( //foreach row
"xxx1#something.com","xxx2#something.com","xxx3#something.com"
)
| distinct MailboxOwnerUPN
You can see it in the docs at https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/inoperator#arguments where "col" is the "column to filter"
Related
I have a RequestTelemetry operation. It has two events within it. If I query for only the operation, I get the request, one row for each TrackEvent item, when I expand the > for the operation in the Results pane I have two rows, one for each event, but the same operation.
requests
| where operation_Name == 'my_operation'
Will give me a row for each event, as these events have the same operation id.
Using the operation_id from this query, I can query customEvents,
customEvents
| where operation_id == 'myid'
I get the events, two in this case, two rows, as expected just like my requests query.
I have tried a join with,
requests
| join kind=inner customEvents on operation_Name
| where operation_Name == 'my_operation'
and I get the rows, each having a different customEvent (the two I did) like above. It is the same result if I add another filter | operation_id == 'myid'.
How can I have one Kusto query that has a single row for the operation and when I expand it in the results, I can see all the events associated with it? Not multiple rows, one for each event in an operation, but a single row for operation and have > that I can expand underneath to show all the events? In addition, the events would be expandable like they are if I had queried by themselves.
I also looked at mv-expand, mv_bag, and Kusto Query Ingestion with commands, but I am not sure if I'm on the right track.
Edit: It would look like this in the Results pane as a single row,
SendOperation
myevent1
myevent2
From (example snippet, not guaranteed to compile):
Using(IOperationHolder<DependencyTelemetry> op = new TelemetryClient().StartOperation<DependencyTelemetry>("SendOperation"))
{
TelemetryClient telemetry = new TelemetryClient();
telemetry.TrackEvent
(
"myevent1",
new Dictionary<string, string>()
{
{"some_var_value", JsonConvert.SerializeObject(myobjFromSomewhere)},
{"some_other_var", theVariableINeedToTrack}
}
telemetry.TrackEvent
(
"myevent2",
new Dictionary<string, string>()
{
{"useful_metric", usefulVariableValue},
}
}
}```
Edit:,
timestamp:
id:
name: my operation
event: event1
event: event2
or,
timestamp:
id:
name: my operation
CustomDimensions: {event1:{...}, event2:{...}}
I'm writing a function which takes in raw data table (contains multijson telemetry data) and reformat it to a multiple cols. I use .set MyTable <| myfunction|limit 0 to create my target table based off of the function and use update policy to alert my target table.
Here is the code :
.set-or-append MyTargetTable <|
myfunction
| limit 0
.alter table MyTargetTable policy update
#'[{ "IsEnabled": true, "Source": "raw", "Query": "myfunction()", "IsTransactional": false, "PropagateIngestionProperties": false}]'
But I'm getting ingestion failures: Here is the ingestion failure message :
Failed to invoke update policy. Target Table = 'MyTargetTable', Query = '
let raw = __table("raw", 'All', 'AllButRowStore')
| where extent_id() in (guid(659e3b3c-6859-426d-9c37-003623834455));
myfunction()': Query schema does not match table schema
I double check the query schema and target table; they are the same . I'm not sure what this error means.
Also, I ran count on both the raw and mytarget tables; there are relatively large discrepancies (400 rows for My target and 2000 rows in raw table).
Any advise will be appreciated.
Generally speaking - to find the root of the mismatch between schemas, you can run something along the following lines, and filter for differences:
myfunction
| getschema
| join kind=leftouter (
table('MyTargetTable')
| getschema
) on ColumnOrdinal, ColumnType
In addition - you should make sure the output schema of the function you use in your update policy is 'stable', i.e. isn't affected by the input data
The output schema of some query plugins such as pivot() and bag_unpack() depends on the input data, and therefore it isn't recommended to use those in update policies.
I'm pretty new to KQL and I'm having a difficult time with it (I don't have a background in stats, and I'm not very good at SQL either). I have telemetry data coming in from Microsoft AppCenter that I want to parse out into some charts but I'm trying to first figure out how to split a concatenated string that is essentially a dictionary that has two possible values: true and false. I want to count the number of each, so every key would have 2 values (true/false) which would also each have a numerical count value.
The input string I'm trying to get this data from is of the format Remove Splash/Main Menu Branding=True;Disable Aim Assist=False - unique items are split by ; and each pair is split by =. I am trying to figure out which options my users are using this way. The example string here would be split into:
Remove Splash/Main Menu Branding = True (count 1)
Disable Aim Assist = False (count 1).
If a new item came in that was Remove Splash/Main Menu Branding=True;Disable Aim Assist=True the summarized data would be
Remove Splash/Main Menu Branding = True (count 2)
Disable Aim Assist = False (count 1).
Disable Aim Assist = True (count 1).
So far I've got a query that selects a single item, but I don't know how to count this across multiple rows:
customEvents
| where timestamp > ago(7d)
| where name == "Installed a mod"
| extend Properties = todynamic(tostring(customDimensions.Properties))
| where isnotnull(Properties.["Alternate Options Selected"])
| extend OptionsStr = Properties.["Alternate Options Selected"] //The example string in above
| extend ModName = Properties.["Mod name"]
| where ModName startswith "SP Controller Support" //want to filter only to one mod's options
| extend optionsSplit = split(OptionsStr, ";")
| summarize any(optionsSplit)
I'm not sure how to make counts of it in a dictionary though. If anyone has any suggestions or tips or examples on something like this, I would really appreciate it, thanks.
Here you go:
let MyTable = datatable(Flags:string) [
"Remove Splash/Main Menu Branding=True;Disable Aim Assist=False",
"Remove Splash/Main Menu Branding=True;Disable Aim Assist=True"
];
MyTable
| extend Flags = split(Flags, ";")
| mv-expand Flag = Flags to typeof(string)
| summarize Count = count() by Flag
The output of this is:
| Flag | Count |
|---------------------------------------|-------|
| Remove Splash/Main Menu Branding=True | 2 |
| Disable Aim Assist=False | 1 |
| Disable Aim Assist=True | 1 |
Explanation:
First you split every input string (that contains multiple flags) into substrings, so that each will only have a single flag - you achieve this by using split.
Now your new Flags column has a list of strings (each one containing a single flag), and you want to create a record with every string, so you use the mv-expand operator
Lastly, you want to count how many times every key=value pair appears, and you do it with summarize count() by Flag
In case you want to see one record (in the output) per Key, then you can use the following query instead:
let MyTable = datatable(Flags:string) [
"Remove Splash/Main Menu Branding=True;Disable Aim Assist=False",
"Remove Splash/Main Menu Branding=True;Disable Aim Assist=True"
];
MyTable
| extend Flags = split(Flags, ";")
| mv-expand Flag = Flags to typeof(string)
| parse Flag with Key "=" Value
| project Key, Value
| evaluate pivot(Value, count(Value))
Its output is:
| Key | False | True |
|----------------------------------|-------|------|
| Remove Splash/Main Menu Branding | 0 | 2 |
| Disable Aim Assist | 1 | 1 |
You wrote that you're new to KQL, so you might find the following free Pluralsight courses interesting:
How to start with Microsoft Azure Data Explorer
Basic KQL
Azure Data Explorer – Advanced KQL
P.S. In the future please provide sample input in datatable format (if you're using Kusto Explorer, just select the relevant query results, right-click on the selection, and click Copy as datatable() literal), and also the expected output in a table format, so that it will be easier to understand what you want to achieve.
I have a need to be able to query Azure Data Explorer (ADX) tables dynamically, that is, using application-specific metadata that is also stored in ADX.
If this is even possible, the way to do it seems to be via the table() function. In other words, it feels like I should be able to simply write:
let table_name = <non-trivial ADX query that returns the name of a table as a string>;
table(table_name) | limit 10
But this query fails since I am trying to pass a variable to the table() function, and "a parameter, which is not scalar constant string can't be passed as parameter to table() function". The workaround provided doesn't really help, since all the possible table names are not known ahead of time.
Is there any way to do this all within ADX (i.e. without multiple queries from the client) or do I need to go back to the drawing board?
if you know the desired output schema, you could potentially achieve that using union (note that in this case, the result schema will be the union of all tables, and you'll need to explicitly project the columns you're interested in)
let TableA = view() { print col1 = "hello world"};
let TableB = view() { print col1 = "goodbye universe" };
let LabelTable = datatable(table_name:string, label:string, updated:datetime)
[
"TableA", "MyLabel", datetime(2019-10-08),
"TableB", "MyLabel", datetime(2019-10-02)
];
let GetLabeledTable = (l:string)
{
toscalar(
LabelTable
| where label == l
| order by updated desc
| limit 1
)
};
let table_name = GetLabeledTable('MyLabel');
union withsource = T *
| where T == table_name
| project col1
I am referring sqlcheatsheet - Nested queries
Query 1:
traces
| where customDimensions.Domain == "someDomain"
| where message contains "some-text"
| project itemId=substring(itemId,indexof(itemId,"-"),strlen(itemId))
Result :
itemId
-c580-11e9-888a-8776d3f65945
-c580-11e9-888a-8776d3f65945
-c580-11e9-9b01-c3be0f4a2bf2
Query 2:
traces
| where customDimensions.Domain == "someDomain"
| where itemId has toscalar(
traces
| where customDimensions.Domain == "someDomain"
| where message contains "some-text"
| project itemId=substring(itemId,indexof(itemId,"-"),strlen(itemId)))
Result for the second query returns records matching only last record of sub query
ie:) > -c580-11e9-9b01-c3be0f4a2bf2
Question :
How get entire result set that has matching with all the three items.
My requirement is to take entire sequence of logs for a particular request.
To get that I have below inputs, I could able to take one log, from that I can find ItemId
The itemId looks like "b5066283-c7ea-11e9-9e9b-2ff40863cba4". Rest of all logs related to this request must have "-c7ea-11e9-9e9b-2ff40863cba4" this value. Only first part will get incremented like b5066284 , b5066285, b5066286 like that.
toscalar(), as its name implies, returns a scalar value.
Given a tabular argument with N columns and M rows it'll return the value in the 1st column and the 1st row.
For example: the following will return a single value - 1
let T = datatable(a:int, b:int, c:int)
[
1,2,3,
4,5,6,
7,8,9,
]
;
print toscalar(T)
If I understand the intention in your 2nd query correctly, you should be able to achieve your requirement by using has_any.
For example:
let T = datatable(item_id:string)
[
"c580-11e9-888a-8776d3f65945",
"c580-11e9-888a-8776d3f65945",
"c580-11e9-9b01-c3be0f4a2bf2",
]
;
T
| where item_id has_any (
(
T
| parse item_id with * "-" item_id
)
)