Kusto, Performing operations based on a condition - azure-data-explorer

I am trying to write a Kusto query, where I have a bool variable and based on that variable I want to call different functions.
For example:
let flag = true;
let result = iff(flag == "true", function1, function2)
// function1 will return a different table and function2 will return another table.
The above code is not possible because the methods like iff() and case() can only be operated on scalar values.
So, Is there any way that I could achieve this ?

the common technique is using a union:
union (function1 | where flag), (function2 | where not(flag))
Here is a full example:
let A = datatable(col1:string)["A"];
let B = datatable(col1:string)["B"];
let funcA = view(){
A
};
let funcB= view(){
B
};
let flag = true;
union (funcA() | where flag), (funcB() | where not(flag))
col1
A
And when the flag is false:
let A = datatable(col1:string)["A"];
let B = datatable(col1:string)["B"];
let funcA = view(){
A
};
let funcB= view(){
B
};
let flag = false;
union (funcA() | where flag), (funcB() | where not(flag))
col1
B

Related

Create a dynamic dictionary from a column for keys and a column for values in Kusto

If I have a table like below, how do I create a dictionary of dynamic type from the 2 columns? E.g. {"a":"1", "b":"2", etc}
let test = datatable (
keys: string,
vals: string
) [
"a,b,c,d", "1,2,3,4"
];
There is the split() and zip() function but they create array of arrays and that doesn't work with todynamic()
Alternate variation
let test = datatable (keys: string, vals: string) ["a,b,c,d", "1,2,3,4"];
test
| mv-apply k = split(keys, ",") to typeof(string)
,v = split(vals, ",") to typeof(string)
on (summarize make_bag(bag_pack(k, v)))
keys
vals
bag_
a,b,c,d
1,2,3,4
{"a":"1","b":"2","c":"3","d":"4"}
Fiddle
You could try something like this, assuming both input arrays have the same length:
test
| extend keys = split(keys, ","),
vals = split(vals, ",")
| mv-apply with_itemindex = i k = keys to typeof(string) on (
summarize bag = make_bag(pack(k, vals[i]))
)
| project bag

KQL - Make the datasource variable

i have 2 datasources which i want to be variable based on condition. But i cannot call them using variables. im not sure how can i make this work
let T1 = TableMetric;
let T2 = TableImperial;
let T3 = othertable;
let tablemetrics = case(#uommetrics=="meter",T1,#uommetrics=="imperial",T2,T3);
tablemetrics
Semantic error: '' operator: Failed to resolve scalar expression named
'T1'. Query: 'let T1 = TableMetric; let T2 = TableImperial; let T3 =
othertable; let tablemetrics =
case('a'=="meter",T1,'a'=="imperial",T2,T3); tablemetrics
This logic can be implemented with union operator
.set TableMetric <| datatable(i:int)[1,2,3]
.set TableImperial <| datatable(i:int)[4,5,6]
.set othertable <| datatable(i:int)[7,8,9]
let uommetrics = "xxx";
union
(TableMetric | where uommetrics == "meter")
,(TableImperial | where uommetrics == "imperial")
,(othertable | where uommetrics !in ("meter", "imperial"))
i
7
8
9

Kusto Query Dynamic sort Order

I have started working on Azure Data Explorer( Kusto) recently.
My requirement to make sorting order of Kusto table in dynamic way.
// Variable declaration
let SortColumn ="run_date";
let OrderBy="desc";
// Actual Code
tblOleMeasurments
| take 10
|distinct column1,column2,column3,run_date
|order by SortColumn OrderBy
Here My code working fine till Sortcolumn but when I tried to add [OrderBy] after [SortColumn] kusto gives me error .
My requirement here is to pass Asc/desc value from Variable [OrderBy].
Kindly assist here with workarounds and solutions which help me .
The sort column and order cannot be an expression, it must be a literal ("asc" or "desc"). If you want to pass the sort column and sort order as a variable, create a union instead where the filter on the variables results with the desired outcome. Here is an example:
let OrderBy = "desc";
let sortColumn = "run_date";
let Query = tblOleMeasurments | take 10 |distinct column1,column2,column3,run_date;
union
(Query | where OrderBy == "desc" and sortColumn == "run_date" | order by run_date desc),
(Query | where OrderBy == "asc" and sortColumn == "run_date" | order by run_date asc)
The number of union legs would be the product of the number of candidate sort columns times two (the two sort order options).
An alternative would be sorting by a calculated column, which is based on your sort_order and sort_column. The example below works for numeric columns
let T = range x from 1 to 5 step 1 | extend y = -10 * x;
let sort_order = "asc";
let sort_column = "y";
T
| order by column_ifexists(sort_column, "") * case(sort_order == "asc", -1, 1)

Kusto: How to convert table value to scalar and return from user defined function

I have the following user-defined functions with the intention of using a case conditional to output a table of 0s or 1s saying whether or not an account is active.
case needs scalar values as it's arguments, ie pro_account_active(account) and basic_account_active(account) need to be scalar values.
I'm struggling to get around the limitation of toscalar:
User-defined functions can't pass into toscalar() invocation
information that depends on the row-context in which the function is
called.
I think if there was a function I can use in place of the "??????" that would convert active to a scalar and return it from the function it would work.
Any help greatly appreciated
let basic_account_active=(account:string) {
basic_check_1(account) // returns 0 or 1 row only
| union basic_check_2(account)
| summarize result_count = count()
| extend active = iff(result_count == 2, 1, 0)
| ??????
};
let pro_account_active=(account:string) {
pro_check_1(account) // returns 0 or 1 row only
| union pro_check_2(account)
| summarize result_count = count()
| extend active = iff(result_count == 2, 1, 0)
| ??????
};
let is_active=(account_type:string, account:string) {
case(
account_type == 'pro', pro_account_active(account),
account_type == 'basic', basic_account_active(account),
-1
)
};
datatable(account_type:string, account:string)
[
'pro', '89e5678a92',
'basic', '9d8263da45',
'pro', '0b975f2454a',
'basic', '112a3f4753',
]
| extend result = is_active(account_type, account)
You can convert the output of a query to a scalar by using the toscalar() function, i.e.
let basic_account_active=(account:string) {
toscalar(basic_check_1(account) // returns 0 or 1 row only
| union basic_check_2(account)
| summarize result_count = count()
| extend active = iff(result_count == 2, 1, 0))};
From your example it looks that you have two tables per each account type and if both have entrees for a specific account, then the account is considered active. Is that correct? If so, I would use the "join" operator to find all the entrees in the applicable tables and count them. Here is an example of one way to do it (there are other ways as well).
let basicAccounts1 = datatable(account_type:string, account:string)[ 'basic', '9d8263da45', 'basic', '111111'];
let basicAccounts2 = datatable(account_type:string, account:string)[ 'basic', '9d8263da45', 'basic', '222222'];
let proAccounts1 = datatable(account_type:string, account:string)[ 'pro', '89e5678a92', 'pro', '111111'];
let proAccounts2 = datatable(account_type:string, account:string)[ 'pro', '89e5678a92', 'pro', '222222'];
let AllAccounts = union basicAccounts1, basicAccounts2, proAccounts1, proAccounts2
| summarize count() by account, account_type;
datatable(account_type:string, account:string)
[
'pro', '89e5678a92',
'basic', '9d8263da45',
'pro', '0b975f2454a',
'basic', '112a3f4753',
]
| join kind=leftouter hint.strategy=broadcast (AllAccounts) on account, account_type
| extend IsActive = count_ >=2
| project-away count_, account1, account_type1
The results are:

Are dynamic variables possible in SQR (not dynamic SQL)

I'm writing an SQR program to send a vendor a file containing employee info. The file contains a number of fields for which I've assigned the variables
$Code_1
$Code_2
$Code_3
....
Each code has an associated rate, and I've assigned similar variables ($Rate_1, $Rate_2, etc...)
I have a lookup table that has the columns EMPLID, JOBCODE, HOURLY_RT. I need to loop through for each employee to get all of the codes/rates. It's possible that some employees will have more/fewer than others. Is it possible to have "dynamic" variables, like we do for dynamic sql? For example, something like $Code_[$i]? The thought was to do something like this:
let #i = 1
begin-select
EC.JOBCODE
EC.HOURLY_RT
let $Code_[$i] = &EC.JOBCODE
let $Rate_[$i] = &EC.HOURLY_RT
let #i = #i + 1
FROM PS_ACME_LOOKUP EC
WHERE EC.EMPLID = &J.EMPLID
end-select
This doesn't work, but I wondering if there's a similar (or better) way to accomplish this. I suppose I could do an evaluate of the counter: when #i = 1, $Code_1 = ... when #i=2, $Code_2 =... But I'm hoping there's a better way.
Thanks
Edit - Just for added clarification, for each employee, a single line will be written to a file, with the fields for each of these values (populated or not) - so the line will have:
$EMPLID $Code_1 $Code_2 $Code_3.....$Rate_1 $Rate_2 $Rate_3
For further clarification the lookup table will have multiple rows for each employee, so the table might look like this:
EMPLID JOBCODE HOURLY_RT
0001 ABC 10.50
0001 DEF 9.75
0001 GHI 9.50
When I populate the variables, looping through the table, I would want $Code_1 = 'ABC', $Rate_1 = 10.50, $Code_2 = 'DEF', Rate_2 = 9.75 etc...
You can use arrays in SQR.
To set up the array:
Create-Array Name=WorkArray Size = 100
Field=Code
Field=Rate
Let #NumCodesForEmp = 0
To add data in your Select Block - also use on-break before and after procedures:
Begin-Select
EC.Emplid () on-break print=never before=Init-Emp After=Process-Emp
Let $Emplid = &EC.Emplid
add 1 to #NumCodesForEmp
Put &EC.JobCode &EC.Rate into WorkArray(#NumCodesForEmp) Code Rate
Write the before procedure to initialize:
Begin-Procedure Init-Emp
Let #NumCodesForEmp = 0
End-Procedure
When done with the employee:
Begin-Procedure Process-Emp
Let #I = 1
Let $OutputLine = $Emplid
While #I <= #NumCodesForEmp
Get $Code $Rate From WorkArray(#I) Code Rate
Let $OutputLine = $Outputline || ',' || $Code || ',' || $Rate
add 1 to #I
End-While
! This assumes that file number 10 is open
Write #10 from $OutputLine
End-Procedure
However, I think you could do everything without an array - use the before and after procedures as so:
Begin-Procedure Init-Emp
Let $OutputLine = &EC.Emplid
End-Procedure
Begin-Procedure Process-Emp
Write #10 from $OutputLine
End-Procedure
Then the Select Block would look like this:
Begin-Select
EC.Emplid () on-break print=never before=Init-Emp After=Process-Emp
EC.JobCode
EC.Rate
Let $OutputLine = $OutputLine || ',' || &EC.Jobcode || ',' || &EC.Rate
When using on-break, make sure you sort by emplid. This is much simpler if your need is just to write a file from data from a table.

Resources