I have started working on Azure Data Explorer( Kusto) recently.
My requirement to make sorting order of Kusto table in dynamic way.
// Variable declaration
let SortColumn ="run_date";
let OrderBy="desc";
// Actual Code
tblOleMeasurments
| take 10
|distinct column1,column2,column3,run_date
|order by SortColumn OrderBy
Here My code working fine till Sortcolumn but when I tried to add [OrderBy] after [SortColumn] kusto gives me error .
My requirement here is to pass Asc/desc value from Variable [OrderBy].
Kindly assist here with workarounds and solutions which help me .
The sort column and order cannot be an expression, it must be a literal ("asc" or "desc"). If you want to pass the sort column and sort order as a variable, create a union instead where the filter on the variables results with the desired outcome. Here is an example:
let OrderBy = "desc";
let sortColumn = "run_date";
let Query = tblOleMeasurments | take 10 |distinct column1,column2,column3,run_date;
union
(Query | where OrderBy == "desc" and sortColumn == "run_date" | order by run_date desc),
(Query | where OrderBy == "asc" and sortColumn == "run_date" | order by run_date asc)
The number of union legs would be the product of the number of candidate sort columns times two (the two sort order options).
An alternative would be sorting by a calculated column, which is based on your sort_order and sort_column. The example below works for numeric columns
let T = range x from 1 to 5 step 1 | extend y = -10 * x;
let sort_order = "asc";
let sort_column = "y";
T
| order by column_ifexists(sort_column, "") * case(sort_order == "asc", -1, 1)
Related
Using query_parameters, how can I:
specify a result column name (ex: summarize ResultColumnName = count())
specify the value of a bin, when value is actually the name of a column in the table
This is easiest to summarize with an example:
let myTable = datatable (Timestamp:datetime)
[datetime(1910-06-11),
datetime(1930-01-01),
datetime(1997-06-25),
datetime(1997-06-25)];
let UntrustedUserInput_ColumnName = "MyCount"; // actually from query_parameters
let UntrustedUserInput_BinValue = "Timestamp"; // actually from query_parameters
let UntrustedUserInput_BinRoundTo = "365d"; // actually from query_parameters
// the query I really want to perform
myTable
| summarize MyCount=count() by bin(todatetime(Timestamp), totimespan(365d));
// what the query looks like if I use query_parameters
myTable
| summarize UntrustedUserInput_ColumnName=count() by bin(todatetime(UntrustedUserInput_BinValue), totimespan(UntrustedUserInput_BinRoundTo));
Results:
Timestamp MyCount
--------- -------
1909-09-26T00:00:00Z 1
1929-09-21T00:00:00Z 1
1996-09-04T00:00:00Z 2
Column1 UntrustedUserInput_ColumnName
------- -----------------------------
4
I can't find a solution to #1.
It appears #2 can almost be solved by using column_ifexists, but I don't have a "default" to fall back on, I'd rather just fail if the column doesn't exist.
Treating column names as variables is not possible since columns names are part of the result schema coming out of each operator (with the exception of the "evaluate" operator, see specifically the pivot plugin).
There actually is a way to set variable names to a column, using a hacky trick:
let VariableColumnName = "TestColumn"; // the new column name that you want
range i from 1 to 5 step 1 // this is just a sample query
| project pack(VariableColumnName, i) // this created a JSON
| evaluate bag_unpack(Column1) // unpacking the JSON creates a column with a dynamic name
This will return a column named TestColumn, which is set in VariableColumnName.
I am trying to write a kusto query to retrieve a custom property as below.
I want to retrieve count of pkgName and corresponding organization. I could retrieve the count of pkgName and the code is attached below.
let mainTable = union customEvents
| extend name =replace("\n", "", name)
| where iif('*' in ("*"), 1 == 1, name in ("*"))
| where true;
let queryTable = mainTable;
let cohortedTable = queryTable
| extend dimension = customDimensions["pkgName"]
| extend dimension = iif(isempty(dimension), "<undefined>", dimension)
| summarize hll = hll(itemId) by tostring(dimension)
| extend Events = dcount_hll(hll)
| order by Events desc
| serialize rank = row_number()
| extend dimension = iff(rank > 10, 'Other', dimension)
| summarize merged = hll_merge(hll) by tostring(dimension)
| project ['pkgName'] = dimension, Counts = dcount_hll(merged);
cohortedTable
Please help me to get the organization along with each pkgName projected.
Please try this simple query:
customEvents
| summarize counts=count(tostring(customDimensions.pkgName)) by pkgName=tostring(customDimensions.pkgName),organization=tostring(customDimensions.organization)
Please feel free to modify it to meet your requirement.
If the above does not meet your requirement, please try to create another table which contains pkgName and organization relationship. Then use join operator to join these tables. For example:
//create a table which contains the relationship
let temptable = customEvents
| summarize by pkgName=tostring(customDimensions.pkgName),organization=tostring(customDimensions.organization);
//then use the join operator to join these tables on the keyword pkgName.
I have a query that Counts 2 columns from 2 separate tables using subqueries, which works. Now I have to implement into this query the ability to filter out these results based on the Date of a Call Record. I will post the query in which I am working with:
SELECT (m.FirstName || " " || m.LastName) AS Members,
(
SELECT count(CallToLineOfficers.MemberID)
FROM CallToLineOfficers
WHERE CallToLineOfficers.MemberID = m.MemberID
)
+ (
SELECT count(CallToMembers.MemberID)
FROM CallToMembers
WHERE CallToMembers.MemberID = m.MemberID
) AS Tally
FROM Members AS m, Call, CallToMembers, CallToLineOfficers
Join Call on CallToMembers.CallID = Call.CallID
and CallToLineOfficers.CallID = Call.CallI
WHERE m.FirstName <> 'None'
-- and Call.Date between '2017-03-21' and '2017-03-22'
GROUP BY m.MemberID
ORDER BY m.LastName ASC;
Ok, so table Call stores the Date and its PK is CallID. Both CallToLineOfficers and CallToMembers are Bridge Tables that also contain only CallID and MemberID. With the current query, where the Date is commented out, that Date range should only return all names, but a count of 1 should appear under 1 person's name.
I have tried joining Call.CallID with both Bridge Tables' CallIDs without any luck, though I think this is the right way to do it. Could someone help point me in the right direction? I am lost. (I tried explaining this the best I could, so if you need more info, let me know.)
UPDATED: Here is a screenshot of what I am getting:
Based on the provided date in the sample, the new results, with the Date, should be:
Bob Clark - 1
Rob Catalano - 1
Matt Butler - 1
Danielle Davidson - 1
Jerry Chuska - 1
Tom Cramer - 1
Everyone else should be 0.
At the moment, the subqueries filter only on the member ID. So for any member ID in the outer query, they return the full count.
To reduce the count, you have to filter in the subqueries:
SELECT (FirstName || " " || LastName) AS Members,
(
SELECT count(*)
FROM CallToLineOfficers
JOIN Call USING (CallID)
WHERE MemberID = m.MemberID
AND Date BETWEEN '2017-03-21' AND '2017-03-22'
)
+ (
SELECT count(*)
FROM CallToMembers
JOIN Call USING (CallID)
WHERE MemberID = m.MemberID
AND Date BETWEEN '2017-03-21' AND '2017-03-22'
) AS Tally
FROM Members AS m
WHERE FirstName <> 'None'
ORDER BY LastName ASC;
I have:
TABLE MESSAGES
message_id | conversation_id | from_user | timestamp | message
I want:
1. SELECT * WHERE from_user <> id
2. GROUP BY conversation_id
3. SELECT in every group row with MAX(timestamp) **(if there are two same timestamps in a group use second factor as highest message_id)** !!!
4. then results SORT BY timestamp
to have result:
2|145|xxx|10000|message
6|1743|yyy|999|message
7|14|bbb|899|message
with eliminated
1|145|xxx|10000|message <- has same timestamp(10000) as message(2) belongs to the same conversation(145) but message id is lowest
5|1743|me|1200|message <- has message_from == me
example group with same timestamp
i want from this group row 3 but i get row 2 from query
SELECT max(message_timestamp), message_id, message_text, message_conversationId
FROM MESSAGES
WHERE message_from <> 'me'
GROUP BY message_conversationId
ORDER by message_Timestamp DESC
what is on my mind to do union from message_id & timestamp and then get max???
Your query is based on non-standard use of GROUP BY (I think SQLite allows that only for compatibility with MySQL) and I'm not at all sure that it will produce determinate results all the time.
Plus it uses MAX() on concatenated columns. Unless you somehow ensure that the two (concatenated) columns have fixed widths, the results will not be accurate for that reason as well.
I would write the query like this:
SELECT
m.message_timestamp,
m.message_id,
m.message_text,
m.message_conversationId
FROM
( SELECT message_conversationId -- for every conversation
FROM messages as m
WHERE message_from <> 'me'
GROUP BY message_conversationId
) AS mc
JOIN
messages AS m -- join to the messages
ON m.message_id =
( SELECT mi.message_id -- and find one message id
FROM messages AS mi
WHERE mi.message_conversationId -- for that conversation
= mc.message_conversationId
AND mi.message_from <> 'me'
ORDER BY mi.message_timestamp DESC, -- according to the
mi.message_id DESC -- specified order
LIMIT 1 -- (this is the one part)
) ;
Try below sql to achieve your purpose by group by twice.
select m.*
from
Messages m
-- 3. and then joining to get wanted output columns
inner join
(
--2. then selecting from this max timestamp - and removing duplicates
select conversation_id, max(timestamp), message_id
from
(
-- 1. first select max message_id in remainings after the removal of duplicates from mix of cv_id & timestamp
select conversation_id, timestamp, max(message_id) message_id
from Messages
where message <> 'me'
group by conversation_id, timestamp
) max_mid
group by conversation_id
) max_mid_ts on max_mid_ts.message_id = m.message_id
order by m.message_id;
http://goo.gl/MyZjyU
ok it was more simple than I thought:
basically to change select from:
max(message_timestamp)
to:
max(message_timestamp || message_id)
or max(message_timestamp + message_id)
so it will search for max on concatenation of timestamp and message_id
ps. after a digging - it's working only if message id is growing with timestamp ( order of insertion is preserved )
edit:
edit2 :
so why it works ?
SELECT max(message_timestamp+message_id), message_timestamp, message_id, message_conversationId, message_from,message_text
FROM MESSAGES
WHERE message_conversationId = 1521521
AND message_from <> 'me'
ORDER by message_Timestamp DESC
it is possible do a SELECT CASE, decode, nvl or another query function when I need verify if the return of a select query is empty or has a value?
For example, I have this:
Record | type | id_customer
-------+--------+-------------
1 | T | cus1
2 | A | cus2
3 | T | cus3
4 | | cus4
If I do this:
select decode(type,'T','Main','A','Adicional','none') from table where record=1;
I get Main.
If I fo this:
select decode(type,'T','Main','A','Adicional','none') from table where record=4;
I get none.
But if I do this:
select decode(type,'T','Main','A','Aditional','none') from table where record=5;
I get nothing, and is logic. So, I need get the decode value when the row exist and a text if the rows no exist.
So, I tried with SELECT CASE but is not posible get a value using COUNT. For example like this:
SELECT
CASE
WHEN count(1)>0 THEN decode(type,'T','Main','A','Aditional','none')
ELSE '-'
END
FROM TABLE WHERE record=5;
And get a ' - ', or the same if the record is 2, get 'Aditional'
Thanks a lot.
You can use aggregate functions min or max outside expression:
select max(decode(type,'T','Main','A','Aditional','none'))
from table
where record=5;
If query returns one row, you get value of that row. If query returns 0 rows, you get NULL.
Then you can replace NULL using nvl:
select nvl(max(decode(type,'T','Main','A','Aditional','none')), ' - ')
from table
where record=5;
EDIT
Also, if you need to choose one string from several:
select decode(max(decode(type,'T', 2, 'A', 1, 0)), 0, 'none', 1, 'Additional', 2, 'Main', null, ' - ')
from table
where record=5;
This is an option:
select decode(type,'T','Main','A','Aditional','none')
from table
where record = 5
union all
select '-'
from dual
where not exists (select 1 from table where record = 5);
It selects records with record = 5 and unifies them with '-', if no records exits with record = 5. Check out this Fiddle.