I have a where condition which I want to run over a set of tables in my Azure Data Explorer DB. I found "Find in ()" operator in Kusto query quite useful, works fine when I pass list of tables as intended.
find withsource=DataType in (AppServiceFileAuditLogs,AzureDiagnostics)
where TimeGenerated > ago(31d)
project _ResourceId, _BilledSize, _IsBillable
| where _IsBillable == true
| summarize BillableDataBytes = sum(_BilledSize) by _ResourceId, DataType | sort by BillableDataBytes nulls last
However, in my scenario, I would like to decide the list of tables at run time using another query.
Usage
| where TimeGenerated > ago(32d)
| where StartTime >= startofday(ago(31d)) and EndTime < startofday(now())
| where IsBillable == true
| summarize BillableDataGB = sum(Quantity) / 1000 by DataType
| sort by BillableDataGB desc
|project DataType
find withsource=DataType in (<pass resulting table expression from above query here as comma separated list of tables>)
where TimeGenerated > ago(31d)
project _ResourceId, _BilledSize, _IsBillable
| where _IsBillable == true
| summarize BillableDataBytes = sum(_BilledSize) by _ResourceId, DataType | sort by BillableDataBytes nulls last
Found some examples of passing all tables in a database or cluster using wildcards but that does not fit my scenario. Can somebody help me here.
Here is one way to achieve this:
let Tables = toscalar(Usage
| where TimeGenerated > ago(32d)
| where StartTime >= startofday(ago(31d)) and EndTime < startofday(now())
| where IsBillable == true
| summarize by DataType);
union withsource=T *
| where T in (Tables)
| count
Note that there is a significance to the toscalar expression, it precalculates the list of tables and optimizes the filter on the union expression. I also updated your query to avoid unnecessary work.
I need help creating a single SELECT statement as part of a CREAT VIEW statement that contains multiple, separate filtering or grouping requirements.
I am working on an SQLite database to track usage of our local food pantry, where we have two types of visitors, “Scheduled” or “Drop-In”, visiting on different days. One of the central tables is the “visit_log” table that tracks each visit by date, time, type of visit, and people in the household.
I’m trying to create a VIEW that summarizes that “visit_log” grouped by the visit_date, and for both number of records and SUM of household size, displaying the number of “Drop-Ins”, the number of “Scheduled” and the total of the two types.
Here is the “visit_log”
CREATE TABLE "visit_log" ("visit_date" DATE, "visit_time" TIME, "client_relation" TEXT, "household_size" INTEGER)
Here is a sample of the “visit_log” table’s content. (We have not started recording the visit_time yet, so those values are blank).
"visit_date","visit_time","client_relation","household_size"
"6/9/20","","Scheduled","1"
"6/9/20","","Scheduled","1"
"6/9/20","","Drop-In","2"
"6/9/20","","Drop-In","3"
"6/9/20","","Drop-In","8"
"6/9/20","","Drop-In","5"
"6/16/2020","","Scheduled","1"
"6/16/2020","","Scheduled","1"
"6/16/2020","","Drop-In","4"
"6/16/2020","","Drop-In","5"
"6/16/2020","","Drop-In","2"
"6/16/2020","","Drop-In","2"
"6/16/2020","","Drop-In","5"
"6/16/2020","","Drop-In","1"
I can create three separate VIEW, one for each type and one for the two combined. But my goal is to have the results of these three VIEWs in one.
Here are the three VIEWs. First is for the two client types combined.
CREATE VIEW "visit_summary" AS SELECT
visit_date,
COUNT (*) AS households_total,
SUM (household_size) AS individuals_total
FROM
"visit_log"
GROUP By visit_date
This yields
"visit_date","households_total","individuals_total"
"06/09/2020","12","44"
"06/16/2020","8","21"
"06/23/2020","7","20"
"06/30/2020","10","22"
"07/07/2020","7","18"
Next is the VIEW for the Drop-Ins
CREATE VIEW "visit_summary_dropin" AS SELECT
visit_date,
COUNT (*) AS households_dropin,
SUM (household_size) AS individuals_dropin
FROM
"visit_log"
WHERE client_relation = "Drop-In"
GROUP By visit_date
This yields
"visit_date","households_dropin","individuals_dropin"
"06/09/2020","10","42"
"06/16/2020","6","19"
"06/23/2020","4","13"
"06/30/2020","6","12"
"07/07/2020","6","16"
Finally is the VIEW for the Scheduled
CREATE VIEW "visit_summary_scheduled" AS SELECT
visit_date,
COUNT (*) AS households_schedualed,
SUM (household_size) AS individuals_scheduled
FROM
"visit_log"
WHERE client_relation = "Scheduled"
GROUP By visit_date
This yields
"visit_date","households_schedualed","individuals_scheduled"
"06/09/2020","2","2"
"06/16/2020","2","2"
"06/23/2020","3","7"
"06/30/2020","4","10"
"07/07/2020","1","2"
What I'm hoping to create is a single VIEW that yields
"visit_date","households_total","individuals_total","households_dropin","individuals_dropin","households_schedualed","individuals_scheduled"
"06/09/2020","12","44","10","42","2","2"
etc…
So my ultimate question, finally, is how to create a single VIEW containing something like multiple WHERE classes to define different columns?
You can do it with conditional aggregation:
CREATE VIEW visit_summary_scheduled_all AS
SELECT visit_date,
COUNT(*) households_total,
SUM(household_size) individuals_total,
SUM(client_relation = 'Drop-In') households_dropin,
SUM(CASE WHEN client_relation = 'Drop-In' THEN household_size END) individuals_dropin,
SUM(client_relation = 'Scheduled') households_scheduled,
SUM(CASE WHEN client_relation = 'Scheduled' THEN household_size END) individuals_scheduled
FROM visit_log
GROUP By visit_date
See the demo.
Results:
| visit_date | households_total | individuals_total | households_dropin | individuals_dropin | households_scheduled | individuals_scheduled |
| ---------- | ---------------- | ----------------- | ----------------- | ------------------ | -------------------- | --------------------- |
| 6/16/2020 | 8 | 21 | 6 | 19 | 2 | 2 |
| 6/9/20 | 6 | 20 | 4 | 18 | 2 | 2 |
I am trying to write a query that will get me the average time between 2 custom events, sorted by user session. I have added custom tracking events throughout this application and I want to query the time it takes the user from 'Setup' event to 'Process' event.
let allEvents=customEvents
| where timestamp between (datetime(2019-09-25T15:57:18.327Z)..datetime(2019-09-25T16:57:18.327Z))
| extend SourceType = 5;
let allPageViews=pageViews
| take 0;
let all = allEvents
| union allPageViews;
let step1 = materialize(all
| where name == "Setup" and SourceType == 5
| summarize arg_min(timestamp, *) by user_Id
| project user_Id, step1_time = timestamp);
let step2 = materialize(step1
| join
hint.strategy=broadcast (all
| where name == "Process" and SourceType == 5
| project user_Id, step2_time=timestamp
)
on user_Id
| where step1_time < step2_time
| summarize arg_min(step2_time, *) by user_Id
| project user_Id, step1_time,step2_time);
let 1Id=step1_time;
let 2Id=step2_time;
1Id
| union 2Id
| summarize AverageTimeBetween=avg(step2_time - step1_time)
| project AverageTimeBetween
When I run this query it produces this error message:
'' operator: Failed to resolve table or column or scalar expression named 'step1_time'
I am relatively new to writing queries with AI and have not found many resources to assist with this problem. Thank you in advance for your help!
I'm not sure what the let 1id=step1_time lines are intended to do.
those lines are trying to declare a new value, but step1_time isn't a thing, it was a field in another query
i'm also not sure why you're doing that pageviews | take 0 and unioning it with events?
let allEvents=customEvents
| where timestamp between (datetime(2019-09-25T15:57:18.327Z)..datetime(2019-09-25T16:57:18.327Z))
| extend SourceType = 5;
let step1 = materialize(allEvents
| where name == "Setup" and SourceType == 5
| summarize arg_min(timestamp, *) by user_Id
| project user_Id, step1_time = timestamp);
let step2 = materialize(step1
| join
hint.strategy=broadcast (allEvents
| where name == "Process" and SourceType == 5
| project user_Id, step2_time=timestamp
)
on user_Id
| where step1_time < step2_time
| summarize arg_min(step2_time, *) by user_Id
| project user_Id, step1_time,step2_time);
step2
| summarize AverageTimeBetween=avg(step2_time - step1_time)
| project AverageTimeBetween
if I remove the things I don't understand (like union with 0 pageviews, and the lets, I get a result, but I don't have your data so I had to use other values than "Setup" and "Process" so I don't know if it is what you expect?
you might want to look at the results of the step2 query without the summarize to just see what you're getting matches what you expect.
is it possible to get the query to summarize from multiple Application insights? I cant get it working with Union command.
Example query:
union
app("applicationinsight02").requests,
app("applicationinsight03").requests
availabilityResults
| where timestamp > ago(30d)
// check whether location failed within 5m bin
| summarize _failure=iff(countif(success == 0)>0, 1, 0) by name, location, bin(timestamp, 5m)
// check whether all locations failed within 5m bin
| summarize _failureAll=iff(sum(_failure)>=3, 1, 0) by name, bin(timestamp, 5m)
// count all failed 5 minute bins and total number of bins
| summarize _failuresCount=sum(_failureAll), _totalCount=count() by name
| project ["Name"] = name, ["SLA"] = todouble(_totalCount - _failuresCount) / todouble(_totalCount) * 100
| order by ["SLA"]
Yes, something like so
union
app("application-insights-01").requests,
app("application-insights-02").requests
| where timestamp > ago(1h)
| summarize sum(itemCount) by appName, bin(timestamp, 5m)
That will summarize the requests and show you the split by appname (the app insights resource name). Amend the where clause to fit your requirements
An example for availability results with your query would look like so, just replace application-insights-01/02 with your instance names
union
app("application-insights-01").availabilityResults,
app("application-insights-02").availabilityResults
| where timestamp > ago(1h)
| summarize _failure=iff(countif(success == 0)>0, 1, 0) by name, location, bin(timestamp, 5m)
| summarize _failureAll=iff(sum(_failure)>=3, 1, 0) by name, bin(timestamp, 5m)
| summarize _failuresCount=sum(_failureAll), _totalCount=count() by name
| project ["Name"] = name, ["SLA"] = todouble(_totalCount - _failuresCount) / todouble(_totalCount) * 100
| order by ["SLA"]
User wants a count of unique sessions per week in application insights. I have the query working, including a pivot, but the Week columns are out of order. I would prefer if they were in order.
pageViews
| where timestamp < now()
| summarize Sessions= dcount(session_Id)
by Week=bin(datepart("weekOfYear", timestamp), 1), user_AuthenticatedId
| order by Week
| evaluate pivot(Week, sum(Sessions))
| join kind=innerunique (pageViews
| summarize MostRecentRequest = max(timestamp) by user_AuthenticatedId)
on $right.user_AuthenticatedId == $left.user_AuthenticatedId
| project-away user_AuthenticatedId1
I've tried ordering by timestamp before the summarize, and ordering by week after the summarize (still in there) and no luck.
There's currently a "trick" that will work: serialize right after your order by
pageViews
| where timestamp < now()
| where isnotempty(user_AuthenticatedId)
| summarize Sessions= dcount(session_Id)
by Week=bin(datepart("weekOfYear", timestamp), 1), user_AuthenticatedId
| order by Week
| serialize // <--------------------------------- RIGHT HERE
| evaluate pivot(Week, sum(Sessions))
| join kind=innerunique (pageViews
| summarize TotalSessions=dcount(session_Id), MostRecentRequest = max(timestamp) by user_AuthenticatedId)
on $right.user_AuthenticatedId == $left.user_AuthenticatedId
| project-away user_AuthenticatedId1
| top 100 by TotalSessions desc
gets me this in workbooks, with the weeks in descending order (I also added total session count to sort/top by with some custom column settings set):
the custom settings I have for the column settings in workbooks:
delete all the #'d columns that are there by default and add one for ^[0-9]+$ set to heatmap:
I refactored query a bit for my own comprehension. I took the the left and right into "views". Thought I'd share.
let users_MostRecent_Session =
pageViews
| summarize
TotalSessions=dcount(session_Id)
, MostRecentRequest = max(timestamp)
by
user_AuthenticatedId
;
//
let users_sessions_ByWeek =
pageViews
| where timestamp < now()
| where isnotempty(user_AuthenticatedId)
| summarize
Sessions= dcount(session_Id)
by
Week=bin(datepart("weekOfYear", timestamp), 1)
, user_AuthenticatedId
| order by Week
| serialize
| evaluate pivot(Week, sum(Sessions))
;
//
//
users_sessions_ByWeek
| join kind=innerunique
users_MostRecent_Session
on user_AuthenticatedId
| project-away user_AuthenticatedId1
| top 100 by TotalSessions desc