(for-each-row scenario).in kusto - azure-data-explorer

Query1
cluster(x).database('$systemdb').Operations
| where Operation == "DatabaseCreate" and Database contains "oci-"| where State =='Completed'
and StartedOn between (datetime(2020-04-07) .. 3d)
| distinct Database , StartedOn
| order by StartedOn desc
Output of my query1 is list of databases , now I have to pass each db value into query2 to get buildnumber
Query2:
set query_take_max_records=5000;
let view=datatable(Property:string,Value:dynamic)[];
let viewFile=datatable(FileName:string)[];
alias database db = cluster(x).database('y');
let latestInfoFile = toscalar((
union isfuzzy=true viewFile,database('db').['TextFileLogs']
| where FileName contains "AzureStackStampInformation"
| distinct FileName
| order by FileName
| take 1));
union isfuzzy=true view,(
database('db').['TextFileLogs']
| where FileName == latestInfoFile
| distinct LineNumber,FileLineContent
| order by LineNumber asc
| summarize StampInfo=(toobject(strcat_array(makelist(FileLineContent,100000), "\r\n")))
| mvexpand bagexpansion=array StampInfo
| project Property=tostring(StampInfo[0]), Value=StampInfo[1]
)|where Property contains "StampVersion" | project BuildNumber = Value;

database() function: is a special scoping function, and it does not support non-constant arguments due to security consideration.
As a result - you cannot use sub-query to fetch list of databases and then operate on this list as input for database() function.
This behavior is described at:
https://learn.microsoft.com/en-us/azure/kusto/query/databasefunction?pivots=azuredataexplorer
Syntax
database(stringConstant)
Arguments
stringConstant: Name of the database that is referenced. Database identified can be either DatabaseName or PrettyName. Argument has to be constant prior of query execution, i.e. cannot come from sub-query evaluation.

Related

Kusto - how to ingest from query using management (.show) function

I want to have a table that stores only daily tables sizes.
But it won't work this way:
.set-or-replace async tables_daily_storage <|
(
.show cluster extents
| where MinCreatedOn >= startofday(now())
| project DatabaseName,TableName,OriginalSize,D=bin(MinCreatedOn,1d)
| summarize total_size=sum(OriginalSize) by DatabaseName, TableName
)
Because I used .show function which is a management function.
Is there anything to get around this problem?
Simply remove the brackets
doc
.set-or-replace async tables_daily_storage <|
.show cluster extents
| where MinCreatedOn >= startofday(now())
| project DatabaseName,TableName,OriginalSize,D=bin(MinCreatedOn,1d)
| summarize total_size=sum(OriginalSize) by DatabaseName, TableName

How to write Kusto query to get results in one table?

I have 2 KQL queries and I want to combine them in order to display two rows as one result. Not just result of first query, then result of second query:
R_CL
| where isnotempty(SrcIP_s)
| project Message
| take 1;
R_CL
| where isempty(SrcIP_s)
| project Message
| take 1
See sample R_L below.I would like to see 2 rows as result, one with SrcIP_s not empty, and the second with SrcIP_s empty (in this case it will be always same one)
let R_CL = datatable ( SrcIP_s:string, Message:string)
["1.1.1.1" ,"one",
"" ,"two",
"2.2.2.2","three",
"3.3.3.3","four"];
R_CL
| project SrcIP_s, Message
A simple solution for this would be to use the union operator like this:
let query1 = R_CL
| where isnotempty(SrcIP_s)
| project Message
| take 1;
let query2 = R_CL
| where isempty(SrcIP_s)
| project Message
| take 1;
query1
| union query2;
I know this is an old request - but here's a sample query using views and a union for your single query:
Your two separate queries...
R_CL
| where isnotempty(SrcIP_s)
| project Message
| take 1;
R_CL
| where isempty(SrcIP_s)
| project Message
| take 1
would become:
let Query1 = view () {
R_CL
| where isnotempty(SrcIP_s)
| project Message
| take 1;
};
let Query2 = view () {
R_CL
| where isempty(SrcIP_s)
| project Message
| take 1
};
union withsource="TempTableName" Query1, Query2

JanusGraph - Warning about all vertices scan after index was created

I am using Janusgraph 0.2.0 and have the following vertex defined (in Python):
class Airport(TypedVertex):
type = goblin.VertexProperty(goblin.String, card=Cardinality.single)
airport_code = goblin.VertexProperty(goblin.String,
card=Cardinality.single)
airport_city = goblin.VertexProperty(goblin.String,
card=Cardinality.single)
airport_name = goblin.VertexProperty(goblin.String,
card=Cardinality.single)
airport_region = goblin.VertexProperty(goblin.String,
card=Cardinality.single)
airport_runways = goblin.VertexProperty(goblin.Integer,
card=Cardinality.single)
airport_longest_runway = goblin.VertexProperty(goblin.Integer,
card=Cardinality.single)
airport_elev = goblin.VertexProperty(goblin.Integer,
card=Cardinality.single)
airport_country = goblin.VertexProperty(goblin.String,
card=Cardinality.single)
airport_lat = goblin.VertexProperty(goblin.Float,
card=Cardinality.single)
airport_long = goblin.VertexProperty(goblin.Float,
card=Cardinality.single)
I then defined an index for this node on the airport code field using the following commands (some commands were excluded to keep it short).
mgmt.makePropertyKey('type').dataType(String.class).cardinality(Cardinality.SINGLE).make()
mgmt.makePropertyKey('airport_city').dataType(String.class).cardinality(Cardinality.SINGLE).make()
mgmt.makePropertyKey('airport_code').dataType(String.class).cardinality(Cardinality.SINGLE).make()
mgmt.makePropertyKey('airport_country').dataType(String.class).cardinality(Cardinality.SINGLE).make()
airport_code = mgmt.getPropertyKey('airport_code')
airport_city = mgmt.getPropertyKey('airport_city')
airport_country = mgmt.getPropertyKey('airport_country')
mgmt.buildIndex('by_airport_code_unique', Vertex.class).addKey(airport_code).unique().buildCompositeIndex()
mgmt.buildIndex('by_airport_city', Vertex.class).addKey(airport_city).buildCompositeIndex()
mgmt.buildIndex('by_airport_country', Vertex.class).addKey(airport_country).buildCompositeIndex()
mgmt.awaitGraphIndexStatus(graph, 'by_airport_code_unique').call()
mgmt.awaitGraphIndexStatus(graph, 'by_airport_city').call()
mgmt.awaitGraphIndexStatus(graph, 'by_airport_country').call()
After the creating, I use a script to describe the :schema and I see that all the indexes are Registered:
| Graph Index . | Type . | Element | Unique | Backing | PropertyKey | Status |
|-----------------------:|:-----|:--------|:-------|:--------|:-----------|:--------|
| by_airport_code_unique | Composite | JanusGraphVertex | true | internalindex | airport_code | REGISTERED |
| by_airport_city | Composite | JanusGraphVertex | false | internalindex | airport_city | REGISTERED |
| by_airport_country | Composite | JanusGraphVertex | false | internalindex | airport_country | REGISTERED |
When I try to insert the second vertex with the same airport_code, as expected, I get an exception on constraint violation. However, if I go into the gremlin console and run a traversal to retrieve the vertices by their airport_code:
g.V().has('airport_code').values()
I get a warning: WARN org.janusgraph.graphdb.transaction.StandardJanusGraphTx - Query requires iterating over all vertices [()]. For better performance, use indexes
I had a similar problem a few weeks ago, and the issue was that I was trying to define indexes based on labels and I was told that at the time, janusgraph does not support indexes on labels. However, I don't think this is the case here.
Any suggestions or ideas on why my index is not working or not being used?
Thanks in advance for any help.
--MD
You are seeing the warning because your query does not utilize the index. A composite index is used for equality matches.
Composite indexes are very fast and efficient but limited to equality lookups for a particular, previously-defined combination of property keys. Mixed indexes can be used for lookups on any combination of indexed keys and support multiple condition predicates in addition to equality depending on the backing index store.
In order to leverage a composite index, you need to provide the property and a value to match. For example:
g.V().has('airport_code', 'JFK').toList()
I'm not sure why the index wasn't ENABLED after creation, perhaps something in the steps you left out. If you create the index within the same management transaction as the property keys, it should be ENABLED rather than REGISTERED. Check out the index lifecycle wiki.

In Application Insights analytics how to query what percent of users are impacted by specific exception?

I use this query to display exceptions:
exceptions
| where application_Version == "xyz"
| summarize count_=count(itemCount), impactedUsers=dcount(user_Id) by problemId, type, method, outerMessage, innermostMessage
| order by impactedUsers
How to query what percent of users are impacted by specific exception?
I would check all users by this query:
customEvents
| where application_Version == "xyz"
| summarize dcount(user_Id)
You're almost there with what you have, you just need to connect the two:
use let + toscalar to define the results of a query as a number
reference that in your query (i used *1.0 to force it to be a float, otherwise you get 0, and used round to get 2 decimals, adjust that however you need)
making your query:
let totalUsers = toscalar(customEvents
| where application_Version == "xyz"
| summarize dcount(user_Id));
exceptions
| where application_Version == "xyz"
| summarize count_=count(itemCount),
impactedUsers=dcount(user_Id),
percent=round(dcount(user_Id)*1.0/totalUsers*100.0,2)
by problemId, type, method, outerMessage, innermostMessage
| order by impactedUsers

What data type is returned by this LocalStorage command?

There is an LocalStorage example in the Qt documentation
function findGreetings() {
var db = LocalStorage.openDatabaseSync("QQmlExampleDB", "1.0", "The Example QML SQL!", 1000000);
db.transaction(
function(tx) {
// Some other commands
// Show all added greetings
var rs = tx.executeSql('SELECT * FROM Greeting');
}
)
}
What's the data type of rs?
See the Quick Local Storage QML module documentation:
results = tx.executeSql(statement, values)
This method executes a SQL statement, binding the list of values to
SQL positional parameters ("?").
It returns a results object, with the following properties:
| Type | Property | Value | Applicability |
-----------------------------------------------------------------------------------------
| int | rows.length | The number of rows in the result | SELECT |
-----------------------------------------------------------------------------------------
| var | rows.item(i) | Function that returns row i of the result | SELECT |
-----------------------------------------------------------------------------------------
| int | rowsAffected | The number of rows affected by a modification | UPDATE,DELETE |
-----------------------------------------------------------------------------------------
| string | insertId | The id of the row inserted | INSERT |
results = tx.executeSql(statement, values)
This method executes a SQL statement, binding the list of values to SQL positional parameters ("?").
It returns a results object, with the following properties: link
If all you want is to know the type of returned object, just do:
var rs = tx.executeSql(...);
console.log(rs);
qml: [object Object]

Resources