my logs contain in customDimensions multiple entries.
{"MessageType":"EventLog","Properties":"{\"Action\":\"Manual Trigger\",\"City\":\"New York\"}","AppBuild":"22"}
How do I extract the entries in "Properties" and put them in separate columns?
{\"Action\":\"Manual Trigger\",\"City\":\"New York\"}
Action
City
Manual Trigger
New York
For some reason, the entries in "Properties" do not have the same order. Sometimes "City" can be first.
I could extract properties, but this is only halfway
let events = customEvents
| extend properties = tostring(customDimensions["Properties"])
or
let events = customEvents
| extend properties = tostring(customDimensions.Properties)
Splitting the result could work, except, that the entries do not have the same order for each log entry.
you could use the bag_unpack() plugin.
for efficiency, make sure you specify the output schema (in this case (Action: string, City: string))
datatable(customDimensions:dynamic)
[
dynamic({"MessageType":"EventLog","Properties":"{\"Action\":\"Manual Trigger\",\"City\":\"San Francisco\"}","AppBuild":"21"}),
dynamic({"MessageType":"EventLog","Properties":"{\"Action\":\"Manual Trigger\",\"City\":\"New York\"}","AppBuild":"22"})
]
| extend properties = parse_json(tostring(customDimensions.Properties))
| evaluate bag_unpack(properties) : (Action: string, City: string)
Action
City
Manual Trigger
San Francisco
Manual Trigger
New York
Related
I have a RequestTelemetry operation. It has two events within it. If I query for only the operation, I get the request, one row for each TrackEvent item, when I expand the > for the operation in the Results pane I have two rows, one for each event, but the same operation.
requests
| where operation_Name == 'my_operation'
Will give me a row for each event, as these events have the same operation id.
Using the operation_id from this query, I can query customEvents,
customEvents
| where operation_id == 'myid'
I get the events, two in this case, two rows, as expected just like my requests query.
I have tried a join with,
requests
| join kind=inner customEvents on operation_Name
| where operation_Name == 'my_operation'
and I get the rows, each having a different customEvent (the two I did) like above. It is the same result if I add another filter | operation_id == 'myid'.
How can I have one Kusto query that has a single row for the operation and when I expand it in the results, I can see all the events associated with it? Not multiple rows, one for each event in an operation, but a single row for operation and have > that I can expand underneath to show all the events? In addition, the events would be expandable like they are if I had queried by themselves.
I also looked at mv-expand, mv_bag, and Kusto Query Ingestion with commands, but I am not sure if I'm on the right track.
Edit: It would look like this in the Results pane as a single row,
SendOperation
myevent1
myevent2
From (example snippet, not guaranteed to compile):
Using(IOperationHolder<DependencyTelemetry> op = new TelemetryClient().StartOperation<DependencyTelemetry>("SendOperation"))
{
TelemetryClient telemetry = new TelemetryClient();
telemetry.TrackEvent
(
"myevent1",
new Dictionary<string, string>()
{
{"some_var_value", JsonConvert.SerializeObject(myobjFromSomewhere)},
{"some_other_var", theVariableINeedToTrack}
}
telemetry.TrackEvent
(
"myevent2",
new Dictionary<string, string>()
{
{"useful_metric", usefulVariableValue},
}
}
}```
Edit:,
timestamp:
id:
name: my operation
event: event1
event: event2
or,
timestamp:
id:
name: my operation
CustomDimensions: {event1:{...}, event2:{...}}
I am trying to identify shared mailboxes that aren't in use. Checked "Search-MailboxAuditLog" already and some mailboxes do not return any results even tho auditing enabled, but can see activity in Azure sentinel.
Is there a way to improve below Kusto code? (During testing tried mailboxes with activities but sometimes do not get any results from the query)
With Kusto, Is there a way to loop through "mbs" like powershell "foreach ( $item in $mbs)"?
Thanks,
let mbs = datatable (name: string)
[
"xxx1#something.com",
"xxx2#something.com",
"xxx3#something.com",
];
OfficeActivity
| where OfficeWorkload == "Exchange" and TimeGenerated > ago(30d)
| where MailboxOwnerUPN in~ (mbs)
| distinct MailboxOwnerUPN
Update : Need help with the query
Input would be list of shared mailbox UPNs
Output would be list of shared mailboxes with any activity, example MBs with any action in “Operation" filed
"in" doesn't work on datatables (tabular inputs) like that; it is not a "filter", it is an "operator". The "where" is effectively the "foreach" you are referring to.
Given the sample input, the query could probably be written as:
OfficeActivity //tabular input with many records
| TimeGenerated > ago(30d) //Filter records to window of interest first
| where OfficeWorkload == "Exchange" //foreach row
| where MailboxOwnerUPN in~ ( //foreach row
"xxx1#something.com","xxx2#something.com","xxx3#something.com"
)
| distinct MailboxOwnerUPN
You can see it in the docs at https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/inoperator#arguments where "col" is the "column to filter"
I have a need to be able to query Azure Data Explorer (ADX) tables dynamically, that is, using application-specific metadata that is also stored in ADX.
If this is even possible, the way to do it seems to be via the table() function. In other words, it feels like I should be able to simply write:
let table_name = <non-trivial ADX query that returns the name of a table as a string>;
table(table_name) | limit 10
But this query fails since I am trying to pass a variable to the table() function, and "a parameter, which is not scalar constant string can't be passed as parameter to table() function". The workaround provided doesn't really help, since all the possible table names are not known ahead of time.
Is there any way to do this all within ADX (i.e. without multiple queries from the client) or do I need to go back to the drawing board?
if you know the desired output schema, you could potentially achieve that using union (note that in this case, the result schema will be the union of all tables, and you'll need to explicitly project the columns you're interested in)
let TableA = view() { print col1 = "hello world"};
let TableB = view() { print col1 = "goodbye universe" };
let LabelTable = datatable(table_name:string, label:string, updated:datetime)
[
"TableA", "MyLabel", datetime(2019-10-08),
"TableB", "MyLabel", datetime(2019-10-02)
];
let GetLabeledTable = (l:string)
{
toscalar(
LabelTable
| where label == l
| order by updated desc
| limit 1
)
};
let table_name = GetLabeledTable('MyLabel');
union withsource = T *
| where T == table_name
| project col1
I have an ASP.NET Web API app using Oracle's Entity Framework driver. I have an entity defined for a view as follows:
CREATE OR REPLACE FORCE VIEW "PHASE_TWO"."EDIPRODUCT" ("ID", "STK_NUM", "TITLE", "ISBN", "UPC", "ITEMNO", "LONGFORMAT", "ABRIDGED", "WEB_TITLES_ID", "OCLC", "GENRE", "RELYEAR", "ORIG_REL", "LANG", "ORIG_STKNUM", "PUBLISHER", "PEOPLELIST", "SALES_ORG", "NOT_AVAIL") AS
SELECT sap_product.id,
sap_product.stk_num,
sap_product.longdesc AS title,
sap_product.isbn,
sap_product.upc,
sap_product.itemno,
sap_product.longformat,
sap_product.abridged,
mwt_product.web_titles_id,
mwt_product.oclc,
mwt_product.genre,
mwt_product.RELYEAR,
sap_product.orig_rel,
sap_product.lang,
sap_product.orig_stknum,
UPPER (publisher.name) publisher,
(SELECT LISTAGG (p.FULLNAME, ', ') WITHIN GROUP (
ORDER BY pp.rank) AS People
FROM people p
JOIN product_people pp
ON p.id = pp.peopleid
WHERE pp.stk_num = sap_product.stk_num
GROUP BY pp.STK_NUM
) PeopleList,
sppg.PRICING_TYPE as sales_org,
sap_product.not_avail
FROM sap_product
JOIN mwt_product ON sap_product.stk_num = mwt_product.stk_num
JOIN publisher ON mwt_product.publisherid = publisher.id
JOIN SAP_PRODUCT_PRICING_GROUP sppg on sppg.STK_NUM = mwt_product.stk_num and sppg.MARKED_FOR_DELETION = 0
WHERE mwt_product.WEB_PRODUCTS_ID > 0;
This view works as expected in SQL Developer. My getEDIPRODUCT function (yes, it's VB.NET) in my controller is as follows:
' GET: odata/EDIPRODUCTs
<EnableQuery>
Function GetEDIPRODUCT() As IQueryable(Of EDIPRODUCT)
Dim results As IQueryable
results = db.EDIPRODUCT
For Each _product In results
Console.Write(_product)
Next
Return results
End Function
I just added the for loop in order to inspect the results. What I see when I inspect the results is the same product record is returned for each row. The value for the ID is duplicate and the only other field that should have variant values (sppg.PRICING_TYPE as sales_org) also just repeats.
I have other views where this does not occur. The correct number of records are always returned, but the first record retrieved is always just repeated in each row of the result set. Any idea what could be going on here?
I never actually resolved this issue and am still interested in why this fails, but I rewrote the portion of the app that uses this view to use OData's $expand to retrieve the related data.
Actually I spend whole day on the EntityFramework for foreign key.
assume we have two table.
Process(app_id,process_id)
LookupProcessId(process_id, process_description)
you can understand two tables with names, first table ,use process_id to indicate every application, and description is in the seoncd table.
Actually i try many times and figure out how to do inquery: it was like
Dim result = (from x in db.Processes where x.LookupProcess is (from m in db.LookupProcessIds where descr = "example" select m).FirstOrDefault() select x).FirstOrDefault()
First I want to ask is there easier way to do it.
Second i want to ask question is about insert
p As New AmpApplication.CUEngData.Process
p.app_id=100
p.LookupProcess = (from m in db.LookupProcessIds where descr = "example" select m).FirstOrDefault()
db.AddToProcesses(p)
db.SaveChanges()
from appearance it looks fine, but it give me error says
Entities in 'AmpCUEngEntities.Processes' participate in the 'FK_Process_LookupProcess' relationship. 0 related 'LookupProcess' were found. 1 'LookupProcess' is expected.
can i ask is that insert wrong? and is that my query correct?
For your first question:
Dim result = (from x in db.Processes
where x.LookupProcess.descr = "example"
select x).FirstOrDefault()
Actually, you missed some concepts from DataEntityModel, and its Framework. To manipulate data, you have to call object from contextual point of view. Those allow you to specify to the ObjectStateManager the state of an DataObject. In your case, if you have depending data from FK, you will have to add/update any linked data from leaf to root.
This example demonstrate simple (no dependances) data manipulation. A select if existing and an insert or update.
If you want more info about ObjectStateManager manipulation go to http://msdn.microsoft.com/en-us/library/bb156104.aspx
Dim context As New Processing_context 'deseign your context (this one is linked to a DB)
Dim pro = (From r In context.PROCESS
Where r.LOOKUPPROCESS.descr = LookupProcess.descr
Select r).FirstOrDefault()
If pro Is Nothing Then 'add a new one
pro = New context.PROCESS With {.AP_ID = "id", .PROCESS_ID = "p_id"}
context.PROCESS.Attach(pro)
context.ObjectStateManager.ChangeObjectState(pro, System.Data.EntityState.Added)
Else
'update data attibutes
pro.AP_ID = "id"
pro.PROCESS_ID = "p_id"
context.ObjectStateManager.ChangeObjectState(pro, System.Data.EntityState.Modified)
'context.PROCESS.Attach(pro)
End If
context.SaveChanges()
I hope this will help. Have a nice day!
For your first question, to expand on what #jeroenh suggested:
Dim result = (from x in db.Processes.Include("LookupProcess")
where x.LookupProcess.descr = "example"
select x).FirstOrDefault()
The addition of the Include statement will hydrate the LookupProcess entities so that you can query them. Without the Include, x.LookupProcess will be null which would likely explain why you got the error you did.
If using the literal string as an argument to Include is not ideal, see Returning from a DbSet 3 tables without the error "cannot be inferred from the query" for an example of doing this using nested entities.
For your second question, this line
p.LookupProcess = (from m in db.LookupProcessIds
where descr = "example" select m).FirstOrDefault()
Could cause you problems later on because if there is no LookupProcessId with a process_description of "example", you are going to get null. From MSDN:
The default value for reference and nullable types is null.
Because of this, if p.LookupProcess is null when you insert the entity, you will get the exception:
Entities in 'AmpCUEngEntities.Processes' participate in the 'FK_Process_LookupProcess' relationship. 0 related 'LookupProcess' were found. 1 'LookupProcess' is expected.
To avoid this kind of problem, you will need to check that p.LookupProcess is not null before it goes in the database.
If Not p.LookupProcess Is Nothing Then
db.AddToProcesses(p)
db.SaveChanges()
End If