How do I use dynamic keyword in Kusto - azure-data-explorer

I'm using Azure Resource Graph which is using Kusto language for query Azure resources and confused how I can create my own objects via dynamic keyword from existing ones. Example is below which is showing that I'm trying to just assign the same value to disk to dynamic object osDisk but it fails with InvalidQuery. What am I doing wrong?
where type =~ 'Microsoft.Compute/virtualmachines'
| extend disk = properties.storageProfile.osDisk
| extend osDisk = dynamic({"osdisk" : properties.storageProfile.osDisk})
|project disk, osDisk
Error
Please provide below info when asking for support: timestamp = 2019-07-20T01:55:46.6283092Z, correlationId = 297ad2ed-81f2-49b3-86b2-5f38e2394923. (Code: BadRequest) Query is invalid. Please refer to the documentation for the Azure Resource Graph service and fix the error before retrying. (Code: InvalidQuery)
Removing dynamic line option returns results properly

try using pack(): https://learn.microsoft.com/en-us/azure/kusto/query/packfunction
print disk = "disk_value", properties = dynamic({"storageProfile":{"osDisk":"osDisk_value"}})
| project disk, osDisk = pack("osDisk", properties.storageProfile.osDisk)

Related

Customize AppInsights dashboard - show log query

We have an Office COM add-in and we want to capture telemetry using AppInsights. We're currently analyzing the feasibility.
The first item we've tried was to capture the host version.
Our prototype sends a JSON object to AppInsights. The JSON looks like
{
"Office":{"Version":"365","Build":"13530.201011"}​}
}
and it is sent to AppInsights as a trace (one single call to telemetryClient.TrackTrace );
This way we'll be able to easily add more information as needed about the host application, for example whether it is 32 bit or 64 bit.
Analyzing data is also simple, the following Kusto query easily extracts what we need:
traces
| extend environment=parse_json( message )
| extend officeVersion = tostring( environment.Office.Version )
| summarize cnt = cnt() by officeVersion
| project officeVersion, cnt
We want to pin this Kusto query to the AppInsights dashboard.
How can we do this?
Please try the "pin to dashboard" button in the logs query screen. Note that you should run the query first, then the button will be enabled:

Xamarin Android SQLite data reading gives me the "SQLite.SQLiteException: no such table: tblLidhja" exception

I am trying to build an Android Xamarin Application. I have created an SQLite database named regjistri.sqlite . The database it's not corrupted, I can open it via SQLite browser on my PC. This database I have save it to my android Documents folder.
I am trying to access it like below via sqlite-net-pcl library.
string dbPath = Path.Combine(System.Environment.GetFolderPath(System.Environment.SpecialFolder.MyDocuments), "regjistri.sqlite");
var db = new SQLiteConnection(dbPath);
var data = db.Query<object>("SELECT COUNT(*) FROM tblLidhja ");
Also I have gice the application the reading and writing permissions. When I debug it via USB cable, it give me the "SQLite.SQLiteException: no such table: tblLidhja" exception. Can someone help me whats may be wrong, or what should I change to read some data?
Before getting data , first need to check whether this table is exist in data base .You can refer to this document to check .
By the way ,having a try with this way to get count from database:
var db = new SQLiteConnection("DbPath");
var stocksStartingWithA = db.Query<TableName>("SELECT * FROM DbNmae ");
int count = stocksStartingWithA.Count; //get count from table
Here is similar discussion with java code, but it also can be refer in xamrin project.

querying across application insights resource with REST API

I have data which are stored in 3 different application insights resources, thanks to query across resource feature added last year (https://azure.microsoft.com/en-us/blog/query-across-resources/) was possible to query those 3 application insights at once with app identifier.
I'm trying to execute this query through app insights REST API : https://dev.applicationinsights.io (app insights REST API) for a very basic need from a static HTML page (no backend)
but without luck
I do suspect that app identifier isn't supported, it is actually the case ? any workaround for my use case (no backend).
Here is an example with the query in the body. My queries are quite complex and have a lot of let statements and therefore passing the query in the body is easier. There are some PowerShell quirks in the example below, but I'll update with a C# example tomorrow.
The let statement in the example below is pretty pointless, it's mostly there to show that you can do complex queries with let expressions etc.
AppId is the Application Insights resource ID - and NOT the instrumentation key. The API key is just a long string and you can create up to 10 of them AFAIK.
You will find both the id and keys under API Access (I've added a screenshot as it's easy to get them confused). When you use the app() function use the app id.
$app1Id = "GUID"
$app2Id = "GUID"
$app1Key = "string"
$app2Key = "string"
# EXAMPLE: "X-Api-Key" = "key1:GUID1,key2:GUID2"
$headers = #{ "X-Api-Key" = "${app1Key}:$app1Id,${app2Key}:$app2Id"; "Content-Type" = "application/json" }
# EXAMPLE: "query" = "union app('GUID1').something, app('GUID2').something | limit 5"
$query = #{"query" = "let days=1d;union app('$app1Id').exceptions,app('$app2Id').exceptions | where timestamp > ago(days)"}
$body = ConvertTo-Json $query | % { [regex]::Unescape($_) }
$result = Invoke-RestMethod "https://api.applicationinsights.io/v1/apps/$app1Id/query" -H $headers -Body $body -Method POST
The query above will return all the exceptions for the two Application insights resources for the last day. You can do a query across 10 resources at the time of writing, 200 requests per 30 seconds or a max of 86,400 requests per day (UTC). Other limits apply if you use ADD.
NOTE: the extra {} in the header is a PowerShell quirk in regards to variables and the use of the colon char, and as you can see in the example you should not bracket the keys in the header :)
Checked with the dev team that owns that service:
You should be able to put the api key in as apiKey1:appId1,apiKey2:appId2 in the api key box and this should work.
the [object ProgressEvent] response is a bug in the explorer that should have really showed you an error.
And as a workaround, you could always do the queries inside the azure portal itself in workbooks for any of the AI resources, or hypothetically also from the analytics portal for any of the AI resources, and those wouldn't require the API key at all, if all you needed was to see the data.

Java API to query CommonCrawl to populate Digital Object Identifier (DOI) Database

I am attempting to create a database of Digital Object Identifier (DOI) found on the internet.
By manually searching the CommonCrawl Index Server manually I have obtained some promising results.
However I wish to develop a programmatic solution.
This may result in my process only requiring to read the index files and not the underlying WARC data files.
The manual steps I wish to automate are these:-
1). for each CommonCrawl Currently available index collection(s):
2). I search ... "Search a url in this collection: (Wildcards -- Prefix: http://example.com/* Domain: *.example.com) " e.g. link.springer.com/*
3). this returns almost 6MB of json data that contains approx 22K unique DOIs.
How can I browse all available CommonCrawl indexes instead of searching for specific URLs?
From reading the API documentation for CommonCrawl I cannot see how I can browse all the indexes to extract all DOIs for all domains.
UPDATE
I found this example java code https://github.com/Smerity/cc-warc-examples/blob/master/src/org/commoncrawl/examples/S3ReaderTest.java
that shows how to access a common crawl dataset.
However when I run it I receive this exception
"main" org.jets3t.service.S3ServiceException: Service Error Message. -- ResponseCode: 404, ResponseStatus: Not Found, XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>common-crawl/crawl-data/CC-MAIN-2016-26/segments/1466783399106.96/warc/CC-MAIN-20160624154959-00160-ip-10-164-35-72.ec2.internal.warc.gz</Key><RequestId>1FEFC14E80D871DE</RequestId><HostId>yfmhUAwkdNeGpYPWZHakSyb5rdtrlSMjuT5tVW/Pfu440jvufLuuTBPC25vIPDr4Cd5x4ruSCHQ=</HostId></Error>
In fact every file I try to read results in the same error. Why is that?
what is the correct common crawl uri's for their datasets?
The data set location has changed since more than one year, see announcement. However, many examples and libraries still contain the old pointers. You can access the index files for all crawls back to 2013 on s3://commoncrawl/cc-index/collections/CC-MAIN-YYYY-WW/indexes/cdx-00xxx.gz - replace YYYY-WW with year and week of the crawle and expand xxx to 000-299 to get all 300 index parts. New crawl data is announced on the Common Crawl group, or read more about how to access the data.
To get the example code to work replace lines 24 and 25 with:
String fn = "crawl-data/CC-MAIN-2013-48/segments/1386163035819/warc/CC-MAIN-20131204131715-00000-ip-10-33-133-15.ec2.internal.warc.gz";
S3Object f = s3s.getObject("commoncrawl", fn, null, null, null, null, null, null);
Also note that the commoncrawl group have an updated example.

StructureGroup Details using the Content Delivery/Broker API

I am trying to get all the structure groups published in a given publication using the PublicationID. I am expecting to get the structure groups with StructureGroupCriteria by passing the Root Structure Group TCM ID but getting page ids (I am expecting SGs).
Now I am trying to loop through the list and get details of each structuregroup. I did not find any API (.net) to get these details and also the API is returning only Pages.
What I have done and working so far using StructureGroupCriteria, returns list of Page IDs instead of SG IDs
PublicationCriteria pubCriteria = new PublicationCriteria(pubID);
// Root StructureGroup TCM ID -- tcm:45-3-4
StructureGroupCriteria sgCriteria = new StructureGroupCriteria("tcm:45-3-4", true);
Criteria allSGsInPub = CriteriaFactory.And(pubCriteria, sgCriteria);
Query allSGs = new Query(allSGsInPub);
string[] sgInfo = allSGs.ExecuteQuery();
Response.Write("Total : " + sgInfo.Length);
foreach (string sgid in sgInfo ) {
// HOW DO I get the Structure Group Details here
//TCMURI sgURI = new TCMURI(sgid);
}
Q # 1 : How to get the all the structuregroups and individual structure group details? (May be something simple, I am not able to find right API).
Q # 2 : How can I get all the structuregroups using ItemTypeCriteria sgCriteria = new ItemTypeCriteria(4); // 4 is SG Item Type .
When I tried this option, the query worked successfully but no results returned. Is this the expected behavior and should we always use StructureGroupCriteria instead of ItemTypeCriteria?
The reason for this approach, I want to avoid using the Root StructureGroup ID which is required with the above code. But at the moment, none of the approaches returning StructureGroup information and I always get Page Information.
Tridion Version: 2011 SP1, .net API.
Note: When I publish I am checking the publish SG info checkbox and published successfully. On Broker DB side, I can see the information on the taxnonomy table as well.
I was playing with Odata service and accidentally I found that I can get all my structure group information from Odata web service.
/cd_webservice/odata.svc/StructureGroups?$filter=PublicationId%20eq%2045
Also, the results are returning child structure groups with a depth parameter.
Just to clarify , using Broker API it is not feasible to get the structure groups (my original question). However, the workaround solution is to use OData Service to get the Structure Groups.
I don't think you will get Structure Groups returned by the Query object.
According to the documentation, when you publish Structure Group information the Structure Group hierarchy is published to the Content Delivery side where it is stored as a taxonomy.
Have you tried using the Taxonomy APIs to get the information you need?

Resources