We want to pull data from Application Insights using the REST APIs (see https://dev.applicationinsights.io/documentation/overview). Specifically, we need to retrieve data with "GET /query". According to https://learn.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference, there is an implicit limit on the number of rows that can be returned to the client; and there is also a timeout limit.
When we pull data directly from a Kusto cluster/table, we set 'OptionNoRequestTimeout' and 'OptionNoTruncation' properties to TRUE in the Kusto ClientRequestProperties, which is provided in the Kusto client interface.
I am wondering how I can achieve the same when pulling data with AppInsights "GET /query" REST API to lift the limitation on timeout and number of rows to return?
The AppInsight Analytics Reference briefly mentions that "To lift the row count limit, use the notruncation client request option." w/o explaining how. How shall I set these Kusto request properties on the HTTP request URL, which is sent to the target AppInsights application with a HttpClient?
You cannot currently override the values for row count (500,000) or total size (64,000,000 bytes) through the REST API.
you can, however, override the timeout, by adding a special header:
Prefer: wait=[seconds]
like
Prefer: wait=600
for 10 minutes, which is the max.
The notruncation part of your question is something you use with set in your actual query, like
set notruncation;
MyTable | take 500000
Related
I have a set of monitized APIs on Apigee Edge , While calling the revenue report API
{{apigee-baseURL}}/v1/mint/organizations/{{org}}/revenue-reports
(billing report also), I am able to configure returning custom transaction attribute via specifying the transactionCustomAttributes propoert (normal) ... however I can't control to remove the Provider Status parameter from the response (It always returns), thus If I want to aggregate all charges regardless to status values, I need manual (or developer) work, the question is : Is there any way to remove the Provider Status parameter from the report (and thus aggregate the relevant charge rate values) ?
I checked the documentation for this and found nothing
I am trying to get a full list of every group in Azure AD. I am currently able to get 999 records with the following uri:
https://graph.microsoft.com/v1.0/groups?$top=999
According to the documentation from Microsoft there are only a couple OData query parameters available, none of which appear to be able to navigate to the next page. It also states the maximum page size is 999. I have tried using the $skip parameter to skip a certain number of records, but it is not supported:
{"error":{"code":"Request_BadRequest","message":"'$skip' is not supported by the service.",...
Is there any way to get a full list of all AAD groups? We have several thousand that I would need to get.
Some queries against Microsoft Graph return multiple pages of data either due to server-side paging or due to the use of the $top query parameter to specifically limit the page size in a request. When more than one query request is required to retrieve all the results, Microsoft Graph returns an #odata.nextLink property in the response that contains a URL to the next page of results.
For example, the following URL requests all the users in an organization with a page size of 5, specified with the $top query parameter:
https://graph.microsoft.com/v1.0/groups?$top=5
If the result contains more results, Microsoft Graph will return an #odata.nextLink property similar to the following along with the first page of results:
You can retrieve the next page of results by sending the URL value of the #odata.nextLink
ref doc - https://learn.microsoft.com/en-us/graph/paging
With $top, you can customize the result size within the range of 1 and 1000. Because of your question, I guess that 1000 is exclusive, so the range goes from 1 to 999 (inclusive). Read more about the query parameter $top here. I got the range information from List messages.
The response you get from List groups seems to not contain #odata.nextLink which you would normally expect in such a case, so GET https://graph.microsoft.com/v1.0/groups does not support pagination. That would also explain why you get an error if you try to use $skip. You can read more about $skip here.
In order to get the full list of all groups, I would stop using the query parameter $top.
I want to retrieve highest 10 scores from firebase . When i search stackoverflow i found some answers on here Firebase retrieve highest 100 score but i can not imagine how to implement them with REST API . My code is like this. I send request to end point and get response .
RestClient.Get<User>(databaseURL + getScoreText.text + ".json").Then(response =>
{
user = response;
UpdateScore();
});
Also for 2nd way, i can get all users' info by some json parser package and parse them and get the highest scores . But if my database gets bigger , i think i ll have problems . I need a way like "orderby". Do you know any way to implement "orderby" method to my restclient code ?
Reading the firebase docs:
Filtered data is returned unordered: When using the REST API, the filtered results are returned in an undefined order since JSON interpreters don't enforce any ordering. If the order of your data is important you should sort the results in your application after they are returned from Firebase.
It seems like scaling is a real limitation of the realtime database.
Firestore can do this pretty easily if you want to switch.
If you're sticking with realtime DB, I would save a separate database-reference that only ever contains the top 10 scores.
So when a player gets a new score, they would only upload it to this reference if it's higher than the 10th top score. This would mean you would only have a DB reference containing 10 or less top scores, and scaling wouldn't be a problem. I'm not very familiar with realtime DB, but I'm sure you could write a security to rule to enforce this in a secure, server authoritative manner, at the very least a cloud function.
I have setup an api that will query our Cosmos db and return the JSON results back to the front end app. There is a user defined limit on the number of results. If the number of results exceed the limit then I pass back the token to the front end and they can call for the next group of rows. The issue is I would like to provide a count of the Total Number of Matches back to the application. I have looked at the query statistics but don't see where there is a total count.
On the call to CreateDocumentQuery, i'm setting MaxItemCount to the limit, and RequestContinuation either null or the continuationToken. Looking at QueryMetrics I found RetrievedDocumentCount, but that does not seem to have the correct value.
Thanks,
J
x-ms-max-item-count request header controls how many documents should be returned to user.
Default value is 100
if your query returns 150 documents, your request will return first 100 documents and it will return a continuation token in response header(x-ms-continuation). If there is a token, you need to send another request with the given token to get the rest of the data.
SDK should be doing that for you automatically. Can you share some of your code. I might have a better answer then.
You can check out my post about this too.
https://h-savran.blogspot.com/2019/04/introduction-to-continuation-tokens-in.html
I have a BOSUN server running which is configured to operate on OPENTSDB. I am sending data to OPENTSDB using SCOLLECTOR every minute. What I am looking for is a way to get all possible Tag_Values for a given Tag_Key and Metric_Name which is present in OPENTSDB.
For Example Suppose I have,
metric: data.queue.capacity
tags: queue={queue1,queue2,queue3...}
I am storing capacity for a Number of Queues.This data is there in OPENTSDB.
I can query BOSUN for values of metric=data.queue.capacity using query:
q("sum:data.queue.capacity{queue=*}{}","start_time","end_time")
which gives output:
{queue:queue1} :{"timestamp1":"value1","timestamp2":"value2",...}
{queue:queue2} :{"timestamp1":"value1","timestamp2":"value2",...}
.
.
.
The result returned is grouped by different Queue values.
It is hard to use above response data to get all possible Queue names because:
It only returns data between start time and end time, So if a Queue name say
queue_not does not have a metric point between start_time and end_time, it will not
come in the response.
This gives a lot of data in response, I can aggregate but I don't think this
is the best way to get Tag values.
What I want is an API where I can give a Metric_Name and Tag_Key and i should get a JSON data having all possible Tag_Values present in OPENTSDB for that Metric_Name,Tag_Key.
After Searching I have found partial solutions:
BOSUN have API to get all Tag_values, But for only those metrics which are relayed through BOSUN.
API: /api/tagv/{tagk}/{metric}
This API accept's Tag_key and Metric_Name and for this combination gives all values of Tag_Values.
eg: http://bosun-host:port/api/tagv/{tagk}/{metric}
response:
[
"tag_value1",
"tag_value2"
]
Still I don't have a solution to get Tag_values for metric which are not relayed through BOSUN but directly to OPENTSDB.
The solution may not involve BOSUN it could be OPENTSDB API too.
As far as Bosun as concerned, relaying the datapoints to Bosun for that tag metadata is the standard way. You can use tsdbrelay so metrics get copied to Bosun but don't flow through Bosun (this way if Bosun is restarted or is down, metric flow is not interrupted).
With OpenTSDB itself you could look into OpenTSDB Metadata - but Bosun't doesn't use this feature of OpenTSDB.