I have a BOSUN server running which is configured to operate on OPENTSDB. I am sending data to OPENTSDB using SCOLLECTOR every minute. What I am looking for is a way to get all possible Tag_Values for a given Tag_Key and Metric_Name which is present in OPENTSDB.
For Example Suppose I have,
metric: data.queue.capacity
tags: queue={queue1,queue2,queue3...}
I am storing capacity for a Number of Queues.This data is there in OPENTSDB.
I can query BOSUN for values of metric=data.queue.capacity using query:
q("sum:data.queue.capacity{queue=*}{}","start_time","end_time")
which gives output:
{queue:queue1} :{"timestamp1":"value1","timestamp2":"value2",...}
{queue:queue2} :{"timestamp1":"value1","timestamp2":"value2",...}
.
.
.
The result returned is grouped by different Queue values.
It is hard to use above response data to get all possible Queue names because:
It only returns data between start time and end time, So if a Queue name say
queue_not does not have a metric point between start_time and end_time, it will not
come in the response.
This gives a lot of data in response, I can aggregate but I don't think this
is the best way to get Tag values.
What I want is an API where I can give a Metric_Name and Tag_Key and i should get a JSON data having all possible Tag_Values present in OPENTSDB for that Metric_Name,Tag_Key.
After Searching I have found partial solutions:
BOSUN have API to get all Tag_values, But for only those metrics which are relayed through BOSUN.
API: /api/tagv/{tagk}/{metric}
This API accept's Tag_key and Metric_Name and for this combination gives all values of Tag_Values.
eg: http://bosun-host:port/api/tagv/{tagk}/{metric}
response:
[
"tag_value1",
"tag_value2"
]
Still I don't have a solution to get Tag_values for metric which are not relayed through BOSUN but directly to OPENTSDB.
The solution may not involve BOSUN it could be OPENTSDB API too.
As far as Bosun as concerned, relaying the datapoints to Bosun for that tag metadata is the standard way. You can use tsdbrelay so metrics get copied to Bosun but don't flow through Bosun (this way if Bosun is restarted or is down, metric flow is not interrupted).
With OpenTSDB itself you could look into OpenTSDB Metadata - but Bosun't doesn't use this feature of OpenTSDB.
Related
Say I need to find 95th and 99th latency of an API with endpoint: "/api/v1/merchant/{merchantId}/logo".
In Kibana I tried searching: "/api/v1/merchant" AND "/logo". But, it returned multiple results, one for each merchantId.
How to search such that single URL is returned which combines the metrics for all merchantIds to give the output?
What you are trying to do is to filter for "api v1 merchant * logo" (mixing wildcards and string query to preserve the order) which is not possible at the moment. But if the filter "api v1 merchant" and logo is correct (aka you don't have APIs like /api/v1/merchant/logo without the id) then there is a solution.
You need a new visualization: just filter for the query and not aggregate the results by API.keyword.
I want to retrieve highest 10 scores from firebase . When i search stackoverflow i found some answers on here Firebase retrieve highest 100 score but i can not imagine how to implement them with REST API . My code is like this. I send request to end point and get response .
RestClient.Get<User>(databaseURL + getScoreText.text + ".json").Then(response =>
{
user = response;
UpdateScore();
});
Also for 2nd way, i can get all users' info by some json parser package and parse them and get the highest scores . But if my database gets bigger , i think i ll have problems . I need a way like "orderby". Do you know any way to implement "orderby" method to my restclient code ?
Reading the firebase docs:
Filtered data is returned unordered: When using the REST API, the filtered results are returned in an undefined order since JSON interpreters don't enforce any ordering. If the order of your data is important you should sort the results in your application after they are returned from Firebase.
It seems like scaling is a real limitation of the realtime database.
Firestore can do this pretty easily if you want to switch.
If you're sticking with realtime DB, I would save a separate database-reference that only ever contains the top 10 scores.
So when a player gets a new score, they would only upload it to this reference if it's higher than the 10th top score. This would mean you would only have a DB reference containing 10 or less top scores, and scaling wouldn't be a problem. I'm not very familiar with realtime DB, but I'm sure you could write a security to rule to enforce this in a secure, server authoritative manner, at the very least a cloud function.
I need to get the Marketo Leads who have changes on "progressionStatus" field (inside membership) with the API.
I can get all the leads related to a Program (with Get Leads by ProgramID API) without issues, but my need is to get those Leads with changes on "progressionStatus" column.
I was thinking to use the CreatedAt / UpdatedAt fields of the Program, so then, get all the leads related to those programs. But I didn't get the accurate results that I want.
Also, I tried to use the GET Lead changes API and use "fields" parameter to "progressionstatus" but that field don't exist.
It is possible to resolve this?
Thanks in advance.
You can get the list of Leads with progression status change by querying against the Get Lead Activities endpoint.
The Get Lead Changes endpoint could sound as a good candidate, but that endpoint only returns changes on the lead fields. Progression status change is not stored on the lead directly, so at the end that won't work. On the other hand the Get Leads by ProgramId endpoint returns –amongst others– the actual value of progressionStatus (program status of the lead in the parent program) but not the “change” itself, so you cannot process the resultset based on that.
The good news is that the progression status change is an activity type and luckily we have the above mentioned Get Lead Activities endpoint (which is also mentioned as the Query in the API docs) available to query just that. This endpoint also allows for filtering by activityTypeIds to narrow down the resultset to a single activity type.
Basically you have to call the GET /rest/v1/activities.json enpoint and pass the values of activityTypeIds and nextPageToken as query parameters (next to the access token obviously). So, first you need to get the internal Id of the activity type called “Change Status in Progression”. You can do that by querying the GET /rest/v1/activities/types.json endpoint and look for a record with that name. (I don't know if this Id changes from instance to instance, but in ours it is the #104). Also, to obtain a nextPageToken you have to make a call to GET /rest/v1/activities/pagingtoken.json as well, where you have to specify the earliest datetime to retrieve activities from. See more about Paging Tokens.
Once you have all of these bits at hand, you can make your request like that:
GET https://<INSTANCE_ID>.mktorest.com/rest/v1/activities.json?activityTypeIds=<TYPE_ID>&nextPageToken=<NEXTPAGE_TOKEN>&access_token=<ACCESS_TOKEN>
The result it gives is an array with items like below, which is easy to process further.
{
"id":712630,
"marketoGUID":"712630",
"leadId":824864,
"activityDate":"2017-12-01T08:51:13Z",
"activityTypeId":104,
"primaryAttributeValueId":1104,
"primaryAttributeValue":"PROGRAM_NAME",
"attributes":[
{"name":"Acquired By","value":true},
{"name":"New Status ID","value":33},
{"name":"Old Status ID","value":32},
{"name":"Reason","value":"Filled out form"},
{"name":"Success","value":false},
{"name":"New Status","value":"Filled-out Form"},
{"name":"Old Status","value":"Not in Program"}
]
}
Knowing the leadIds in question, you can make yet another request to fetch the actual lead records.
We want to pull data from Application Insights using the REST APIs (see https://dev.applicationinsights.io/documentation/overview). Specifically, we need to retrieve data with "GET /query". According to https://learn.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference, there is an implicit limit on the number of rows that can be returned to the client; and there is also a timeout limit.
When we pull data directly from a Kusto cluster/table, we set 'OptionNoRequestTimeout' and 'OptionNoTruncation' properties to TRUE in the Kusto ClientRequestProperties, which is provided in the Kusto client interface.
I am wondering how I can achieve the same when pulling data with AppInsights "GET /query" REST API to lift the limitation on timeout and number of rows to return?
The AppInsight Analytics Reference briefly mentions that "To lift the row count limit, use the notruncation client request option." w/o explaining how. How shall I set these Kusto request properties on the HTTP request URL, which is sent to the target AppInsights application with a HttpClient?
You cannot currently override the values for row count (500,000) or total size (64,000,000 bytes) through the REST API.
you can, however, override the timeout, by adding a special header:
Prefer: wait=[seconds]
like
Prefer: wait=600
for 10 minutes, which is the max.
The notruncation part of your question is something you use with set in your actual query, like
set notruncation;
MyTable | take 500000
I tried to automate test to validate GA events.
My approach is :-
List item use google analytics real time reporting api.
Before the test ends i will hit this api and collect the last 30 mins data
This data will be a huge chunk of formattedJson string
and in this string i will search my GA events which was supposed to push.
This approach seems to be in-efficient.
My issue is to find the analytics data which corresponds to test user.
Each user has unique user id, hence, i am trying for making the request such that api returns me the filtered data based on some custom dimension "custom:user_id='user_unique_id'" .
Is it possible to get all data having condition e.g 'custom:user_id="XYZ"'.
Please advise, how to get all ga events data for a specific event label / custom dimension ? Also, does it support dimensionFilterClauses like reporting api v4 ?
We can do it by filtering.
e.g rt:eventCategory==ProductPage
earlier i was using quote, rt:eventCategory=='ProductPage' which wasn't supported.