opentsdb query Filters literal_or - opentsdb

JSON
{
"start": "2016/04/12 04:16:00",
"end": "2016/04/12 16:16:00",
"queries": [
{
"aggregator": "avg",
"downsample": "1h-avg",
"filters":
[
{
"filter": "1000009727000|1000009727018|1000009727028|10000097270404",
"groupBy": false,
"tagk": "neId",
"type": "literal_or"
}
],
"metric": "150214"
}
]
}
meet problem
“No such name for 'tagv' 10000097270404”
others tagv exist
how to solve this problem

Found the answer in https://groups.google.com/forum/#!topic/opentsdb/_47BHtew1sQ :
The default is to error out but users can set "tsd.query.skip_unresolved_tagvs" to true and it will pass the query through.
And it worked with OpenTSDB 2.3 for me.

Related

Google analytics reporting dimensions & metrics are incompatible

We have custom dimension define in Google Analytics Data API v1Beta for extracting data from Google Analytics GA4 account.
This worked fine till yesterday.
Error:
Please remove customEvent:institution_id to make the request compatible. The request's dimensions & metrics are incompatible. To learn more, see https://ga-dev-tools.web.app/ga4/dimensions-metrics-explorer/
POST - https://analyticsdata.googleapis.com/v1beta/properties/{propertyId}:runReport
Body
{
"dateRanges": [
{
"startDate": "2022-08-29",
"endDate": "2022-12-07"
}
],
"dimensions": [
{
"name": "customEvent:institution_id"
},
{
"name": "pagePathPlusQueryString"
}
],
"dimensionFilter": {
"andGroup": {
"expressions": [
{
"filter": {
"fieldName": "customEvent:institution_id",
"inListFilter": {
"values": [
"47339486-a1e5-47be-abce-e4270af23rte"
]
}
}
},
{
"filter": {
"fieldName": "pagePathPlusQueryString",
"stringFilter": {
"matchType": "PARTIAL_REGEXP",
"value": "/clip/.+",
"caseSensitive": false
}
}
}
]
}
},
"metrics": [
{
"name": "screenPageViews"
}
],
"metricAggregations": [
"TOTAL"
],
"limit": "10000"
}
Dimensions that include the query string like pagePathPlusQueryString are only compatible with a limited set of dimensions & metrics.
This changed was announced 2022-09-13 Schema compatibility changes announcement.
It went live earlier this week. So the cause if the error is that the "customEvent:institution_id" dimension is not compatible with "pagePathPlusQueryString".

Searching Google Calendar extendedProperties does not return any match

I am using Google Calendar API v3 in to search for a specific property/value. I was able to successfully create the extended property as you can see here:
{
"kind": "calendar#events",
"etag": "\"p33cf16d5ousve0g\"",
"summary": "Ryan's Personal Camptivities",
"updated": "2022-04-29T15:11:23.687Z",
"timeZone": "America/Los_Angeles",
"accessRole": "owner",
"defaultReminders": [
{
"method": "popup",
"minutes": 10
}
],
"nextSyncToken": "CNjwmaXHufcCENjwmaXHufcCGAUg67Ly0wE=",
"items": [
{
"extendedProperties": {
"private": {
"OutlookEventGO": "AAMkADNkODYzZGQ5LWY5ZGItNGFjZi1iMWMzLTI3MmY0YmIxMTcxNQBGAAAAAAC5E322QQ7ORLEmN7zGNq0zBwAUl432Bg61QItQhHQwcosuAAAAAAENAAAUl432Bg61QItQhHQwcosuAAF6QI0zAAA="
}
]
}
but when I go to search for the extendedProperty, i get no response:
https://www.googleapis.com/calendar/v3/calendars/ryan%40camptivities.com/events?privateExtendedProperty=OutlookEventGO%3DAAMkADNkODYzZGQ5LWY5ZGItNGFjZi1iMWMzLTI3MmY0YmIxMTcxNQBGAAAAAAC5E322QQ7ORLEmN7zGNq0zBwAUl432Bg61QItQhHQwcosuAAAAAAENAAAUl432Bg61QItQhHQwcosuAAF6QI0zAAA%3D&
{
"kind": "calendar#events",
"etag": "\"p33cf16d5ousve0g\"",
"summary": "Ryan's Personal Camptivities",
"updated": "2022-04-29T15:11:23.687Z",
"timeZone": "America/Los_Angeles",
"accessRole": "owner",
"defaultReminders": [
{
"method": "popup",
"minutes": 10
}
],
"nextSyncToken": "CNjwmaXHufcCENjwmaXHufcCGAUg67Ly0wE=",
"items": []
}
Is this because of the encoding of the extendedProperty value? If so, how do I fix this? If not, what am I doing wrong?
Thanks in advance...
The problem is the = at the end of your privateExtendedProperty value
This seems to create issues when it comes to the url encoding of the property.
If you use a property value without an =, e.g.
AAMkADNkODYzZGQ5LWY5ZGItNGFjZi1iMWMzLTI3MmY0YmIxMTcxNQBGAAAAAAC5E322QQ7ORLEmN7zGNq0zBwAUl432Bg61QItQhHQwcosuAAAAAAENAAAUl432Bg61QItQhHQwcosuAAF6QI0zAAA
you can perfectly retrieve it with out request:
https://www.googleapis.com/calendar/v3/calendars/ryan%40camptivities.com/events?privateExtendedProperty=OutlookEventGO%3DAAMkADNkODYzZGQ5LWY5ZGItNGFjZi1iMWMzLTI3MmY0YmIxMTcxNQBGAAAAAAC5E322QQ7ORLEmN7zGNq0zBwAUl432Bg61QItQhHQwcosuAAAAAAENAAAUl432Bg61QItQhHQwcosuAAF6QI0zAAA
Your privateExtendedProperty seems to be encoded, so if the = is important for you to decode the value back, you will need to add it back programatically when require.

The language expression property '0' can't be evaluated, property name must be a string - ARM Template error while adding Key Vault access policy

I've been working on an issue and seem to be stuck, so asking on so in case anyone can help.
To describe the issue, I've got an existing Azure Key Vault setup, and wish to add a number of access policies to this resource group. It needs to be conditional as if the function name is "false" then that function should not be added to key vault access policy.
variable section:
"variables": {
"functionAccess": {
"value": [
{
"name": "[parameters('Function_1')]"
},
{
"name": "[parameters('Function_2')]"
},
{
"name": "[parameters('Function_3')]"
}
]
}
}
My Template :
{
"apiVersion": "2016-10-01",
"condition": "[not(equals(variables('functionAccess')[CopyIndex()].name, 'false'))]",
"copy": {
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "Serial",
"name": "accessPolicies"
},
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"accessPolicies": [
{
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[CopyIndex()].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[CopyIndex()].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
When I deploy my ARM template for the azure key vault I got this error message:
The language expression property '0' can't be evaluated, property name must be a string.
also tried below, but same error:
{
"apiVersion": "2018-02-14",
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"copy": [
{
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "serial",
"name": "accessPolicies",
"input": {
"condition": "[not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false'))]",
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[copyIndex('accessPolicies')].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
There are a few options for dealing with filtering an array for copy operation. I deploy my ARM templates from PowerShell scripts and use PowerShell to setup parameter values. When I need special logic handle different inputs for different environments, I let PowerShell handle it.
If you must handle the filtering in ARM and you have the option to input a CSV list of functions, then perhaps the following will work. You can then use the functionAccessArray to iterate over in the copy operation.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"functionAccessCsv": "Function-0,Function-1,false,Function-4,false,Function-6,Function-7",
"functionAccessFiltered": "[replace(replace(variables('functionAccessCsv'), 'false', ''), ',,', ',')]",
"functionAccessArray": "[split(variables('functionAccessFiltered'), ',')]"
},
"resources": [
],
"outputs": {
"functionAccessCsvFiltered": {
"type": "string",
"value": "[variables('functionAccessFiltered')]"
},
"functionAccessArray": {
"type": "array",
"value": "[variables('functionAccessArray')]"
}
}
}
The result:
I just had the same issue. By using an array parameter with a default value instead of a variable, I got it to work.
"parameters": {
"functionAccess": {
"type": "array",
"defaultValue": [
"value1",
"value2",
"value3"
]
}
}

Request probleme with Google Cloud Datastore and Filter

I'm currently doing some tests on google datastore, but I'm having a problem with my queries.
If I believe in the documentation https://cloud.google.com/datastore/docs/concepts/queries we can realize a filter on several columns with the instruction EQUALS.
But when testing, I get an error from the API.
While searching on Datastore's github, I found this reference: https://github.com/GoogleCloudPlatform/google-cloud-dotnet/issues/304 which corresponds to my problem, except that for my case the query to the look good.
Here is the request sent:
{
{
"kind": [{
"name": "talk.message"
}],
"filter": {
"compositeFilter": {
"op": "AND",
"filters": [{
"propertyFilter": {
"property": {
"name": "Conversation"
},
"op": "EQUAL",
"value": {
"stringValue": "2f16c14f6939464ea687d316438ad4cb"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "LESS_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-15T10:43:31.474166300Z"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "GREATER_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-14T10:43:31.474175100Z"
}
}
}
]
}
}
}
}
And here is the answer from the API:
{Grpc.Core.RpcException: Status(
StatusCode=FailedPrecondition,
Detail="no matching index found. recommended index is:
- kind: talk.message
properties:
- name: Conversation
- name: CreatedOn"
)
According to the documentation, this should be good... but it's not !
What am I missing ?
Your query includes both an EQUALS (on Conversation) and a non-EQUALS filter (on CreatedOn), therefore you need a composite index to fulfil the query. So your query is valid, but it needs a composite index to be able to run the query.

copyIndex() inside a listKeys()

We're trying to deploy an ARM template which deploys a Stream Analytics job with n Event Hubs outputs depending on an input parameter.
Right now we're having success with all but the listKeys() function inside the outputs property copy loop function which gets each Event Hub's primary keys:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
We get the error:
17:44:31 - Error: Code=InvalidTemplate; Message=Deployment template
validation failed: 'The template resource
'tailor-router-axgf7t3gtspue' at line '129' and column '10' is not
valid: The template function 'copyIndex' is not expected at this
location. The function can only be used in a resource with copy
specified. Please see https://aka.ms/arm-copy for usage details..
Please see https://aka.ms/arm-template-expressions for usage
details.'.
However, if we change this to be a specific index:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[0].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
it works.
Is copyIndex('propertyName') inside a listKeys() a supported function?
If not, is there a workaround that would achieve the same effect?
Kind regards,
Nick
Stream Analytics job resource definition:
{
"apiVersion": "2016-03-01",
"type": "Microsoft.StreamAnalytics/StreamingJobs",
"name": "[variables('routerStreamAnalyticsName')]",
"location": "[variables('location')]",
"dependsOn": [ "clientsEventHubCopy" ],
"tags": {
"boundedContext": "[variables('boundedContextName')]"
},
"properties": {
"sku": {
"name": "[parameters('routerStreamAnalyticsSkuTier')]"
},
"outputErrorPolicy": "drop",
"eventsOutOfOrderPolicy": "adjust",
"eventsOutOfOrderMaxDelayInSeconds": 0,
"eventsLateArrivalMaxDelayInSeconds": 5,
"dataLocale": "en-US",
"compatibilityLevel": "1.0",
"inputs": [
{
"name": "input0",
"properties": {
"type": "stream",
"serialization": {
"type": "Avro"
},
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[parameters('input0EventHubNamespace')]",
"sharedAccessPolicyName": "[parameters('input0EventHubSharedAccessPolicyName')]",
"sharedAccessPolicyKey": "[parameters('input0EventHubSharedAccessPolicyKey')]",
"eventHubName": "[parameters('input0EventHubName')]"
}
}
}
}
],
"transformation": {
"name": "routing",
"properties": {
"streamingUnits": "[parameters('routerStreamAnalyticsSkuTier')]",
"query": "omitted"
}
},
"copy": [
{
"name": "outputs",
"count": "[length(parameters('clients'))]",
"input": {
"name": "[parameters('clients')[copyIndex('outputs')].id]",
"properties": {
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[variables('clientEventHubNamespace')]",
"sharedAccessPolicyName": "[variables('clientEventHubClientSharedAccessName')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]",
"eventHubName": "[parameters('clients')[copyIndex('outputs')].id]"
}
},
"serialization": {
"type": "Avro"
}
}
}
}
]
}
},
Thanks for reporting this and sorry for the inconvenience.
I just talked to the ARM team, we had an issue when copyindex was inside the index tags eg 'array[copyindex()]'. It should be fixed now.
Let us know how it goes.
Thanks,
JS - Azure Stream Analytics

Resources