Problem in segment creation in Apache UNOMI - apache-karaf

I was exploring Unomi from past 2 weeks, during this process I created user session and events and created few segment on user properties and event type but I got stuck when I wanted to create segment on event properties.
To start with this is the event that we are sending to unomi -
{
"source":{
"itemType":"pageView",
"scope":"the-coolest-website",
"itemId":"pageView"
},
"events":[
{
"eventType":"view",
"scope":"the-coolest-website",
"source":{
"itemType":"pageView",
"scope":"the-coolest-website",
"itemId":"pageView"
},
"properties":{
"myProperty":"myValue"
}
}
]
}
What we are trying to do is to create a segment which targets events which have this myProperty:myValue key-value pair.
The segment which I created is as follows:
{
"metadata": {
"id": "myPropertySegment",
"name": "myPropertySegment",
"scope":"the-coolest-website",
"description": "The user has this segment if it has acessed our products Website.",
"readOnly": true
},
"condition": {
"parameterValues": {
"subConditions": [
{
"parameterValues": {
"propertyName": "myProperty",
"comparisonOperator": "equals",
"propertyValue": "myValue"
},
"type": "eventPropertyCondition"
}
],
"operator": "and"
},
"type": "booleanCondition"
}
}
In above Event properties segment users are not being qualified having event property as "myProperty":"myValue".
Created another segment on event Type :
{
"metadata": {
"id": "eventCondition",
"name": "eventCondition",
"scope": "the-coolest-website",
"description": "The user has this segment if it has acessed our products Website.",
"readOnly": true
},
"condition": {
"parameterValues": {
"subConditions": [
{
"parameterValues": {
"propertyName": "eventType",
"comparisonOperator": "equals",
"propertyValue": "myAwesomeUserEvent"
},
"type": "eventTypeCondition"
}
],
"operator": "and"
},
"type": "booleanCondition"
}
}
In above Event type segment users are being qualified having eventType as myAwesomeUserEvent.
Please help me while creating segment on event properties and let know if I am doing any mistake while creating segment.

Related

Put Azure Key Vault value in parameter array

I am trying to deploy a App service webapp via ARM template and need to put a secret from a key vault into an app setting (env variable).
I have always simply used an array of values from a parameters file to populate these app settings, but now I am struggling to get a keyvault value into that array. Something like shown below in an ARM parameter file.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"someStringParam": {
"value": "stringLiteralValueHere"
},
"envVars": {
"value": [
{
"name": "envVarKeyName",
"value": "stringLiteralValueHere"
},
{
"name": "KVsecret1",
"value": ##KEY VAULT SECRET HERE##
}
]
}
}
}
I have tried using a reference to the keyvault for the value but that errors on deployment.
{
"name": "KVsecret1",
"reference": {
"keyVault": {
"id": "/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.KeyVault/vaults/<vault_name>"
},
"secretName": "secret1"
}
}
I have also tried using a parameter inside of the parameter file, but that just used the literal string for the value.
"parameters": {
"KVsecret1": {
"reference": {
"keyVault": {
"id": "/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.KeyVault/vaults/<vault_name>"
},
"secretName": "KVsecret1"
}
},
"envVars": {
"value": [
{
"name": "envVarKeyName",
"value": "stringLiteralValueHere"
},
{
"name": "KVsecret1",
"value": "[parameters('KVsecret1')]"
}
]
}
}
Is this possible??
EDIT: Adding some detail here.
I am also trying to shoe horn a reference to another resource to get put the app insights instrumentation key into an app setting. Below is what I would like to do, but the copy function needs to use the name of the property and that is dynamic in this case as it changes with the each member of the array from the parameter file.
{
"type": "Microsoft.Web/sites/config",
"apiVersion": "2022-03-01",
"name": "[concat(parameters('backEndwebAppName'),'/appsettings')]",
"kind": "string",
"properties": {
"APPINSIGHTS_INSTRUMENTATIONKEY": "[reference(concat('microsoft.insights/components/',parameters('appInsightsName')),'2020-02-02').InstrumentationKey]",
"secret1FromKeyvault": "[parameters('secret1FromKeyvault')]",
"copy": [
{
"name": "envVarsFromParams",
"count": "[length(parameters('backEndEnvVariables'))]",
"input": {
"name": "[parameters('backEndEnvVariables')[copyIndex('envVarsFromParams').name]]",
"value": "[parameters('backEndEnvVariables')[copyIndex('envVarsFromParams').value]]"
}
}
]
},
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('backEndwebAppName'))]"
]
},
This isn't possible today within the param file, but in your scenario (if it's as simple as your OP example) you can just union the two in your template. So in your parameter file, you have 2 params kvSecret (the reference) and envVars (all your other env vars) and then in the template use:
"variables": {
"keySecretObj": {
"name": "kvSecret",
"value": "[parameters('kvSecret')]"
},
"envVarsFinal": "[union(parameters(variables('kvSecretObj`), parameters(`envVars`))]"
That help?

Not able to filter out required property in Azure TSI Gen1 Get Events API response

I am using the below request body to fetch only the required property values.
"searchSpan": {
"from": {
"dateTime": "2021-11-20T00:00:00.000Z"
},
"to": {
"dateTime": "2021-11-20T23:00:00.000Z"
}
},
"predicateString": "[Params.Name] = 'power'",
"take": 100
}
}
The URL is like below:
https://12345678a-bcde-3e91-blah-2292933292aa.env.timeseries.azure.com/events?api-version=2016-12-12
Despite specifying the required property the response returns all properties as if it has not seen the predicate string. What might I be doing wrong?
{
"warnings": [],
"events": [
{
"schema": {
"rid": 0,
"$esn": "my-event-hub",
"properties": [
{
"name": "mytimestamp",
"type": "DateTime"
},
{
"name": "Params.Name",
"type": "String"
},
{
"name": "Params.Value",
"type": "Double"
}
]
},
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"energy",
60
]
},
{
"schemaRid": 0,
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"power",
10
]
},
{
"schemaRid": 0,
"$ts": "2021-11-20T10:01:50Z",
"values": [
"2021-11-20T10:01:50Z",
"strength",
200
]
},
]
}
Edit
I'm getting "Properties count error" in the TSI overview page. This might quite be the root cause but I don't know for sure
"For Time Series Insights environment ABC: You have used all 641/600 properties in your environment".

Search for available items for booking

I am working on a online booking system of items.
I am using mongo to store booking and item details
Item
{
id: "3",
"name": "",
"description": "",
"extra": [{}]
}
Booking
{
"id": "",
"itemId":""
"startDate": millis,
"endDate": millis,
"status": "",
"userId": ""
}
I have to implement search b/w dates. The search should return only available items for the specified period. How can I build a scalable search for this? I am planning to use elastic also for search. Any suggestion related to new technology also welcome.
I'd suggest making the booking the base object and putting the item info inside it. That is to say:
Set up mapping:
PUT bookings
{
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"item": {
"properties": {
"id": {
"type": "keyword"
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"description": {
"type": "text"
},
"extra": {
"type": "nested"
}
}
},
"startDate": {
"type": "date",
"format": "epoch_millis"
},
"endDate": {
"type": "date",
"format": "epoch_millis"
},
"status": {
"type": "keyword"
},
"userId": {
"type": "keyword"
}
}
}
}
Ingest the simplest booking
POST bookings/_doc
{
"item": {
"id": "987"
},
"startDate": 1587110540025,
"endDate": 1587220730025
}
Restricting the *Date fields and only returning the corresponding item:
GET bookings/_search
{
"_source": "item",
"query": {
"bool": {
"must": [
{
"range": {
"startDate": {
"gte": "17/04/2020",
"format": "dd/MM/yyyy"
}
}
},
{
"range": {
"endDate": {
"lte": "18/04/2020",
"format": "dd/MM/yyyy"
}
}
}
]
}
}
}
Note that although our date fields are defined as epoch_millis, we can still query using human-readable date strings, provided we specify the format. You can of course use milliseconds if you prefer.
While indexing the items to Elasticsearch you can check bookings. Think that, you are indexing items and you get the item from Mongo. Also, you can get the bookings for this item and you can add a field like bookingCount inside the item document of Elasticsearch. While searching you can use bookingCount field to search without booking items.
In generally, the indexing is async operations. You can use queue. So, this will reduce latency for the user operations. And, you can do what you want in there. You can get a summary with bookings and you can put inside the item.
{
id: "3",
"name": "",
"description": "",
"extra": [{}],
"bookingCount": "",
"bookingsByStatus": {
"status_1": 1233,
"status_2": 1233,
...
}
}
But this is a business decision. And after any update of items and booking, you need yo update the item from Elasticsearch index. Also, you can use other solution like mentione by #jzzfs.

Google Tag Manager - Python - Creating a custom event trigger

I am trying to automate some GTM tasks. The below code to create a "All Pages" trigger for Google Analytics works.
def CreateGATrigger(service, workspace):
"""Create the GA Trigger.
Args:
service: the Tag Manager service object.
workspace: the workspace to create the trigger within.
Returns:
The created trigger.
"""
GA_trigger = {
'name': 'All Pages',
'type': 'PAGEVIEW'
}
return service.accounts().containers().workspaces().triggers().create(
parent=workspace['path'],
body=GA_trigger).execute()
However, the below code gives me the below error. The goal of the below code is to create a custom event trigger that fires when the event variable contains "formSubmit".
def CreateformSubmitTrigger(service, workspace):
formSubmit_trigger = {
"name": 'formSubmit',
"type": 'customEvent',
'customEventFilter': [
{
'parameter': [
{
'type': 'template',
'key': 'arg0',
'value': '{{event}}'
},
{
'type': 'template',
'key': 'arg1',
'value': 'formSubmit'
}
],
'type': 'contains'
}
]
}
return service.accounts().containers().workspaces().triggers().create(
parent=workspace['path'],
body=formSubmit_trigger).execute()
Error: ( X'd out the account and container )
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/tagmanager/v2/accounts/XXXXXXXXXX/containers/XXXXXXX/workspaces/1/triggers?alt=json returned "Bad Request">
I got this to work using the below code.
def CreateformSubmitTrigger(service, workspace):
formSubmit_trigger = {
"name": "formSubmit",
"type": "customEvent",
"customEventFilter": [
{
"type": "equals",
"parameter": [
{
"type": "template",
"key": "arg0",
"value": "{{_event}}"
},
{
"type": "template",
"key": "arg1",
"value": "formSubmit"
}
]
}
],
"filter": [
{
"type": "contains",
"parameter": [
{
"type": "template",
"key": "arg0",
"value": "{{Event}}"
},
{
"type": "template",
"key": "arg1",
"value": "formSubmit"
}
]
}
],
}
return service.accounts().containers().workspaces().triggers().create(
parent=workspace['path'],
body=formSubmit_trigger).execute()

copyIndex() inside a listKeys()

We're trying to deploy an ARM template which deploys a Stream Analytics job with n Event Hubs outputs depending on an input parameter.
Right now we're having success with all but the listKeys() function inside the outputs property copy loop function which gets each Event Hub's primary keys:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
We get the error:
17:44:31 - Error: Code=InvalidTemplate; Message=Deployment template
validation failed: 'The template resource
'tailor-router-axgf7t3gtspue' at line '129' and column '10' is not
valid: The template function 'copyIndex' is not expected at this
location. The function can only be used in a resource with copy
specified. Please see https://aka.ms/arm-copy for usage details..
Please see https://aka.ms/arm-template-expressions for usage
details.'.
However, if we change this to be a specific index:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[0].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
it works.
Is copyIndex('propertyName') inside a listKeys() a supported function?
If not, is there a workaround that would achieve the same effect?
Kind regards,
Nick
Stream Analytics job resource definition:
{
"apiVersion": "2016-03-01",
"type": "Microsoft.StreamAnalytics/StreamingJobs",
"name": "[variables('routerStreamAnalyticsName')]",
"location": "[variables('location')]",
"dependsOn": [ "clientsEventHubCopy" ],
"tags": {
"boundedContext": "[variables('boundedContextName')]"
},
"properties": {
"sku": {
"name": "[parameters('routerStreamAnalyticsSkuTier')]"
},
"outputErrorPolicy": "drop",
"eventsOutOfOrderPolicy": "adjust",
"eventsOutOfOrderMaxDelayInSeconds": 0,
"eventsLateArrivalMaxDelayInSeconds": 5,
"dataLocale": "en-US",
"compatibilityLevel": "1.0",
"inputs": [
{
"name": "input0",
"properties": {
"type": "stream",
"serialization": {
"type": "Avro"
},
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[parameters('input0EventHubNamespace')]",
"sharedAccessPolicyName": "[parameters('input0EventHubSharedAccessPolicyName')]",
"sharedAccessPolicyKey": "[parameters('input0EventHubSharedAccessPolicyKey')]",
"eventHubName": "[parameters('input0EventHubName')]"
}
}
}
}
],
"transformation": {
"name": "routing",
"properties": {
"streamingUnits": "[parameters('routerStreamAnalyticsSkuTier')]",
"query": "omitted"
}
},
"copy": [
{
"name": "outputs",
"count": "[length(parameters('clients'))]",
"input": {
"name": "[parameters('clients')[copyIndex('outputs')].id]",
"properties": {
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[variables('clientEventHubNamespace')]",
"sharedAccessPolicyName": "[variables('clientEventHubClientSharedAccessName')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]",
"eventHubName": "[parameters('clients')[copyIndex('outputs')].id]"
}
},
"serialization": {
"type": "Avro"
}
}
}
}
]
}
},
Thanks for reporting this and sorry for the inconvenience.
I just talked to the ARM team, we had an issue when copyindex was inside the index tags eg 'array[copyindex()]'. It should be fixed now.
Let us know how it goes.
Thanks,
JS - Azure Stream Analytics

Resources