How to apply SCAN query using FilterExpression for nested Json attributes in DynamoDB Query in PHP - amazon-dynamodb

[{
"forms": [{
"id": "f1",
"title": "Form1",
"update_history": [{
"version": "A",
"updated_at": "2016-12-10 12:12:10"
}, {
"version": "B",
"updated_at": "2017-01-01 05:17:19"
}, {
"version": "C",
"updated_at": "2017-02-07 03:22:39"
}]
},{
"id": "f2",
"title": "Form2",
"update_history": [{
"version": "B",
"updated_at": "2016-12-10 12:12:10"
}, {
"version": "C",
"updated_at": "2017-01-01 05:17:19"
}, {
"version": "D",
"updated_at": "2017-02-07 03:22:39"
}]
},{
"id": "f3",
"title": "Form13",
"update_history": [{
"version": "A",
"updated_at": "2016-12-10 12:12:10"
}, {
"version": "E",
"updated_at": "2017-01-01 05:17:19"
}, {
"version": "F",
"updated_at": "2017-02-07 03:22:39"
}]
}]
}]
I have above data stored in Dynamo-DB table. forms object is parent object. There are 3 list data in update_history nested object. I want all the List data which have version A. Please suggest me query for the same.

The update_history is a List data type contains Map objects on it. This is basically a complex structure on DynamoDB data model.
Assuming, the version A will always be the first occurrence on the List, the below FilterExpression should work. If this assumption is not true, the FilterExpression wouldn't work.
This is the only option available for the above data model. If you can redesign to have only Map or List data type, then we can think about alternate solution.
The below code is on JavaScript:-
You can convert it into PHP code. The AWS SDK API concept is same on all languages.
FilterExpression : 'update_history[0].version = :versionVal',
ExpressionAttributeValues : {
':versionVal' : 'A'
}

Related

JSON path help required for a request

I am new at JSON path request and I have a quite complex request to build.
I work with a JSOn strusture having 2 arrays like this example :
{
"WideXml": {
"Guid": "9cf379c5-dc12-4a63-922a-d242efe9a777",
"ApplicationGuid": "24df8af4-58c2-40dd-8ce8-70becb2df96f",
"Action": "Approval",
"Values": {
"Date": {
"TimeStamp": "2021-11-23T04:00:00Z",
"Value": [{
"Guid": "9c64fb06-60f5-4541-a006-3a92ac576e13",
"Value": "6.7169265747070313",
"Unit": "t",
"UserFields": {
"Field": [{
"Value": "131",
"Key": "BWART"
},
{
"Value": "14702-00-BULK",
"Key": "MATNR"
}
]
}
},
{
"Guid": "6c048d70-1521-4fa1-a462-669730d6b1ed",
"Value": "84.824371337890625",
"Unit": "t",
"UserFields": {
"Field": [{
"Value": "261",
"Key": "BWART"
}, {
"Value": "14366-00-WA0R",
"Key": "MATNR"
}]
}
}
]
}
}
}
}
I need to find the "MATNR" code by searching with Key = 'BWART' and Value = '131'.
I can find the Field document by the request
$.WideXml.Values.Date.Value[*].UserFields.Field[?(#.Key=='BWART' && #.Value=='131')]
But I don't manage to build the query to get the MATNR after having this result...
Can someone help?
Regards.
if the key BWART is always at the same index position i.e 0 in this example, you can try the expression
$.WideXml.Values.Date.Value[?(#.UserFields.Field[0].Key== "BWART" && #.UserFields.Field[0].Value == "131")].UserFields.Field[*]

How to convert json array in to the columns table in kusto

I have one parquet file ,trying to get the data in to the table. In one column it have json with multiple values. Can someone help me how to do in kusto?
Pasting the json's schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"path": {
"type": "string"
},
"partitionValues": {
"type": "object",
"properties": {
"deviceId": {
"type": "string"
},
"date": {
"type": "string"
}
},
"required": [
"deviceId",
"date"
]
},
"size": {
"type": "integer"
},
"modificationTime": {
"type": "integer"
},
"dataChange": {
"type": "boolean"
},
"stats": {
"type": "string"
}
},
"required": [
"path",
"partitionValues",
"size",
"modificationTime",
"dataChange",
"stats"
]
}
If I understood correctly, your PQ file contains a column with a JSON of the specified schema. If you want to ingest it as-is, ingest it into Kusto column with type "dynamic" and query later. If you'd like to ingest just part of this JSON data (like some inner fields), use ingestion mapping and provide appropriate JSON path.
Another option is to ingest as-is into a source table with a Retention policy with SoftDeletePeriod of zero. Define an Update Policy with a KQL query and the transformed data will be pushed into a target table.

Python 3.6 JSON string to Object conversion

How to convert the following json string to a Python object?
{
"action": "create",
"data": {
"geometry": {
"coordinates": [
-95.12,
16.52,
-52.0
],
"type": "Point"
},
"id": "20180303_0000046",
"properties": {
"auth": "UNM",
"depth": [{
"pacific": {
"value":52.0
}
}],
"evtype": "ke",
"flynn_region": "OAXACA, MEXICO",
"lastupdate": "2018-03-03T10:26:00.0Z",
"lat": 16.52,
"lon": -95.12,
"mag": 4.0,
"magtype": "m",
"source_catalog": "EMSC-RTS",
"source_id": "652127",
"time": "2018-03-03T07:09:05.0Z",
"unid": "20180303_0000046"
},
"type": "Feature"
}
}
I want to read quake.data.properties.depth[0].pacific
I have used the following ref as example, but that code could not read "pacific" attribute.
ref: Creating a Python object from a Json string

Marketo - How to save complex object in Lead Database?

I was asked to store leads like below in Marketo's lead database through rest api "POST /rest/v1/leads.json".
{
"action": "createOnly",
"lookupField": "email",
"input": [{
"email": "kjashaedd-1#klooblept.com",
"firstName": "Kataldar-1",
"postalCode": "04828",
"property": [{
"type": "land",
"status": "available"
},
{
"type": "car",
"status": "sold out"
},
{
"type": "bike",
"status": "sold out"
},
{
"type": "laptops",
"status": "available"
}
]
},
{
"email": "kjashaedd-2#klooblept.com",
"firstName": "Kataldar-2",
"postalCode": "04828",
"property": [{
"type": "land",
"status": "sold out"
},
{
"type": "car",
"status": "available"
},
{
"type": "bike",
"status": "sold out"
},
{
"type": "laptops",
"status": "available"
}
]
}
]
}
Input field is not in flat json structure. What could be the best approach? Do I really need to use custom objects in this case? Can I dump "property" object as it is in the lead database and use velocity script to parse it ?
If 'Property' is a custom object, you'll want to call that separately for the record and associated the record with that object via the custom object API
So you can push (create) the record first and then associate (add) the custom object to the record.
You can create a List<Dictionary<string,object>> in C# and retrieve the data and then store it using Entity Framework.
I am considering .Net framework in this case

Get one of many values from freebase?

Suppose I am trying to get the wikipage title of all freebase entities with type "/people/person"
[{
"type": "/people/person",
"mid": null,
"key": [{
"namespace": "/wikipedia/en",
"value": null
}]
}]
This returns me the following
"result": [
{
"type": "/people/person",
"key": [
{
"value": "Ann_Copland",
"namespace": "/wikipedia/en"
},
{
"value": "Jack_Abramof",
"namespace": "/wikipedia/en"
},
{
"value": "Jack_Abramoff",
"namespace": "/wikipedia/en"
},
....
Here Ann_Copland, Jack_Abramof, Jack_Abramoff all redirect to the same wikipedia page.
I only want one of the titles (Ann_Copland, Jack_Abramof, Jack_Abramoff ...) from the freebase query. Of course, I can query as above, and ignore the extra titles, but is there a smarter way of specifying the number of results that I expect?
I am not sure about this, but it seems like the following is returning unique pages:
https://www.googleapis.com/freebase/v1/mqlread?query=[{"name":[],"type":"/people/person"}]
To get unique value from your query add "limit":
https://www.googleapis.com/freebase/v1/mqlread?query=[{
"type": "/people/person",
"mid": null,
"key": [{
"limit": 1,
"namespace": "/wikipedia/en",
"value": null
}]
}]

Resources