Using nested fields in the projection of a DynamoDB GSI - amazon-dynamodb

I've got a Dynamo table storing documents that look like this:
{
"guid": "<some UUID>"
"created_at": 1550778260030,
"display_name": "person",
"updated_at": 1550778260030,
"info": {
"locked": false,
"confirmed": true,
"deactivated": false,
"email": "person#example.com"
}
}
The table has a Global Secondary Index managed by Terraform defined as such:
global_secondary_index {
name = "display_name_index"
hash_key = "display_name"
projection_type = "INCLUDE"
non_key_attributes = [
"updated_at",
"info.email",
"created_at"
]
}
However when I query the table the info.email field isn't returned:
aws dynamodb query \
--table-name "accounts" \
--index-name "display_name_index" \
--key-condition-expression "display_name = :display_name" \
--expression-attribute-values '{":display_name":{"S":"person"}}'
{
"Count": 1,
"Items": [
{
"created_at": {
"N": "1550778260030"
},
"display_name": {
"S": "person"
}
"updated_at": {
"N": "1550778260030"
}
}
],
"ScannedCount": 1,
"ConsumedCapacity": null
}
If I change the non_key_attributes to include info it returns the full info blob just fine, and I can use a projection-expression of info.email to retrieve that field just fine:
{
"Count": 1,
"Items": [
{
"info": {
"M": {
"email": {
"S": "person#example.com"
}
}
}
}
],
"ScannedCount": 1,
"ConsumedCapacity": null
}
The Dynamo docs do specify that index keys have to be top-level, but they don't mention anything about non-key attributes in a projection having to be top-level. Therefore I'd assume that anything that works in a projection-expression should work in an index projection, but that seems to not be the case?
Am I doing something wrong with this index definition or the query? Or does Dynamo just not support nested non-key attributes as part of an index's projection?

In simple words nested attribute can not be used as a GSI projection. It is not supported in DDB yet.

I walked into the same thing. I see there are no answers jet to your question. Not sure I have the right answer, but maybe it can help you out.
First of all, I think it's very weird creating GSI the API allows you to add a projection of "info.email" (this will also visible on the index overview page) but can never be retrieved again.
I found out when creating a GSI you are stuck on the attributes you have provided.
On the other hand, creating LSI you can use the attributes you have provided while creating the LSI.
You can found a little about this in this document (search for "Projected Attributes"):
https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/SecondaryIndexes.html
I hope you can do something with this info.

Related

Artifactory: how can I obtain a complete dump of all fields for an artifact?

I'm interested in the value of one field (Module ID) but there seems to be no way of obtaining this specifically. A complete dump of all field values would also suffice but I haven't succeeded in finding a way to do that either. I've looked at and tried the searches available within the documentation here: https://www.jfrog.com/confluence/display/JFROG/Artifactory+REST+API#ArtifactoryRESTAPI-SEARCHES
If it helps, I'm trying to query an on-premise installation of Artifactory.
More fields can be added to the AQL using the "include" element.
For example - to list all artifacts under "libs-release-local" repository, including their module names, run the following query:
items.find(
{
"repo":{"$eq":"libs-release-local"}
}
).include("artifact.module")
Response example:
{
"results": [
{
"repo": "libs-release-local",
"path": "org/jfrog/test/multi2/2.17.0",
"name": "multi2-2.17.0.jar",
"type": "file",
"size": 1022,
"created": "2021-09-11T13:51:33.878Z",
"created_by": "deployer",
"modified": "2021-09-11T13:51:33.631Z",
"modified_by": "deployer",
"updated": "2021-09-11T13:51:33.881Z",
"artifacts": [
{
"modules": [
{
"module.name": "org.jfrog.test:multi2:2.17.0"
}
]
}
]
}
]
}
You can find all of the required information under AQL documentation.

Query in a key-value list field DynamoDB

I'm attempting make a query in a field that is list of key-value, as below.
Unfortunatelly I'm novice in DynamoDB, sorry.
My goal is get a record using a filter like:
Where
TransactionParameterList.Name = "HOSTING_NR" and
TransactionParameterList.OPERATION = "1234"
It's possible?
How?
I'm using AWS .NET SDK
Regards!
"TransactionID": "657",
"TransactionInstanceIdentifier": "919C5A0E-8786-4B86-87BF-5080E2639406",
"TransactionParameterList": [
{
"Name": "HOSTING_NR",
"Value": "1234"
},
{
"Name": "OPERATION",
"Value": "GetData"
}
],
"TransactionStepID": "1491",
"TransactionStepInstanceIdentifier": "0B0763AB-A847-479D-8EFE-43B842F2B2EB"
}
As of now DynamoDB does not support querying list of map items.
workaround in discussion forum here

Serialized Entities displaying only ID

I'm using JMSSerializer and FOSRestBundle. I have a fairly typical object graph, including some recursion.
What I would like to accomplish is that included objects beyond a certain depth or in general are listed only with their ID, but when serialized directly, with all data.
So, for example:
Users => Groups => Users
when requesting /user/1 the result should be something like
{ "id": 1, "name": "John Doe", "groups": [ { "id": 10 }, { "id": 11 } ] }
While when I request /group/10 it would be:
{ "id": 10, "name": "Groupies", "users": [ { "id": 1 }, { "id": 2 }, { "id": 4 } ] }
With #MaxDeph I can hide the included arrays completely, so I get
{ "id": 1, "name": "John Doe", "groups": [] }
But I would like to include just the IDs so that the REST client can fetch them if it needs them, or consult his cache, or do whatever.
I know I can manually cobble this together using groups, but for consistency reasons I was wondering if I can somehow enable this behaviour in my entire application, maybe even with a reference to maxdepth so I can control where to include IDs and where to include full objects?
For the sake of those finding this:
I found no other solution, but doing this with groups works just fine and gives me the result I was looking for.

"Reverse formatting" Riak search results

Let's say I have an object in the test bucket in my Riak installation with the following structure:
{
"animals": {
"dog": "woof",
"cat: "miaow",
"cow": "moo"
}
}
When performing a search request for this object, the structure of the search results is as follows:
{
"responseHeader": {
"status": 0,
"QTime": 3,
"params": {
"q": "animals_cow:moo",
"q.op": "or",
"filter":"",
"wt": "json"
}
},
"response": {
"numFound": 1,
"start": 0,
"maxScore": "0.353553",
"docs": [
{
"id": "test",
"index": "test",
"fields": {
"animals_cat": "miaow",
"animals_cow": "moo",
"animals_dog": "woof"
},
"props": {}
}
]
}
}
As you can see, the way the object is stored, the cat, cow and dog keys are nested within animals. However, when the search results come back, none of the keys are nested, and are simply separated by _.
My question is this: Is there any way provided by Riak to "reverse format" the search, and return the fields of the object in the correct (nested) format? This becomes a problem when storing and returning user data that might possibly contain _.
I do see that the latest version of Riak (beta release) provides a search schema, but I can't seem to see whether my question would be answered by this.
What you receive back in the search result is what the object looked like after passing through the json analyzer. If you need the data formatted differently, you can use a custom analyzer. However, this will only affect newly put data.
For existing data, you can use the id field and issue a get request for the original object, or use the solr query as input to a MapReduce job.

Freebase: Get name & Wikipedia ID in one query in a certain language

Is it possible to do one query in MQL to obtain the name and the wikipedia ID for a certain language from Freebase? If that's possible, is it also possible to do this for a set of languages (eg. german & english)?
Asked and answered, but here's a slightly better form of the query:
[{
"id": "/en/white_house",
"mid": null,
"de:name": {
"lang": "/lang/de",
"value": null
},
"en:name": null,
"wiki_de:key": {
"/type/key/namespace": "/wikipedia/de_id",
"value": null,
"optional": True,
},
"wiki_en:key": {
"/type/key/namespace": "/wikipedia/en_id",
"value": null,
"optional": True,
}
}]​
The Wikipedia keys will be escaped if they contain special characters, so you should consult
http://wiki.freebase.com/wiki/MQL_key_escaping for how to unescape them.
Some of the reasons this query is better include:
English is the default language, so it doesn't need to be specified for names
it removes the ambiguity of the namespace lookup. Your original query is actually looking for the key "white_house" in any namespace (and finding it in "/en" which is equivalent to the id "/en/white_house")
Note that you don't need to do the lookup by ID. You can use any lookup facility that MQL provides such as looking up by one of the Wikipedia keys or using "name~=":"white house" to find all topics containing that string or anything else that works for your starting data and your use case.
After playing around with MQL I finally came up with the following query (for white_house):
[{
"id": null,
"mid": null,
"de:name": {
"lang": "/lang/de",
"value": null
},
"en:name": {
"lang": "/lang/en",
"value": null
},
"key": {
"value": "white_house"
},
"wiki_de:key": {
"/type/key/namespace": "/wikipedia/de_id",
"value": null
},
"wiki_en:key": {
"/type/key/namespace": "/wikipedia/en_id",
"value": null
}
}]

Resources