DynamoDB primarykey timestamp - amazon-dynamodb

iam trying to query a dataset in dynamodb where the primary key is a timestamp.
First i wanted to get all data for a specific sensorId.
I tried with a scan (scan.json):
{
"sensorId": {
"AttributeValueList": [{
"S": "1234"
}],
"ComparisonOperator": "EQ"
}
}
This Json was used via CLI command:
aws dynamodb scan --table-name sensorData --scan-filter file://scan.json
That was successful and gave me all data for the specified sensorid.
Now if i want to have only the timestamp and sensorId as result, i read about projection-expression and tried to do a query (query.json):
{
":Id":{"S":"1234"}
}
aws cli command
aws dynamodb query --table-name sensorData --key-condition-expression "sensorId= :Id" --expression-attribute-values file://query2.json --projection-expression "timestamp"
Gave me :
An error occurred (ValidationException) when calling the Query
operation: Invalid ProjectionExpression: Attribute name is a reserved
keyword; reserved keyword: timestamp
But replacing "timestamp" with "sensorId for testing purpose gave me:
An error occurred (ValidationException) when calling the Query
operation: Query condition missed key schema element: timestamp
And i understand that that sensorId is not valid for key-expression..
KeyConditionExpression accepts only key attributes, hash key and range key.
But how to get the result?
I want only the timestamps for a sensorId.
Is my Primarykey wrong? should be better to use sensorId as primary key together with timestamp as range key?


Based on your scenario, its better to change your keys. Use SensorID as the Partition Key and Timestamp as the Sort Key.
This way you can query (Without scanning all the items) the items for a given SensorID. Also its possible to sort them in order of the Timestamp.
If you have a larger dataset (More than several Killobytes) it would be efficient to create a LSI or GSI to project the required attributes for a given query.
Note: TimeStamp is a reserved word in DynamoDB. You can use Expression Attribute Names to avoid the errors when using reserved attributes in query expressions.

Related

Dynamodb saying the provided key element does not match the scheme

I had created a table in dynamo db called Music with item Taylor Swift. I wanted to get this item through command line but it repeatedly gives me this error.
~ % aws dynamodb get-item --table-name Music --key '{ "Artist" : { "S" : "Taylor Swift"}}'
An error occurred (ValidationException) when calling the GetItem operation: The provided key element does not match the schema
I triple checked if the attribute type was right. What else could be wrong?

For the primary key, you must provide all of the attributes. For example, with a simple
primary key, you only need to provide a value for the partition key. For a composite
primary key, you must provide values for both the partition key and the sort key.

Proper syntax for get-item command, for dynamodb of the aws

I am writing get-item command for dynamodb of the aws.
Here is my command:
aws dynamodb get-item --table-name foo --key '{\" bar \":{\"S\":\"aaaa\"},\"timestamp\":{\"N\":\"1603610188890\"}}'
As you can see, the table foo has composite primary key:
partition key "bar" and sort key "timestamp".
What is the proper syntax to use comparison for the sort key "timestamp"?
How I can to change my command to get the items whose timestamp is between 1603010188890 and 1603610188890?
Thanks.

The get-item operation can only retrieve a single item, with a specific key. To retrieve items - possibly more than one - in a certain sort-key range you need to use a different request - query.
The "query" request has a key-conditions or key-condition-expression (these are the older and newer, respectively, syntax, for the same thing). With that parameter you can say that you want items where the partition key is equal something, and the sort key is between two values.

dynamodb query multiple values from GSI

I have a dynamo DB table (id(pk),name(sk),email,date,itemId(number))
and GSI on (itemId pk, date(sk)
trying to query for an array of itemIds [1,2,3,4] but getting error using the IN statement in KeyExperssionValue when doing
aws.DocClient.query
const IdsArrat = [1,2,3,4,5];
const query: {
IndexName: 'accountId-createdAt-index',
KeyConditionExpression: 'itemId IN (:a1,:a2,:a3)',
ExpressionAttributeValues: {
{
':a1':1,
':a2':2,
.......
}
},
ScanIndexForward: false,
},
getting error using the IN statement in.
This it possible to query for multiple values on GSI in dynamoDb ?

You're trying to query for multiple different partition key's in a GSI. This can only be done by doing multiple individual queries (3 in the example). It's also possible with a GSI that multiple values would get returned for a single Partition key lookup, so it's better to query the partition key "itemId" individually.
See the following for reference:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-KeyConditionExpression

It's not possible to have a IN and join multiple values in a query , but it's possible to use BatchGetItem to request multiple queries that are solved in parallel . This is actually very close to the IN solution you want.
The result will be a list of the elements in the table.
There are limits in the number of queries in the size of the result set < 16 MB and the number of queries < 100.
Please check this document for details :
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html

refering to this answer https://stackoverflow.com/a/70494101/7706503, you could try partiQL to construct similar statement for querying gsi table with multiple key,
select * from table."gsi_index_name" where partition_key in [key1,key2]
then you could send the statement with low level api in one shot, for example, in dotnet, it's called ExecuteStatementAsync

How to use aws cli with dynamodb to query for index whose name contains ':'?

Reading the documentation I can see that key filter expression should look like:
partitionKeyName = :partitionkeyval
however, the values of my GSI contain : in themselves. So using this:
aws dynamodb query --table-name my_table --index-name mark_id_index --select ALL_ATTRIBUTES --key-condition-expression mark_id=:bit:BT-EU:dev
causes the following error:
An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: Syntax error; token: ":BT", near: ":bit:BT-"
They also mention using # if the word is reserved, but I think in my case it is the character that is reserved. I have tried to google around # variables but cannot find anything.

DynamoDB: Filter Expression can only contain non-primary key attributes

I can query a GSI via DynamoDB console as can be seen in the screenshot.
When I run the same query with Boto3 on terminal with the following code:
table.query(
IndexName='date-timestamp-index',
KeyConditionExpression=Key('date').eq('20161231'),
FilterExpression=Attr('timestamp').between(1483130000, 1483133600) & Attr('tags').exists()
)
I get a ValidationException exception:
Filter Expression can only contain non-primary key attributes: Primary key attribute: timestamp
What am I doing wrong here? Thanks.

Your timestamp field is the sort key of the table, so it cannot be used in FilterExpression. It must be part of the KeyConditionExpression.

Resources