I had created a table in dynamo db called Music with item Taylor Swift. I wanted to get this item through command line but it repeatedly gives me this error.
~ % aws dynamodb get-item --table-name Music --key '{ "Artist" : { "S" : "Taylor Swift"}}'
An error occurred (ValidationException) when calling the GetItem operation: The provided key element does not match the schema
I triple checked if the attribute type was right. What else could be wrong?
For the primary key, you must provide all of the attributes. For example, with a simple
primary key, you only need to provide a value for the partition key. For a composite
primary key, you must provide values for both the partition key and the sort key.
Related
I am looking for way to improve the following query scans. I need to query based on 3 keys
Primary Partition Key
GSI Partition Key
GSI Sort Key
DynamoDB only allows 2 conditions in key-condition-expression. I have to use filter-expression which is scanning too many records.
I am also considering combining 2 keys as sort key in GSI as an alternative. Is this the right way to do this?
aws dynamodb query \
--table-name bundles \
--index-name GSI-RegulationSidBundleStatus \
--key-condition-expression "RegulationSid = :regulationSid AND BundleStatus = :bundleStatus" \
--filter-expression "AccountSid = :accountSid" \
--expression-attribute-values '{
":accountSid": {
"S": "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
},
":regulationSid": {
"S": "YYYYYYYYYYYYYYYYYYYYYYYYYYY"
},
":bundleStatus": {
"S": "APPROVED"
}
}'
Yes, adding multiple keys into the partition key or sort key is a common pattern. To help with identifying keys, it is common to prefix each key with the key type, or an abbreviation, followed by a hash, and a hash between each key.
For your case, a sort key would look similar to:
r#${RegulationSid}#b${BundleStatus}.
This is also common for partition keys, even when you have just one key. In your case: a#${AccountSid}.
When deciding on whether to overload the partition key or sort key, and the order of the keys, look at your access patterns. If you know you have a pattern to get all regulations, you want to put this key first. Or, if you know you need to get all regulations for a given bundle status, put the bundle status first. Then you can use a begins_with query to get these lists of items, reusing the same GSI.
In your code documentation, you can list the used abbreviations to make sure you don't use the same abbreviation for multiple key types.
I suggest always keeping the customer account Id as the first key in the partition key. If you later decide to use Leading Key row-level authorisation this will come in handy.
I am writing get-item command for dynamodb of the aws.
Here is my command:
aws dynamodb get-item --table-name foo --key '{\" bar \":{\"S\":\"aaaa\"},\"timestamp\":{\"N\":\"1603610188890\"}}'
As you can see, the table foo has composite primary key:
partition key "bar" and sort key "timestamp".
What is the proper syntax to use comparison for the sort key "timestamp"?
How I can to change my command to get the items whose timestamp is between 1603010188890 and 1603610188890?
Thanks.
The get-item operation can only retrieve a single item, with a specific key. To retrieve items - possibly more than one - in a certain sort-key range you need to use a different request - query.
The "query" request has a key-conditions or key-condition-expression (these are the older and newer, respectively, syntax, for the same thing). With that parameter you can say that you want items where the partition key is equal something, and the sort key is between two values.
I'm trying to run DynamoDB queries on a table containing HashKey and SortKey fields.
While doing get-item operation, when I provide key with both hashkey, sortkey fields I'm seeing the results. But when I tried with only hashkey field getting the following Exception:
An error occurred (ValidationException) when calling the GetItem operation: One of the required keys was not given a value
Can't we get DynamoDB data based on hashkey only?
When we use both partition key and sort key for a table, the primary key generated based on both. Which means there could be multiple items with the same partition key (hash key).
For an example,
partition key | sort key
1 | A
1 | B
2 | A
2 | B
In order to perform get operation, you need to specify the primary key of the item.
In your table, the hash key is not the primary key. You need to specify both partition key and sort key to get an item from the table.
Further, if you are using query operation then specifying only partition key will work.
iam trying to query a dataset in dynamodb where the primary key is a timestamp.
First i wanted to get all data for a specific sensorId.
I tried with a scan (scan.json):
{
"sensorId": {
"AttributeValueList": [{
"S": "1234"
}],
"ComparisonOperator": "EQ"
}
}
This Json was used via CLI command:
aws dynamodb scan --table-name sensorData --scan-filter file://scan.json
That was successful and gave me all data for the specified sensorid.
Now if i want to have only the timestamp and sensorId as result, i read about projection-expression and tried to do a query (query.json):
{
":Id":{"S":"1234"}
}
aws cli command
aws dynamodb query --table-name sensorData --key-condition-expression "sensorId= :Id" --expression-attribute-values file://query2.json --projection-expression "timestamp"
Gave me :
An error occurred (ValidationException) when calling the Query
operation: Invalid ProjectionExpression: Attribute name is a reserved
keyword; reserved keyword: timestamp
But replacing "timestamp" with "sensorId for testing purpose gave me:
An error occurred (ValidationException) when calling the Query
operation: Query condition missed key schema element: timestamp
And i understand that that sensorId is not valid for key-expression..
KeyConditionExpression accepts only key attributes, hash key and range key.
But how to get the result?
I want only the timestamps for a sensorId.
Is my Primarykey wrong? should be better to use sensorId as primary key together with timestamp as range key?
Based on your scenario, its better to change your keys. Use SensorID as the Partition Key and Timestamp as the Sort Key.
This way you can query (Without scanning all the items) the items for a given SensorID. Also its possible to sort them in order of the Timestamp.
If you have a larger dataset (More than several Killobytes) it would be efficient to create a LSI or GSI to project the required attributes for a given query.
Note: TimeStamp is a reserved word in DynamoDB. You can use Expression Attribute Names to avoid the errors when using reserved attributes in query expressions.
I can query a GSI via DynamoDB console as can be seen in the screenshot.
When I run the same query with Boto3 on terminal with the following code:
table.query(
IndexName='date-timestamp-index',
KeyConditionExpression=Key('date').eq('20161231'),
FilterExpression=Attr('timestamp').between(1483130000, 1483133600) & Attr('tags').exists()
)
I get a ValidationException exception:
Filter Expression can only contain non-primary key attributes: Primary key attribute: timestamp
What am I doing wrong here? Thanks.
Your timestamp field is the sort key of the table, so it cannot be used in FilterExpression. It must be part of the KeyConditionExpression.