I have 2 LSI in my table with a primary partition Key with primary sort key
Org-ID - primary partition Key
ClientID- primary sort Key
Gender - LSI
Section - LSI
I have no issue with querying a table with one LSI, but how to mention 2 LSI in a table schema.
var params = {
TableName:"MyTable",
IndexNames: ['ClientID-Gender-index','ClientID-Section-index'],
KeyConditionExpression : '#Key1 = :Value1 and #Key2=:Value2 and #Key3=:Value3',
ExpressionAttributeNames:{
"#Key1":"Org-ID",
"#Key2":"Gender",
"#Key3":"Section"
},
ExpressionAttributeValues : {
':Value1' :"Microsoft",
':Value2':"Male",
':Value3':"Cloud Computing"
}};
Can anyone fix the issue in IndexName(line 3) or KeyConditionExpression(line 4), I'm not sure about it.
Issue
Condition can be of length 1 or 2 only
You can only query a single DynamoDB index at a time. You cannot use multiple indexes in the same query.
A simple alternative is to use a single index and apply a query filter, but this will potentially require a lot of records to be scanned and the filter only reduces the amount of data transferred over the network.
A more advanced alternative is to make a compound key. You would most likely want to use a GSI, rather than an LSI for this use case. By making a single new column that is the string concatenation of Key1, Key2, and Key3 you can use this GSI to search all three keys at the same time. This will make each individual record bigger by repeating data but it allows for a more complex query pattern.
Related
I have a DynamoDB table with following keys:
id: partition key
created_at: sort key
brand#category#size#color: partition key for global index 'byAttributes'
The global index partition key is a composite of 4 table attributes using '#' as a delimiter.
Is there a way in DynamoDB that I can query the table using only a subset of the attributes using a wildcard for unspecified attributes?
As examples:
byAttributes = 'levis#shirts#*#red'
byAttributes = '*#pants#L#*'
I don't wish to use a FilterExpression because it only filters data after a search. I want to take advantage of the attributes being indexed.
No. But you can create alternative GSIs for different combinations.
You can also include a hierarchical SK value and use begins-with to limit based on zero or more values.
Putting some values in the PK and the rest in a hierarchical SK achieves a lot of combinations.
For example have a GSI:
PK = category,
SK = size#brand#color
Now you can query by category, category/size, category/size/brand, or all four.
If it gets more than four you may want to look at Rockset as an indexing system against DynamoDB data.
I have a simple table consisting of orderID as PK and userID as SK, I found out that in dynamoDB you need to specify both PK and SK to use query. so in this case, how is it possible for me to get all the orders for userID x since I can't ignore orderID since they're the partition key of this table? another way to solve this which works but not recomended is using a scan filter, which scans the whole table then filters the result. it will eventually slow down as the table grow. I wonder how do you guys do it with this scenario?
You can create a Global Secondary Index (GSI) based on userId and query based on that index.
You can read more about indexes and GSI’s in the AWS docs here.
I have a dynamoDB table with two attributes:
A: primary partition key
B: primary sort key
I want to query this table using attribute B since I don't know the value of A. Is it possible to do so?
Is it possible to make B as GSI (global secondary index), how to do and query the table using B, since B is already a sort key.
You need partition-key to query - you can't do it using sort-key alone. You can only scan.
So, the only way out for you is to create a GSI with B as the partition-key.
Update
Yes, you can use range-key as GSI.
The drawback to using GSI are:
There can only be a maximum of 5 GSI per table, so choose wisely what you need to index as GSI can only be specified during table creation and cannot be altered.
GSI will cost you additional money as you will need to assign Provisioned Throughput to it.
GSI is eventually consistent, meaning that DynamoDB does not guarantee that the moment data associated to the table's hash key is written into DB, the data's GSI hash key immediately becomes available for querying. The document states that this is usually immediate, but can be the case that it could take up to seconds for the GSI hash key to become available.
select * from tableName where columnName="value";
How can I fetch a similar result in DynamoDB using java, without using primary key as my attribute (Need to group data based on a value for a particular column).
I have gone through articles regarding getbatchitems, QuerySpec but all these require me to pass the primary key.
Can someone give a lead here?
Short answer is you can't. Whenever you use the Query or GetItem operations in DynamoDB you must always supply the table or index primary key.
You have two options:
Perform a Scan operation on the table and filter by columnName="value". However this requires DynamoDB to look at every item in the table so it is likely to be slow and expensive.
Add a Global Secondary Index to your table. This will require you to define a primary key for the index that contains the columnName you want to query
I'm new to AWS DynamoDB and wanted to clarify something. Is it possible to query a table and filter base on a non-primary key attribute. My table looks like the following
Store
Id: PrimaryKey
Name: simple string
Location: simple string
Now I want to query on the Name, but I think I have to give the key as well from what I know? Apart from that I can use the scan but then I will be loading all the data.
From the docs:
The Query operation finds items based on primary key values. You can query any table or secondary index that has a composite primary key (a partition key and a sort key).
DynamoDB requires queries to always use the partition key.
In your case your options are:
create a Global Secondary Index that uses Name as a primary key
use a Scan + Filter if the table is relatively small, or if you expect the result set will include the majority of the records in the table
There are few designs principals that you can follow while you are using DynamoDB. If you are coming from a relational background, you have already witnessed the query limitations from primary key attributes.
Design your tables, for querying and separating hot and cold data.
Create Indexes for Querying from Non Key attributes (You have two options, Global Secondary Index which you can define at any time and Local Secondary Index which you need to specify at table creation time).
With the Global Secondary Index you can promote any NonKey attribute as the Partition Key for the Index and select another attribute for Sort Key for querying. For Local Secondary Index, you can promote any Non Key attribute as the Sort Key keeping the same Partition Key.
Using Indexes for query is important also to improve the efficiency in using provisioned throughput.
Although having indexes consumes the read throughput from the table, it also saves read through put from in a way that, if you project the right amount of attributes to read, it can give a huge benefit in reading. Check the following example.
Lets say you have a DynamoDB table that has items of 40KB. If you read directly from the table to list 10 items, it consumes 100 Read Throughput Units (For one item 10 Units since one unit can read 4KB and multiply it by 10). If you have an index defined just to project the attributes needed to list which will be having 4KB per item, then it will be consuming only 10 Read Throughput Units(One Unit per item) which makes a huge difference in terms of cost.
With DynamoDB its really important how you define Indexes to optimize for Querying not only from Query capability but also in terms of throughput.
You can not query based non-primary key attribute in Dynamo Db.
If you wanted to still do that you can do it using scan query,but scan is costly operation in DyanmoDB and if table is large, then it will affect performance and not recommended because it will scan each item in table and AWS cost you for all item it scan for that query.
There are two ways to achieve it
Keep Store Id as your PrimaryKey/ Partaion key of Dyanmo DB table and add Name/Location as sort Key (only one as Dyanmo DB accept only one Attribute as sort key by design.
Create Global Secondary Indexes for Querying from Non Key attributes which you are more frequenly required.
There are 3 ways to created GSI in Dyanamo DB, In your case select GSI with option INCLUDE and add Name , Location and store ID in Idex.
KEYS_ONLY – Each item in the index consists only of the table partition key and sort key values, plus the index key values. The KEYS_ONLY option results in the smallest possible secondary index.
INCLUDE – In addition to the attributes described in KEYS_ONLY, the secondary index will include other non-key attributes that you specify.
ALL – The secondary index includes all of the attributes from the source table. Because all of the table data is duplicated in the index, an ALL projection results in the largest possible secondary index.