I need to change the index column of GlobalSecondaryIndex in DynamoDB. At first, I just change the "AttributeName" of GSI. It didn't work. Then I researched online and found that I should delete the current GSI and create a new GSI. Here, I use IndexStatus: DELETING to remove the current GSI. However, it always showed error:Encountered unsupported property IndexStatus. So how to fix this error? I'm coding in yaml.
To change the column of a GSI you'll have to use a two step process.
Remove the GSI from your table.
Add the GSI back to the table with the new column.
Because your question is tagged with yaml I assume you are using CloudFormation, or some form of IaC. You'll need to remove the GSI, deploy the change without the GSI, add the GSI with the new column, and deploy that.
A safer approach might be to create a new GSI with the new column.
Add the new GSI with the new column and deploy the table
Update your code to use the new GSI and deploy the new code
Remove the old GSI with the old column and deploy the table
Related
I have one table, name is Ticket
Ticket {
id,
usage,
affiliationOrganization
createdAt,
....
}
GSI1:
PartionKey: usage
SortKey: affiliationOrganization
After a development time, i want update SortKey of GSI1
PartionKey: usage
SortKey: affiliationOrganization#createdAt
But after modifying the GSI, the old data is no longer in this GSI table.
Only when adding new data will these new data be automatically typed into the GSI table.
(My english is stupid, I'm sorry if I wrote it wrong)
I want to automatically update old data to GSI after it is modified
Avoid data loss in GSI table, query is not missing data
The old data will automatically replicate to the GSI should it be eligible. Please ensure all your old items have an attribute named affiliationOrganization#createdAt.
Note that this must be a single attribute, DynamoDB does not combine attributes automatically, you must do so manually.
I have a DynamoDB GSI with only certain fields in the projection(attributes). I would like to add a new field to this list of attributes. Is it possible to do this without deleting the GSI and recreating it? I did not find an option to do that in DynamoDB console or in update_table cli.
According to UpdateTable API it is only possible to create and delete GSI. So to update the current GSI, the old GSI needs to be deleted and recreated again.
I have a live table in dynamo with about 28 million records in it.
The table has a number of GSI that I'd like to change to be LSIs however LSIs can only be created when the table is created.
I need to create a new table and migrate the data with minimum downtime. I was thinking I'd do the following:
Create the new table with the correct indexes.
Update the code to write records to the old and new table. When this starts, take a note of the timestamp for the first record.
Write a simple process to sync existing data for anything with a create date prior to my first date.
I'd have to add a lock field to the new table to prevent race conditions when an existing record is updated.
When it's all synced we'd swap to using the new table.
I think that will work, but it's fairly complicated and feels prone to error. Has anyone found a better way to do this?
Here is an approach:
(Let's refer to the table with GSIs as oldTable and the new table with LSIs as newTable).
Create newTable with the required LSIs.
Create a DynamoDB tirgger for the oldTable such that for every new record coming to the oldTable insert the same record to the newTable. (This logic needs to be in the AWS Lambda).
Make your application point to the newTable.
Migrate all the records from oldTable to newTable.
Is it possible to modify the Rangekey column after table creation. Such as adding new column/attribute and assigning as RangeKey for the table. Tried searching but cant ble to find any articles about changing the Range or Hash key
No, unfortunately it's not possible to change the hash key, range key, or indexes after a table is created in DynamoDB. The DynamoDB UpdateItem API Documentation is clear about the fact that indexes cannot be modified. I can't find a reference to anywhere in the docs that explicitly states that the table keys cannot be modified, but at present they cannot be changed.
Note that DynamoDB is schema-less other than the hash and range key, and you can add other attributes to new items with no problems. Unfortunately, if you need to modify either your hash key or range key, you'll have to make a new table and migrate the data.
Edit (January 2014): DynamoDB now has support for on the fly global secondary indexes
To change or create an additional sort key, you will need to create a new table and migrate over to it, as both actions cannot be done on existing tables.
DynamoDB streams enable us to migrate tables without any downtime. I've done this to great effective, and the steps I've followed are:
Create a new table (let us call this NewTable), with the desired key structure, LSIs, GSIs.
Enable DynamoDB Streams on the original table
Associate a Lambda to the Stream, which pushes the record into NewTable. (This Lambda should trim off the migration flag in Step 5)
[Optional] Create a GSI on the original table to speed up scanning items. Ensure this GSI only has attributes: Primary Key, and Migrated (See Step 5).
Scan the GSI created in the previous step (or entire table) and use the following Filter:
FilterExpression = "attribute_not_exists(Migrated)"
Update each item in the table with a migrate flag (ie: “Migrated”: { “S”: “0” }, which sends it to the DynamoDB Streams (using UpdateItem API, to ensure no data loss occurs).
NOTE: You may want to increase write capacity units on the table during the updates.
The Lambda will pick up all items, trim off the Migrated flag and push it into NewTable.
Once all items have been migrated, repoint the code to the new table
Remove original table, and Lambda function once happy all is good.
Following these steps should ensure you have no data loss and no downtime.
I've documented this on my blog, with code to assist:
https://www.abhayachauhan.com/2018/01/dynamodb-changing-table-schema/
Is it possible to delete a column from an SQLlite database. I have Googled this and it seems to be impossible.
From the ALTER TABLE manual;
SQLite supports a limited subset of ALTER TABLE. The ALTER TABLE command in SQLite allows the user to rename a table or to add a new column to an existing table. It is not possible to rename a column, remove a column, or add or remove constraints from a table.
So, no, not without dropping and recreating the table.
According to the docs:
It is not possible to rename a column, remove a column, or add or
remove constraints from a table.
I guess it has to be done manually.