How to backfill new AppSync fields using AWS Amplify - amazon-dynamodb

Question

I'm adding a sort field to one of my AppSync tables using GraphQL. The new schema looks like:
type MyTable
#model
#auth(rules: [{allow: owner}])
#key(name: "BySortOrder", fields: ["sortOrder"], queryField: "tableBySortOrder")
{
id: ID!
name: String!
sortOrder: Int
}
However, when retrieving a list using tableBySortOrder I get an empty list because the new field sortOrder is null.
My question is, how do I backfill this data in the DynamoDB table so that my existing users will not be disrupted by this new change? With a traditional database, I would run a SQL update: UPDATE MyTable SET sortOrder = #.
However, I'm new to NoSQL/AWS and couldn't find a way to do this except build a backfill script whenever a user logs into my app. That feels very hacky. What is the best practice for handling this type of scenario?

Answer 1

Have you already created the new field in DDB?
If yes, I think you should backfill it before making the client side change.
Write a script to iterate through and update the table. Options for this:
Java - Call updateItem to update the table if you have any integ tests running.
Bash - Use AWS CLI: aws dynamodb scan --table-name item_attributes --projection-expression "whatever" > /tmp/item_attributes_table.txt and then aws dynamodb update-item --table-name item_attributes --key. This is a dirty way.
Python - Same logic as above.

Answer 2

Ended up using something similar to what Sunny suggested with a nodejs script:
const AWS = require('aws-sdk')
AWS.config.update({
region: 'us-east-1'
})
// To confirm credentials are set
AWS.config.getCredentials(function (err) {
if (err) console.log(err.stack)
// credentials not loaded
else {
console.log('Access key:', AWS.config.credentials.accessKeyId)
console.log('Secret access key:', AWS.config.credentials.secretAccessKey)
}
})
const docClient = new AWS.DynamoDB.DocumentClient()
const table = 'your-table-dev'
const params = {
TableName: table
}
const itemMap = new Map()
// Using scan to retrieve all rows
docClient.scan(params, function (err, data) {
if (err) {
console.error('Unable to query. Error:', JSON.stringify(err, null, 2))
} else {
console.log('Query succeeded.')
data.Items.forEach(item => {
if (itemMap.has(item.owner)) {
itemMap.set(item.owner, [...itemMap.get(item.owner), item])
} else {
itemMap.set(item.owner, [item])
}
})
itemMap.forEach(ownerConnections => {
ownerConnections.forEach((connection, index) => {
connection.sortOrder = index
update(connection)
})
})
}
})
function update(connection) {
const params = {
TableName: table,
Key: {
'id': connection.id
},
UpdateExpression: 'set sortOrder = :s',
ExpressionAttributeValues: {
':s': connection.sortOrder,
},
ReturnValues: 'UPDATED_NEW'
};
console.log('Updating the item...');
docClient.update(params, function (err, data) {
if (err) {
console.error('Unable to update item. Error JSON:', JSON.stringify(err, null, 2));
} else {
console.log('UpdateItem succeeded:', JSON.stringify(data, null, 2));
}
});
}