I am trying to implement an upsert of an item into DynamoDB with optimistic locking. I have the update portion working with a ConditionExpression to check the version. But this fails the save portion as the ConditionExpression is false for saving. Is it possible to write the ConditionExpression so that it will handle both situations?
My code:
result = copy.copy(user)
table = get_db_table()
current_version = result.get_version()
result.update_version()
try:
table.put_item(
Item=result.to_table_item(),
ConditionExpression=Attr(result.get_version_key()).eq(current_version)
)
except ClientError as error:
logger.error(
"Saving to db failed with '%s'",
str(error))
# Restore version
result.set_version(current_version)
raise Exception(ErrorCode.DB_SAVE) from error
return result
Basically, you need to make sure the attribute exists before you can compare something to it. Your condition expression string should be
does_not_exist(current_version) or current_version = expected_current_version
Using Boto3, you can create this using
Attr(result.get_version_key()).not_exists() | Attr(result.get_version_key().eq(current_version))
Related
I'm attempting to send a batch of PartiQL statements in the NodeJS AWS SDK v3. The statement works fine for a single ExecuteStatementCommand, but the Batch command doesn't.
The statement looks like
const statement = `
SELECT *
FROM "my-table"
WHERE "partitionKey" = '1234'
AND "filterKey" = '5678'
`
This code snippet works as expected:
const result = await dynamodbClient.send(new ExecuteStatementCommand(
{ Statement: statement}
))
The batch snippet does not:
const result = await dynamodbClient.send(new BatchExecuteStatementCommand({
Statements: [
{
Statement: statement
}
]
}))
The batch call produces the following error:
"Code": "ValidationError",
"Message": "Select statements within BatchExecuteStatement must specify the primary key in the where clause."
Any insight is greatly appreciated. Thanks for taking the time to reading my question!
Seems like what I needed was a rubber duck.
DynamoDB primary keys consists of partition key + sort key. My particular table has a sort key, which is missing from the statement. Batch jobs cannot handle filtering of responses, and each statement must match a single item in the database.
I'm having some strange feeling abour sqlite3 parameters that I would like to expose to you.
This is my query and the fail message :
#query
'SELECT id FROM ? WHERE key = ? AND (userid = '0' OR userid = ?) ORDER BY userid DESC LIMIT 1;'
#error message, fails when calling sqlite3_prepare()
error: 'near "?": syntax error'
In my code it looks like:
// Query is a helper class, at creation it does an sqlite3_preprare()
Query q("SELECT id FROM ? WHERE key = ? AND (userid = 0 OR userid = ?) ORDER BY userid DESC LIMIT 1;");
// bind arguments
q.bindString(1, _db_name.c_str() ); // class member, the table name
q.bindString(2, key.c_str()); // function argument (std::string)
q.bindInt (3, currentID); // function argument (int)
q.execute();
I have the feeling that I can't use sqlite parameters for the table name, but I can't find the confirmation in the Sqlite3 C API.
Do you know what's wrong with my query?
Do I have to pre-process my SQL statement to include the table name before preparing the query?
Ooookay, should have looked more thoroughly on SO.
Answers:
- SQLite Parameters - Not allowing tablename as parameter
- Variable table name in sqlite
They are meant for Python, but I guess the same applies for C++.
tl;dr:
You can't pass the table name as a parameter.
If anyone have a link in the SQLite documentation where I have the confirmation of this, I'll gladly accept the answer.
I know this is super old already but since your query is just a string you can always append the table name like this in C++:
std::string queryString = "SELECT id FROM " + std::string(_db_name);
or in objective-C:
[#"SELECT id FROM " stringByAppendingString:_db_name];
I am having trouble using AWS Boto3 to query DynamoDB with a hash key and a range key at the same time using the recommend KeyConditionExpression. I have attached an example query:
import boto3
from boto3 import dynamodb
from boto3.session import Session
dynamodb_session = Session(aws_access_key_id=AWS_KEY,
aws_secret_access_key=AWS_PASS,
region_name=DYNAMODB_REGION)
dynamodb = dynamodb_session.resource('dynamodb')
table=dynamodb.Table(TABLE_NAME)
request = {
'ExpressionAttributeNames': {
'#n0': 'hash_key',
'#n1': 'range_key'
},
'ExpressionAttributeValues': {
':v0': {'S': MY_HASH_KEY},
':v1': {'N': GT_RANGE_KEY}
},
'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
'TableName': TABLE_NAME
}
response = table.query(**request)
When I run this against a table with the following scheme:
Table Name: TABLE_NAME
Primary Hash Key: hash_key (String)
Primary Range Key: range_key (Number)
I get the following error and I cannot understand why:
ClientError: An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: Incorrect operand type for operator or function; operator or function: >, operand type: M
From my understanding the type M would be a map or dictionary type and I am using a type N which is a number type and matches my table scheme for the range key. If someone could explain why this error is happening or I am also open to a different way of accomplishing the same query even if you cannot explain why this error exists.
The Boto 3 SDK constructs a Condition Expression for you when you use the Key and Attr functions imported from boto3.dynamodb.conditions:
response = table.query(
KeyConditionExpression=Key('hash_key').eq(hash_value) & Key('range_key').eq(range_key_value)
)
Reference: Step 4: Query and Scan the Data
Hope it helps
Adding this solution as the accepted answer did not address why the query used did not work.
TLDR: Using query on a Table resource in boto3 has subtle differences as opposed to using client.query(...) and requires a different syntax.
The syntax is valid for a query on a client, but not on a Table. The ExpressionAttributeValues on a table do not require you to specify the data type. Also if you are executing a query on a Table resource you do not have to specify the TableName again.
Working solution:
from boto3.session import Session
dynamodb_session = Session(aws_access_key_id=AWS_KEY,aws_secret_access_key=AWS_PASS,region_name=DYNAMODB_REGION)
dynamodb = dynamodb_session.resource('dynamodb')
table = dynamodb.Table(TABLE_NAME)
request = {
'ExpressionAttributeNames': {
'#n0': 'hash_key',
'#n1': 'range_key'
},
'ExpressionAttributeValues': {
':v0': MY_HASH_KEY,
':v1': GT_RANGE_KEY
},
'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
}
response = table.query(**request)
I am the author of a package called botoful which might be useful to avoid dealing with these complexities. The code using botoful will be as follows:
import boto3
from botoful import Query
client = boto3.Session(
aws_access_key_id=AWS_KEY,
aws_secret_access_key=AWS_PASS,
region_name=DYNAMODB_REGION
).client('dynamodb')
results = (
Query(TABLE_NAME)
.key(hash_key=MY_HASH_KEY, range_key__gt=GT_RANGE_KEY)
.execute(client)
)
print(results.items)
I've got a function that I'd like use to delete a row in my database. This is the only way I've used the DELETE statement to remove a row before but I want the 1 to be replaced by a variable called recID so that the value of recID is the row ID number which is deleted. So if recID = 6, I want the function to delete the row with ID = 6. I hope that makes sense.
'DELETE FROM MyRecords WHERE ID=1';
The notation I've been using is the following, if it helps or makes any difference.
db.transaction(function(transaction) {
transaction.executeSql( //DELETE STATEMENT HERE );
});
executeSql supports arguments (check definition).
Use it like:
db.transaction(function(transaction) {
transaction.executeSql("DELETE FROM MyRecords WHERE ID=?", [recId]);
});
If you're certain that your variable, recID, will only ever contain numbers, you can just use:
transaction.executeSql("DELETE FROM MyRecords WHERE ID=" + recID);
If recID comes from outside your application (user input) however, it either needs to be sanitized, or use a prepared statement and use the database API to set the parameter after the statement has been prepared. Otherwise you open yourself up to SQL injection attacks.
I don't know the details of your SQLite wrapper, or what version of SQLite it wraps, but creating a prepared statement using the SQLite3 C API would go something like this:
// sqlite3* db = ...
sqlite3_stmt* stmt;
sqlite3_prepare_v2(db, "DELETE FROM MyRecords WHERE ID=?", -1, &stmt, 0);
sqlite3_bind_int(stmt, 1, recID);
sqlite3_step();
// ...
sqlite3_finalize(stmt);
This simple example excludes all the error checking you'd want to do in a real application, but since you're using a wrapper that has different syntax anyway, you'd have to figure out how it wraps these functions anyway.
I'm trying to make an insertion from one database called suspension to the table called Notification in the ANimals database. My stored procedure is this:
ALTER PROCEDURE [dbo].[spCreateNotification]
-- Add the parameters for the stored procedure here
#notRecID int,
#notName nvarchar(50),
#notRecStatus nvarchar(1),
#notAdded smalldatetime,
#notByWho int
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
INSERT INTO Animals.dbo.Notification
(
NotRecID,
NotName,
NotRecStatus,
NotAdded,
NotByWho
)
values (#notRecID, #notName, #notRecStatus, #notAdded, #notByWho);
END
The null inserting is to replenish one column that otherwise will not be filled, I've tried different ways, like using also the names for the columns after the name of the table and then only indicate in values the fields I've got. I know it is not a problem of the stored procedure because I executed it from the sql server management studio and it works introducing the parameters. Then I guess the problem must be in the repository when I call the stored procedure:
public void createNotification(Notification not)
{
try
{
DB.spCreateNotification(not.NotRecID, not.NotName, not.NotRecStatus,
(DateTime)not.NotAdded, (int)not.NotByWho);
}
catch
{
return;
}
}
And I call the method here:
public void createNotifications(IList<TemporalNotification> notifications)
{
foreach (var TNot in notifications)
{
var ts = RepositoryService._suspension.getTemporalSuspensionForNotificationID(TNot.TNotRecID);
Notification notification = new Notification();
if (ts.Count != 0)
{
notification.NotName = TNot.TNotName;
notification.NotRecID = TNot.TNotRecID;
notification.NotRecStatus = TNot.TNotRecStatus;
notification.NotAdded = TNot.TNotAdded;
notification.NotByWho = TNot.TNotByWho;
if (TNot.TNotToReplace != 0)
{
var suspensions = RepositoryService._suspension.getSuspensionsAttached((int)TNot.TNotToReplace);
foreach (var sus in suspensions)
{
sus.CtsEndDate = TNot.TNotAdded;
sus.CtsEndNotRecID = TNot.TNotRecID;
DB.spModifySuspensionWhenNotificationIsReplaced((int)TNot.TNotToReplace, (int)sus.CtsEndNotRecID, (DateTime) sus.CtsEndDate);
}
DB.spReplaceNotification((int)TNot.TNotToReplace, DateTime.Now);
createNotification(notification);
}
else
{
createNotification(notification);
}
}
}
deleteTemporalNotifications(notifications);
}
It does not record the value in the database. I've been debugging and getting mad about this, because it works when I execute it manually, but not when I automatize the proccess in my application. Does anyone see anything wrong with my code?
Thank you
EDIT: Added more code. It still doesn't work changing that, I mean, the procedure works if I execute it, so I don't know what could be the error. In fact, I don't get any error. Could it be a matter of writin in a table that is not in the database where you have your stored procedure?
I would specify your column names and DONT incude the NULL at all for that column. Just let SQL Server deal with it.
INSERT INTO Animals.dbo.Notification
(
RecID,
[Name],
RecStatus,
Added,
ByWho
)
values (#notRecID, #notName, #notRecStatus, #notAdded, #notByWho);
Run profiler when you try to run it from the application and see what values it realy is sending. That will tell you if the application is creating the correct exec statment to exec the proc.
Also it may be a permissions problem.
Specify your column names:
INSERT INTO Animals.dbo.Notification
(RecID, Name, RecStatus, Added, ByWho)
VALUES
(#notRecID, #notName, #notRecStatus, #notAdded, #notByWho);
"Could it be a matter of writin in a table that is not in the database where you have your stored procedure?"
That may be the problem. You could try adding the "WITH EXECUTE AS OWNER" clause to your stored procedure so that it executes as the owner of the stored procedure. Or grant write permissions for the executing user to the table.
http://msdn.microsoft.com/en-us/library/ms188354.aspx
ok, I finally found out what noone realized lol. It was a very stupid error but got me really mad till I found the problem. It wasn't a problem of permissions, the problem was that I was not executing the procedure from my application, so where I wrote this:
DB.spCreateNotification(not.NotRecID, not.NotName, not.NotRecStatus,
(DateTime)not.NotAdded, (int)not.NotByWho);
When I had to write:
DB.spCreateNotification(not.NotRecID, not.NotName, not.NotRecStatus,
(DateTime)not.NotAdded, (int)not.NotByWho).Execute();
so as you see I was focusing my efforts in much more complex things and I wasn't even executing it...lol.
Thank you all for your answers anyway:)