I've got records that were the result of bad data where the Partition Key is null and I need to clean them up, but I've been unsuccessful so far.
Here's what I've tried:
var scriptResult = await _dbClient.ExecuteStoredProcedureAsync<dynamic>(
GetStoredProcLink("BulkDelete"),
new RequestOptions() { PartitionKey = new PartitionKey(""), EnableScriptLogging = true },
"select * from c where c.documentDbType = "SomeValue"");
I've also tried used Undefined.Value as the parameter to new PartitionKey().
I ripped the stored proc from here and haven't changed anything yet.
Note: This is a partitioned collection if it was not obvious (by /companyId)
I just hit this issue when migrating to new database level throughput provision. This is syntax that got me up and running again when my models did not contain the specified partition Key property:
new RequestOptions() {
PartitionKey = new PartitionKey(Undefined.Value)
}
Ref:
https://www.lytzen.name/2016/12/06/find-docs-with-no-partitionkey-in-azure.html
Null, Undefined, and empty string are all different values in Cosmos DB. You need something like: new RequestOptions { PartitionKey = new PartitionKey(null) }
Related
In my case (I happen to have only two types for each entry, so 2 partitions, and the row key is unique) I can write an iterative set of queries going over all possible partitions like this:
TableOperation retrieveOperation = TableOperation.Retrieve<JobStatus>(Mode.GreyScale.Description(), id);
TableResult query = await table.ExecuteAsync(retrieveOperation);
if (query.Result != null)
{
return new OkObjectResult((JobStatus)query.Result);
}
else
{
retrieveOperation = TableOperation.Retrieve<JobStatus>(Mode.Sepia.Description(), id);
query = await table.ExecuteAsync(retrieveOperation);
if (query.Result != null)
{
return new OkObjectResult((JobStatus)query.Result);
}
}
return new NotFoundResult();
The thing is, that is clearly inefficient (imagine if there were hundreds of types!). Does azure storage tables provide an efficient means to query when you know only the row key?
Does azure storage tables provide an efficient means to query when you
know only the row key?
Simple answer to your question is no, there's no efficient way to query table when you only know the RowKey. Table Service will do full table scan going from one partition to another and find entities with matching RowKey.
In your case, you would probably want to use TableQuery to create your query and then either call ExecuteQuery or ExecuteQuerySegmented to get query results.
TableQuery query = new TableQuery().Where("RowKey eq 'Your Row Key'");
var result = table.ExecuteQuery(query);
I am able to get the last inserted item using low level API as showed in code below using .net core
But is it possible to get the last inserted item using high level API such as Document Model or Object Persistence Model?
Prefer Object Persistence Model if possible. I am not able to find a way to do it, also, I would like DynamoDB to query and return the last item only. I understand that I can get a list of items inserted and get the last item myself in memory, but it is not preferable since it require a lot more read and data transfer.
Thanks
public async Task<DailyStockRecordDao> GetTheLastInsertedItem(string tickerSymbol)
{
QueryRequest request = getTheLastItemRequest(tickerSymbol);
var response = await _dynamoDBClient.QueryAsync(request);
return null;
}
private static QueryRequest getTheLastItemRequest(string tickerSymbol)
{
string partitionName = ":v_PartitionKeyName";
var request = new QueryRequest
{
TableName = Constants.TableName,
KeyConditionExpression = $"{Constants.PartitionKeyName} = {partitionName}",
ExpressionAttributeValues = new Dictionary<string, AttributeValue> {
{ $"{partitionName}", new AttributeValue {
S = tickerSymbol
} }
},
// Optional parameter.
ConsistentRead = false,
Limit = 1,
ExclusiveStartKey = null,
ScanIndexForward = false
};
return request;
}
You can have the below data-structure to achieve this.
pk value
----------------------------------------------
key1 value1
key2 value2
key3 value3
latest {key: key1, value:value1}
While doing write, do 2 writes instead of 1. and while reading just do a get using pk=latest.
Can you add one more column called created_counter, and insert the value starting with numeric 1, then 2, then 3 and so on?
Make the created_counter as sort key.
To reverse the order, set the ScanIndexForward parameter to false.
Use limit 1, to get the last inserted data.
I have an error when I'm trying to use BulkExecutor to update one of the properties in CosmosDb. The error message is "Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index"
Important point- I don't have partition key defined on my collection.
Here is my code:
SetUpdateOperation<string> player1NameUpdateOperation = new SetUpdateOperation<string>("Player1Name", name);
var updateOperations = new List<UpdateOperation>();
updateOperations.Add(player1NameUpdateOperation);
var updateItems = new List<UpdateItem>();
foreach (var match in list)
{
string id = match.id;
updateItems.Add(new UpdateItem(id, null, updateOperations));
}
var executor = new Microsoft.Azure.CosmosDB.BulkExecutor.BulkExecutor(_client, _collection);
await executor.InitializeAsync();
var executeResult = await executor.BulkUpdateAsync(updateItems);
var count = executeResult.NumberOfDocumentsUpdated;
What am I missing?
If I run the bulk executor on a collection without a partition key, I get the same error. If I run it with a collection that does have it and i specify it, the bulk executor works fine.
Pretty sure they just don't support it right now through the bulk executor api, just use the normal cosmos api for updating the doc as a workaround for now.
I am new to aws dynamodb so pardon for any silly mistake. I was trying to fetch two columns from my Activity table. Also I wanted to fetch only those columns where partition key starts with some specific string. Partition key has format activity_EnrolledStudentName.(e.g Dance_studentName) So I wanted to fetch all those items from table where activity is Dance. I was trying to use the following query:
public List<StudentDomain> getAllStudents(String activity) {
List<StudentDomain> scanResult = null;
DynamoDBUtil dynamoDBUtil = new DynamoDBUtil();
AmazonDynamoDB dynamoDBClient = dynamoDBUtil.getDynamoDBClient();
DynamoDBMapper mapper = new DynamoDBMapper(dynamoDBClient);
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression();
scanExpression.withProjectionExpression("studentId, ActivitySkills")
.addFilterCondition(STUDENT_PRIMARY_KEY,
new
Condition().withComparisonOperator(ComparisonOperator.BEGINS_WITH)
.withAttributeValueList(new
AttributeValue().withS(activity)));
scanResult = mapper.scan(StudentDomain.class, scanExpression);
return scanResult;
However I am getting the following error when i executed above query.
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: Can not use both expression and non-expression parameters in the same request: Non-expression parameters: {ScanFilter} Expression parameters: {ProjectionExpression} (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: TMS27PABBC2BS3UU7LID731G0FVV4KQNSO5AEMVJF66Q9ASUAAJG)
Can anyone please suggest where I am mistaken and which other query shall i use otherwise?
It's not completely clear what you are trying to achieve but if I understood correctly then you don't need a scan operation for that which actually scans the whole table and afterwords filter the result.
DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable("TableName");
QuerySpec spec = new QuerySpec()
.withKeyConditionExpression("studentId = : ActivitySkills")
ItemCollection<QueryOutcome> items = table.query(spec);
Iterator<Item> iterator = items.iterator();
Item item = null;
while (iterator.hasNext()) {
item = iterator.next();
System.out.println(item.toJSONPretty());
}
The filter expression you are using are intented to be used as a filter for secondary attributes and not the range or partition keys. At least this is my interpretation of the documentation
Please read the query documentation http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryingJavaDocumentAPI.html
How to query range key programmatically in DynamoDB, I am using .Net AWSSDK ,I am able to query on Hash key with below code :
GetItemRequest request = new GetItemRequest
{
TableName = tableName
};
request.Key = new Dictionary<string,AttributeValue>();
request.Key.Add("ID",new AttributeValue { S = PKValue });
GetItemResponse response = client.GetItem(request);
Please suggest,
Thanks in advance.
There are two kinds of primary key in DynamoDB: Hash-only or Hash-Range.
In the above code I guess your table is Hash-only and you use the hash key to retrieve an element with hashkey equals to PKValue.
If your table is in H-R schema and you want to retrieve a specific element with a hashKey and rangeKey, you can reuse the above code and in addition, add the {"RangeKey", new AttributeValue } into your your request.KEY
On the other hand, query means a different thing in DynamoDB. Query will return you a list of rows sorted in some order.