I created a container to store user information for customers. The partitionKey is customerId. I want to read all items in a logical partition with customerId = X, that should return all user records with customerId = X. Is such an API available in CosmosClient in .NET SDK?
Example :
class User
{
string customerId,
string userId,
string userName
}
Assuming you are using SQL API, you can use Azure Cosmos DB SDK to fetch all the Users for your partition key. Here is the sample code:
var client = new CosmosClient("COSMOS_CONNECTION_STRING");
var container = client.GetContainer("DATABASE_NAME", "CONTAINER_NAME");
var queryDefinition = new QueryDefinition("SELECT * FROM c");
var iterator = container.GetItemQueryIterator<User>(queryDefintion,
requestOptions: new QueryRequestOptions()
{
PartitionKey = new PartitionKey("CUSTOMER_ID")
});
var results = new List<User>();
while (iterator.HasMoreResults)
{
var result = await iterator.ReadNextAsync();
results.AddRange(result.Resource);
}
return results;
Related
I am using Microsoft.Powerplatform.Dataverse.Client to interact with Dataverse, i want to fetch parent and child records using single query, is this possible? Please suggest how to achieve this.
var client = _client.GetServiceClient();
if (client.IsReady)
{
var queryExpression = new QueryExpression("ParentTable")
{
ColumnSet = new ColumnSet(columns),
NoLock = true,
TopCount = 1,
Criteria = GetFilterExpression(filterColumns),
};
var parentCollectionResult = await client.RetrieveMultipleAsync(queryExpression);
entities = parentCollectionResult.Entities;
}
When trying to read from CosmosDb i can select a document via:
Id Query
Id + PartitionKey Query
but how do i select data from CosmosDb when i only have the PartitionKey?
using Microsoft.Azure.Cosmos;
public class CosmosDbService : ICosmosDbService
{
private Container _container;
public CosmosDbService(
CosmosClient cosmosDbClient,
string databaseName,
string containerName)
{
_container = cosmosDbClient.GetContainer(databaseName, containerName);
}
public async Task<Error> GetItemAsync(string partitionKey)
{
// selection only via partitionKey - does not work
var response = await _container.ReadItemAsync<Error>(partitionKey, new PartitionKey(partitionKey));
return response.Resource;
// below one works as i am passing the Id (internally generated by CosmosDB)
var id = "2e4e5727-86ff-4c67-84a6-184b4716d744";
var response = await _container.ReadItemAsync<Error>(id, new PartitionKey(partitionKey));
return response.Resource;
}
}
Question:
Are there any other methods in CosmosDB client which can return the document using the PartitionKey ONLY without the need of Id which I don't know ?
When selecting documents you could try to use QueryDefinition + QueryAsync:
var query = new QueryDefinition("select top 1 * from c");
var partitionKey = "PARTITIONKEY";
var resultSet = container.GetItemQueryIterator<ModelObject>(query, null, new QueryRequestOptions { PartitionKey = new PartitionKey(partitionKey) });
var result = new List<ModelObject>();
while (resultSet.HasMoreResults)
{
var item = await resultSet.ReadNextAsync(ct /* CancellationToken */).ConfigureAwait(false);
var itemList = item.ToList();
result.AddRange(itemList);
}
Instead of a top 1 select you could also do a select * (for example)
I have a dynamo db table CustomerOrders with following fields
Primary partition key: CustomerId (Number)
Primary sort key: DepartmentId (Number)
Order (Serialized Json String)
I would like to do a query on multiple customers in one request without using Sort Key (DepartmentId). So I created a Global Secondary Index on CustomerId and would like to use that to query just using the CustomerId. I see documentation only related to BatchGetItemAsync for running batch queries. I don't see a way to set the IndexName on a BatchGetItemRequest. How can that be done?
Below is my code segment so far:
public async Task<List<CustomerOrder>> GetOrdersAsync(List<int> customerIds)
{
var orders = new List<CustomerOrder>();
var tableKeys = new List<Dictionary<string, AttributeValue>>();
foreach (var x in customerIds)
{
tableKeys.Add(new Dictionary<string, AttributeValue> { { "CustomerId", new AttributeValue { N = x.ToString() } } });
}
var dynamoTable = $"CustomerOrders";
var keysAndAttributes = new KeysAndAttributes
{
AttributesToGet = new List<string> { "CustomerId", "DepartmentId", "Order" },
Keys = tableKeys
};
var request = new BatchGetItemRequest
{
ReturnConsumedCapacity = ReturnConsumedCapacity.INDEXES, // Not sure what this does
RequestItems = new Dictionary<string, KeysAndAttributes> { { dynamoTable, keysAndAttributes } }
};
BatchGetItemResponse result;
do
{
result = await dynamoDbClient.BatchGetItemAsync(request); // Exception gets thrown from here
var responses = result.Responses;
foreach (var tableName in responses.Keys)
{
var tableItems = responses[tableName];
foreach (var item in tableItems)
{
orders.Add(new CustomerOrder
{
CustomerId = int.Parse(item["CustomerId"].N),
DepartmentId = int.Parse(item["DepartmentId"].N),
Order = JsonConvert.DeserializeObject<Order>(item["Order"].S)
});
}
}
// Set RequestItems to the result's UnprocessedKeys and reissue request
request.RequestItems = result.UnprocessedKeys;
} while (result.UnprocessedKeys.Count > 0);
return orders;
}
I am getting The provided key element does not match the schema error with the above code. Please help!
You can't "set the IndexName on a BatchGetItemRequest"
In fact, you can't GetItem() on a GSI/LSI either. GetItem() only works on the table.
And GetItem() always requires the full primary key.
With a partial key, you'd need to perform multiple Query(), one for each hash key.
The GSI isn't doing anything for you. Department as a sort key really isn't doing anything for you either since I assume customerId is unique.
A better structure might have been to have the table defined with only hash key for the primary key;
I have the following linq expression, I am trying to create a new record in to address table and Update customer table with the newly created AdressID, How can i create a new address get the new AddressID and update customer table
if (request == null)
throw new ArgumentNullException(nameof(request));
if (request.AddressToCreate == null)
throw new ArgumentNullException(nameof(request.AddressToCreate));
var address = Mapper.Map<Address>(request.AddressToCreate);
address.CreatedBy = request.AddressToCreate.CreatedBy;
address.CreatedDate = SystemClock.UtcNow;
address.UpdatedBy = request.AddressToCreate.UpdatedBy;
address.UpdatedDate = SystemClock.UtcNow;
await Context.AddAsync(address);
var rps = Context.Customers .Where(rc => rc.ID == request.ID).SingleOrDefault();
rps.AddressID = request.AddressToCreate.ID;
await Context.SaveChangesAsync();
But request.AddressToCreate.ID; returns 0, How can i modify my code to get the New AddressID
I would recommend this:
Firstly you may not have an Id property to your AddressToCreate.
Secondly AddAsync() may not call Context.SaveChangesAsync() so the entity is not commited to database and the id is 0.
Try it like this;
await Context.AddAsync(address);
await Context.SaveChangesAsync(); // this is optional only if you have modified AddAsync() and it doesnt call SaveChangesAsync inside
var rps = Context.Customers.FirstOrDefault(rc => rc.ID == request.ID);
rps.AddressID = address.ID;
await Context.UpdateAsync(rps);
await Context.SaveChangesAsync();
I have a dynamoDB table with a schema as follows:
var request = new CreateTableRequest
{
TableName = tableName,
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("CompanyId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
AttributeDefinitions = new List<AttributeDefinition>
{
new AttributeDefinition("CompanyId", ScalarAttributeType.S),
new AttributeDefinition("Timestamp", ScalarAttributeType.N),
new AttributeDefinition("UserId", ScalarAttributeType.S)
},
GlobalSecondaryIndexes = new List<GlobalSecondaryIndex>
{
new GlobalSecondaryIndex
{
IndexName = "UserIndex",
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("UserId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
Projection = new Projection {ProjectionType = "ALL"},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
}
},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
};
I can query the primary key successfully as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object>{minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(companyId, QueryOperator.GreaterThanOrEqual, sortKeyValues,
new DynamoDBOperationConfig {OverrideTableName = TableName}).GetRemainingAsync();
}
And I can query the global secondary index without any constraint on the range key as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
result = await context.QueryAsync<AuditLogEntry>(userId, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName})
.GetRemainingAsync();
}
But when I try to query the index with a range key constraint:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object> {minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(userId, QueryOperator.GreaterThan, sortKeyValues, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName}).GetRemainingAsync();
}
I get the following error:
Exception thrown: 'System.InvalidOperationException' in AWSSDK.DynamoDBv2.dll
Additional information: Local Secondary Index range key conditions are used but no index could be inferred from model. Specified index name = UserIndex
Googling this error hasn't thrown any light on the issue. The reference to Local Secondary Index has me confused because I'm using a Global index, but I just can't see what's wrong with my code.
I've been able to get the query working by querying directly on the AmazonDynamoDBClient rather than using DynamoDBContext, but I'd really like to understand what I'm doing wrong and be able to use DynamoDBContext.
Any ideas would be appreciated.
In your model definition for AuditLogEntry you need to decorate properties that are part of the global secondary index with attributes - [DynamoDBGlobalSecondaryIndexRangeKey] and or [DynamoDBGlobalSecondaryIndexHashKey]. Example below.
public class AuditLogEntry {
// other properties ...
[DynamoDBProperty("UserId")]
[DynamoDBGlobalSecondaryIndexHashKey("UserIndex")]
public string UserId { get; set; }
}