When trying to read from CosmosDb i can select a document via:
Id Query
Id + PartitionKey Query
but how do i select data from CosmosDb when i only have the PartitionKey?
using Microsoft.Azure.Cosmos;
public class CosmosDbService : ICosmosDbService
{
private Container _container;
public CosmosDbService(
CosmosClient cosmosDbClient,
string databaseName,
string containerName)
{
_container = cosmosDbClient.GetContainer(databaseName, containerName);
}
public async Task<Error> GetItemAsync(string partitionKey)
{
// selection only via partitionKey - does not work
var response = await _container.ReadItemAsync<Error>(partitionKey, new PartitionKey(partitionKey));
return response.Resource;
// below one works as i am passing the Id (internally generated by CosmosDB)
var id = "2e4e5727-86ff-4c67-84a6-184b4716d744";
var response = await _container.ReadItemAsync<Error>(id, new PartitionKey(partitionKey));
return response.Resource;
}
}
Question:
Are there any other methods in CosmosDB client which can return the document using the PartitionKey ONLY without the need of Id which I don't know ?
When selecting documents you could try to use QueryDefinition + QueryAsync:
var query = new QueryDefinition("select top 1 * from c");
var partitionKey = "PARTITIONKEY";
var resultSet = container.GetItemQueryIterator<ModelObject>(query, null, new QueryRequestOptions { PartitionKey = new PartitionKey(partitionKey) });
var result = new List<ModelObject>();
while (resultSet.HasMoreResults)
{
var item = await resultSet.ReadNextAsync(ct /* CancellationToken */).ConfigureAwait(false);
var itemList = item.ToList();
result.AddRange(itemList);
}
Instead of a top 1 select you could also do a select * (for example)
Related
I created a container to store user information for customers. The partitionKey is customerId. I want to read all items in a logical partition with customerId = X, that should return all user records with customerId = X. Is such an API available in CosmosClient in .NET SDK?
Example :
class User
{
string customerId,
string userId,
string userName
}
Assuming you are using SQL API, you can use Azure Cosmos DB SDK to fetch all the Users for your partition key. Here is the sample code:
var client = new CosmosClient("COSMOS_CONNECTION_STRING");
var container = client.GetContainer("DATABASE_NAME", "CONTAINER_NAME");
var queryDefinition = new QueryDefinition("SELECT * FROM c");
var iterator = container.GetItemQueryIterator<User>(queryDefintion,
requestOptions: new QueryRequestOptions()
{
PartitionKey = new PartitionKey("CUSTOMER_ID")
});
var results = new List<User>();
while (iterator.HasMoreResults)
{
var result = await iterator.ReadNextAsync();
results.AddRange(result.Resource);
}
return results;
I have a dynamo db table CustomerOrders with following fields
Primary partition key: CustomerId (Number)
Primary sort key: DepartmentId (Number)
Order (Serialized Json String)
I would like to do a query on multiple customers in one request without using Sort Key (DepartmentId). So I created a Global Secondary Index on CustomerId and would like to use that to query just using the CustomerId. I see documentation only related to BatchGetItemAsync for running batch queries. I don't see a way to set the IndexName on a BatchGetItemRequest. How can that be done?
Below is my code segment so far:
public async Task<List<CustomerOrder>> GetOrdersAsync(List<int> customerIds)
{
var orders = new List<CustomerOrder>();
var tableKeys = new List<Dictionary<string, AttributeValue>>();
foreach (var x in customerIds)
{
tableKeys.Add(new Dictionary<string, AttributeValue> { { "CustomerId", new AttributeValue { N = x.ToString() } } });
}
var dynamoTable = $"CustomerOrders";
var keysAndAttributes = new KeysAndAttributes
{
AttributesToGet = new List<string> { "CustomerId", "DepartmentId", "Order" },
Keys = tableKeys
};
var request = new BatchGetItemRequest
{
ReturnConsumedCapacity = ReturnConsumedCapacity.INDEXES, // Not sure what this does
RequestItems = new Dictionary<string, KeysAndAttributes> { { dynamoTable, keysAndAttributes } }
};
BatchGetItemResponse result;
do
{
result = await dynamoDbClient.BatchGetItemAsync(request); // Exception gets thrown from here
var responses = result.Responses;
foreach (var tableName in responses.Keys)
{
var tableItems = responses[tableName];
foreach (var item in tableItems)
{
orders.Add(new CustomerOrder
{
CustomerId = int.Parse(item["CustomerId"].N),
DepartmentId = int.Parse(item["DepartmentId"].N),
Order = JsonConvert.DeserializeObject<Order>(item["Order"].S)
});
}
}
// Set RequestItems to the result's UnprocessedKeys and reissue request
request.RequestItems = result.UnprocessedKeys;
} while (result.UnprocessedKeys.Count > 0);
return orders;
}
I am getting The provided key element does not match the schema error with the above code. Please help!
You can't "set the IndexName on a BatchGetItemRequest"
In fact, you can't GetItem() on a GSI/LSI either. GetItem() only works on the table.
And GetItem() always requires the full primary key.
With a partial key, you'd need to perform multiple Query(), one for each hash key.
The GSI isn't doing anything for you. Department as a sort key really isn't doing anything for you either since I assume customerId is unique.
A better structure might have been to have the table defined with only hash key for the primary key;
I am implementing a multi-tenant application using cosmosDB. I am using partition keys to separate multiple users data. Following best practices i am trying to allow each tenant to have its own db access token.
I create a user and permission and use the created token to access the partition. But I get the following error:
Partition key provided either doesn't correspond to definition in the collection or doesn't match partition key field values specified
in the document.
ActivityId: 1659037a-118a-4a2d-8615-bb807b717fa7, Microsoft.Azure.Documents.Common/1.22.0.0, Windows/10.0.17134
documentdb-netcore-sdk/1.9.1
My code goes as follows:
Constructor Initiates the client
public Projects (CosmosDbConfig cosmosConfig)
{
config = cosmosConfig;
client = new DocumentClient(new Uri(config.Endpoint), config.AuthKey);
collectionUri = UriFactory.CreateDocumentCollectionUri(config.Database, config.Collection);
config.AuthKey = GetUserToken().Result;;
client = new DocumentClient(new Uri(config.Endpoint), config.AuthKey);
}
The get user function creates the user and retrieves the token. User Ids are partition keys.
private async Task<string> GetUserToken()
{
User user = null;
try
{
try
{
user = await client.ReadUserAsync(UriFactory.CreateUserUri(config.Database, config.PartitionKey));
var permission = await GetorCreatePermission(user, config.Collection, config.PartitionKey);
return permission.Token;
}
catch (Exception ex) {
Console.WriteLine(ex.Message);
}
if (user == null)
{
user = new User
{
Id = config.PartitionKey
};
user = await client.CreateUserAsync(UriFactory.CreateDatabaseUri(config.Database), user);
var permission = await GetorCreatePermission(user, config.Collection, config.PartitionKey);
return permission.Token;
}
else
{
throw new Exception("");
}
}
catch (Exception ex)
{
throw ex;
}
}
Permission are done per collections and holds the collection name as ID since Ids are unique per user.
private async Task<Permission> GetorCreatePermission(User user,
string collection,
string paritionKey)
{
var permDefinition = new Permission
{
Id = collection,
PermissionMode = PermissionMode.All,
ResourceLink = collectionUri.OriginalString,
ResourcePartitionKey = new PartitionKey(paritionKey),
};
var perms = client.CreatePermissionQuery(user.PermissionsLink).AsEnumerable().ToList();
var perm = perms.FirstOrDefault(x => x.Id == collection);
if (perm != null)
{
return perm;
}
else
{
var result = await client.CreatePermissionAsync(user.SelfLink, permDefinition);
perm = result.Resource;
return perm;
}
}
The create function utilizes the new client and this where the error occurs.
public async Task<string> Create(Project p)
{
var result = await client.CreateDocumentAsync(collectionUri, p, new RequestOptions()
{ PartitionKey = new PartitionKey(config.PartitionKey),
});
var document = result.Resource;
return document.Id;
}
Since error says that partition key is incorrect i can suggest you try define partition key pathes while creating collection:
var docCollection = new DocumentCollection();
docCollection.Id = config.CollectionName;
docCollection.PartitionKey.Paths.Add(string.Format("/{0}", config.PartitionKey );
collectionUri = UriFactory.CreateDocumentCollectionUri(config.Database, docCollection);
I have a dynamoDB table with a schema as follows:
var request = new CreateTableRequest
{
TableName = tableName,
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("CompanyId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
AttributeDefinitions = new List<AttributeDefinition>
{
new AttributeDefinition("CompanyId", ScalarAttributeType.S),
new AttributeDefinition("Timestamp", ScalarAttributeType.N),
new AttributeDefinition("UserId", ScalarAttributeType.S)
},
GlobalSecondaryIndexes = new List<GlobalSecondaryIndex>
{
new GlobalSecondaryIndex
{
IndexName = "UserIndex",
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("UserId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
Projection = new Projection {ProjectionType = "ALL"},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
}
},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
};
I can query the primary key successfully as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object>{minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(companyId, QueryOperator.GreaterThanOrEqual, sortKeyValues,
new DynamoDBOperationConfig {OverrideTableName = TableName}).GetRemainingAsync();
}
And I can query the global secondary index without any constraint on the range key as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
result = await context.QueryAsync<AuditLogEntry>(userId, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName})
.GetRemainingAsync();
}
But when I try to query the index with a range key constraint:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object> {minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(userId, QueryOperator.GreaterThan, sortKeyValues, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName}).GetRemainingAsync();
}
I get the following error:
Exception thrown: 'System.InvalidOperationException' in AWSSDK.DynamoDBv2.dll
Additional information: Local Secondary Index range key conditions are used but no index could be inferred from model. Specified index name = UserIndex
Googling this error hasn't thrown any light on the issue. The reference to Local Secondary Index has me confused because I'm using a Global index, but I just can't see what's wrong with my code.
I've been able to get the query working by querying directly on the AmazonDynamoDBClient rather than using DynamoDBContext, but I'd really like to understand what I'm doing wrong and be able to use DynamoDBContext.
Any ideas would be appreciated.
In your model definition for AuditLogEntry you need to decorate properties that are part of the global secondary index with attributes - [DynamoDBGlobalSecondaryIndexRangeKey] and or [DynamoDBGlobalSecondaryIndexHashKey]. Example below.
public class AuditLogEntry {
// other properties ...
[DynamoDBProperty("UserId")]
[DynamoDBGlobalSecondaryIndexHashKey("UserIndex")]
public string UserId { get; set; }
}
LINQ to Entities does not recognize the method 'System.String ToString()' method, and this method cannot be translated into a store expression.
public ActionResult PopulateFromDB(string sidx, string sord, int page, int rows)
{
var context = new NerdDinnerEntities();
var jsonData = new
{
total = 1,
page = page,
sord =sord,
records = context.Authors.Count(),
rows = (from n in context.Authors
select new
{ AuthorId = n.AuthorId ,
cell = new string[] { n.AuthorId.ToString(), n.Name.ToString(), n.Location.ToString() }
}).ToList()
};
return Json(jsonData, JsonRequestBehavior.AllowGet);
}
I am writting ToList or Toarray is it not working the error comes :
public ActionResult PopulateFromDB(string sidx, string sord, int page, int rows)
{
var context = new NerdDinnerEntities();
var jsonData = new
{
total = 1,
page = page,
sord =sord,
records = context.Authors.Count(),
rows = (from n in context.Authors
select new
{ AuthorId = n.AuthorId ,
cell = new string[] { n.AuthorId.ToString(), n.Name.ToString(), n.Location.ToString() }
}).ToList()
};
return Json(jsonData,JsonRequestBehavior.AllowGet);
}
From your code I assume your adding a custom property cell for display/storage purposes on the client-side. I would avoid this as your essentially coupling your API call to one particular client. I would suggest you simply return the data required & deal with it at the client-side specifically e.g.
Server
...
select new
{
Id = n.AuthorId,
Name = n.Name,
Location = n.Location
}).ToList();
...
Client
var response = ...
foreach (var author in response)
{
var cell = new string[] { author.Id.ToString(), author.Name, author.Location };
// do something with cell
}
You should try SqlFunctions.StringConvert to convert this, There is no overload for int so you should cast your number to a double or a decimal.
public ActionResult PopulateFromDB(string sidx, string sord, int page, int rows)
{
var context = new NerdDinnerEntities();
var jsonData = new
{
total = 1,
page = page,
sord =sord,
records = context.Authors.Count(),
rows = (from n in context.Authors
select new
{ AuthorId = n.AuthorId ,
cell = new string[] { SqlFunctions.StringConvert((double)n.AuthorId), n.Name, n.Location }
}).ToList()
};
return Json(jsonData,JsonRequestBehavior.AllowGet);
}
You are not using LinqToSql Classes, if you were using that your code should work, but as you mention that you are using LinqToEntity then You should use SqlFunctions.StringConvert to convert to string.