There is a property in FeedOptions class named PopulateQueryMetrics in Document DB SDK 2.#. What is the equivalent in Cosmos SDK 3.#?
IDocumentQuery<dynamic> documentQuery = documentClient.CreateDocumentQuery<dynamic>(
collectionUri,
sqlQuery,
new FeedOptions
{
EnableCrossPartitionQuery = true,
PopulateQueryMetrics = true
});
You don't need to set this flag anymore, query metrics (plus other important network information and latency metrics) are captured in the FeedResponse.Diagnostics. Reference: https://learn.microsoft.com/dotnet/api/microsoft.azure.cosmos.response-1.diagnostics?view=azure-dotnet#Microsoft_Azure_Cosmos_Response_1_Diagnostics
Just capture this property's value ToString(), for example:
FeedResponse<T> response = await iterator.ReadNextAsync();
Console.WriteLine(response.Diagnostics.ToString());
Related
I have a project that uses CloudTableClient and query the cosmos db in this way:
var table = cloudTableClient.GetTableReference(tableName);
var cosmosResult = await table.ExecuteQuerySegmentedAsync(GetTableQuery<DynamicTableEntity>(queryOptions), tableContinuationToken, GetTableRequestOptions(requestOptions), operationContext);
If I use CosmosClient, I can set the ConnetionMode
CosmosClient client = new CosmosClient(connectionString,
new CosmosClientOptions
{
ConnectionMode = ConnectionMode.Gateway // ConnectionMode.Direct is the default
});
However, with CouldTableClient, seems I can't find an option to set this. Is it possible to use Direct Mode with CouldTableClient or I actually need to move everything to CosmosClient in order to do it.
All these days, in our cosmos db we had non partitioned collections & recently moved our app data to partitioned collections to overcome 10gb cap on single partition.
Few things we have noticed right after introducing partitions.
ResourceResponse.ContentLocation property returns null. (usually it returns collection path like "dbs/developmentdb/colls/accountmodel" as value with non partitioned collections)
For "GetAll" query (provided same data maintained in both partitioned and non partitioned collections)
RUs went up (from 400RUs to 750RUs)
Slower response time
For your reference included below, the code used. Appreciate any of your suggestions to reduce RUs & improve response time (OR) these all are overheads of moving to partitioned collection? please suggest
Sample code used:
var docClient = await _documentClient;
var docDb = await _documentDatabase;
var docCollection = await _documentCollection;
var queryFeed = new FeedOptions()
{
MaxItemCount = -1,
MaxDegreeOfParallelism = -1,
EnableCrossPartitionQuery = true
};
var documentCollectionUri = UriFactory.CreateDocumentCollectionUri(docDb.Id, docCollection.Id);
IDocumentQuery<T> query = docClient.CreateDocumentQuery<T>(documentCollectionUri, queryFeed).AsDocumentQuery();
while (query.HasMoreResults)
{
var page = await query.ExecuteNextAsync<T>();
result.AddRange(page);
_rULogHelper.LogFromFeedResponse(page, docDb.Id, docCollection.Id, DBOperationType.GET.ToString()); //custom logging related code
}
I have an error when I'm trying to use BulkExecutor to update one of the properties in CosmosDb. The error message is "Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index"
Important point- I don't have partition key defined on my collection.
Here is my code:
SetUpdateOperation<string> player1NameUpdateOperation = new SetUpdateOperation<string>("Player1Name", name);
var updateOperations = new List<UpdateOperation>();
updateOperations.Add(player1NameUpdateOperation);
var updateItems = new List<UpdateItem>();
foreach (var match in list)
{
string id = match.id;
updateItems.Add(new UpdateItem(id, null, updateOperations));
}
var executor = new Microsoft.Azure.CosmosDB.BulkExecutor.BulkExecutor(_client, _collection);
await executor.InitializeAsync();
var executeResult = await executor.BulkUpdateAsync(updateItems);
var count = executeResult.NumberOfDocumentsUpdated;
What am I missing?
If I run the bulk executor on a collection without a partition key, I get the same error. If I run it with a collection that does have it and i specify it, the bulk executor works fine.
Pretty sure they just don't support it right now through the bulk executor api, just use the normal cosmos api for updating the doc as a workaround for now.
1) I have a Cosmos DB collection with about 500k documents and which is Partitioned by a property "SITEID". In the Query Request Options only one partition key value can be passed. In my case I have queries where the SITEID in (1,2,3,4) needs to be executed where SiteID is the partition key.
For example, my SP is as follows:
SELECT * FROM c WHERE c.SITEID IN
("SiteId1","SiteId2","SiteId3","SiteId4","SiteId5")
AND c.STATUS IN ("Status1","Status2","Status3","Status4")
I am Calling the above SP using the below SQL API code.
await client.ExecuteStoredProcedureAsync<string>(UriFactory.CreateStoredProcedureUri("DBName", "CollectionName", "Sample"),new RequestOptions { PartitionKey = new PartitionKey("SiteId1") })
In the above SQL API Code, PartitionKey property only supports a Single value. Where I need to pass several partition values. Is there any other options to do this?
2) "EnableCrossPartitionQuery" property is only availbale in the FeedOptions but not in the Request Options class. Client.ExecuteStoredProcedureAsync only supports the RequestOptions parameter not FeedOptions. Now I need to execute a Stored Procedure at once and across all partitions. Is there any other options to pass EnableCrossPartitionQuery in ExecuteStoredProcedureAsync method.
E.g)
client.CreateDocumentQuery<Doc>(UriFactory.CreateDocumentCollectionUri("DBName", "CollectionName"), "select * from c", new FeedOptions { EnableCrossPartitionQuery = true }).ToList()
await client.ExecuteStoredProcedureAsync<string>(UriFactory.CreateStoredProcedureUri("DBName", "CollectionName", "Sample"),new RequestOptions { PartitionKey = new PartitionKey("WGC") })
Stored procedures can only be executed against a single partition. There is nothing you can do about that.
They are not considered a query that returns a feed but a request that could return a response of any type. That's they they don't used the FeedOptions but rather the RequestOptions.
You can still execute your query as a normal document query and set the EnableCrossPartitionQuery to true. Cosmos should recognise the partition key in the query and should limit the requests to the specific partition key values.
I say should because this answer suggests that this is the case but there are some comments that say otherwise. I would suggest you check your metrics regarding the amount of collection hits.
I want to find out details about a Gremlim query - so I set the PopulateQueryMetrics property of the FeedOptions argument to true.
But the FeedResponse object I get back doesn't have the QueryMetrics property populated.
var queryString = $"g.addV('{d.type}').property('id', '{d.Id}')";
var query = client.CreateGremlinQuery<dynamic>(graphCollection, queryString,
new FeedOptions {
PopulateQueryMetrics = true
});
while (query.HasMoreResults)
{
FeedResponse<dynamic> response = await query.ExecuteNextAsync();
//response.QueryMetrics is null
}
Am I missing something?
According to your description, I created my Azure Cosmos DB account with Gremlin (graph) API, and I could encounter the same issue as you mentioned. I found a tutorial Monitoring and debugging with metrics in Azure Cosmos DB and read the Debugging why queries are running slow section as follows:
In the SQL API SDKs, Azure Cosmos DB provides query execution statistics.
IDocumentQuery<dynamic> query = client.CreateDocumentQuery(
UriFactory.CreateDocumentCollectionUri(DatabaseName, CollectionName),
“SELECT * FROM c WHERE c.city = ‘Seattle’”,
new FeedOptions
{
PopulateQueryMetrics = true,
MaxItemCount = -1,
MaxDegreeOfParallelism = -1,
EnableCrossPartitionQuery = true
}).AsDocumentQuery();
FeedResponse<dynamic> result = await query.ExecuteNextAsync();
// Returns metrics by partition key range Id
IReadOnlyDictionary<string, QueryMetrics> metrics = result.QueryMetrics;
Then, I queried my Cosmos DB Gremlin (graph) account via the SQL API above, I retrieved the QueryMetrics as follows:
Note: I also checked that you could specify the SQL expression like this SELECT * FROM c where c.id='thomas' and c.label='person'. For adding new Vertex, I do not know how to construct the SQL expression. Moreover, the CreateDocumentAsync method does not support the FeedOptions parameter.
Per my understanding, the PopulateQueryMetrics setting may only work when using the SQL API. You could add your feedback here.