.net Querying a Global Secondary Index in DynamoDB via DynamoDBContext - asp.net

I have a dynamoDB table with a schema as follows:
var request = new CreateTableRequest
{
TableName = tableName,
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("CompanyId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
AttributeDefinitions = new List<AttributeDefinition>
{
new AttributeDefinition("CompanyId", ScalarAttributeType.S),
new AttributeDefinition("Timestamp", ScalarAttributeType.N),
new AttributeDefinition("UserId", ScalarAttributeType.S)
},
GlobalSecondaryIndexes = new List<GlobalSecondaryIndex>
{
new GlobalSecondaryIndex
{
IndexName = "UserIndex",
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("UserId", KeyType.HASH),
new KeySchemaElement("Timestamp", KeyType.RANGE)
},
Projection = new Projection {ProjectionType = "ALL"},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
}
},
ProvisionedThroughput = new ProvisionedThroughput(5, 6)
};
I can query the primary key successfully as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object>{minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(companyId, QueryOperator.GreaterThanOrEqual, sortKeyValues,
new DynamoDBOperationConfig {OverrideTableName = TableName}).GetRemainingAsync();
}
And I can query the global secondary index without any constraint on the range key as follows:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
result = await context.QueryAsync<AuditLogEntry>(userId, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName})
.GetRemainingAsync();
}
But when I try to query the index with a range key constraint:
var client = new AmazonDynamoDBClient();
using (var context = new DynamoDBContext(client))
{
var sortKeyValues = new List<object> {minTimestamp};
result = await context.QueryAsync<AuditLogEntry>(userId, QueryOperator.GreaterThan, sortKeyValues, new DynamoDBOperationConfig {OverrideTableName = TableName, IndexName = indexName}).GetRemainingAsync();
}
I get the following error:
Exception thrown: 'System.InvalidOperationException' in AWSSDK.DynamoDBv2.dll
Additional information: Local Secondary Index range key conditions are used but no index could be inferred from model. Specified index name = UserIndex
Googling this error hasn't thrown any light on the issue. The reference to Local Secondary Index has me confused because I'm using a Global index, but I just can't see what's wrong with my code.
I've been able to get the query working by querying directly on the AmazonDynamoDBClient rather than using DynamoDBContext, but I'd really like to understand what I'm doing wrong and be able to use DynamoDBContext.
Any ideas would be appreciated.

In your model definition for AuditLogEntry you need to decorate properties that are part of the global secondary index with attributes - [DynamoDBGlobalSecondaryIndexRangeKey] and or [DynamoDBGlobalSecondaryIndexHashKey]. Example below.
public class AuditLogEntry {
// other properties ...
[DynamoDBProperty("UserId")]
[DynamoDBGlobalSecondaryIndexHashKey("UserIndex")]
public string UserId { get; set; }
}

Related

How to read data from CosmosDb when i only have the partitionkey but not the id of the document

When trying to read from CosmosDb i can select a document via:
Id Query
Id + PartitionKey Query
but how do i select data from CosmosDb when i only have the PartitionKey?
using Microsoft.Azure.Cosmos;
public class CosmosDbService : ICosmosDbService
{
private Container _container;
public CosmosDbService(
CosmosClient cosmosDbClient,
string databaseName,
string containerName)
{
_container = cosmosDbClient.GetContainer(databaseName, containerName);
}
public async Task<Error> GetItemAsync(string partitionKey)
{
// selection only via partitionKey - does not work
var response = await _container.ReadItemAsync<Error>(partitionKey, new PartitionKey(partitionKey));
return response.Resource;
// below one works as i am passing the Id (internally generated by CosmosDB)
var id = "2e4e5727-86ff-4c67-84a6-184b4716d744";
var response = await _container.ReadItemAsync<Error>(id, new PartitionKey(partitionKey));
return response.Resource;
}
}
Question:
Are there any other methods in CosmosDB client which can return the document using the PartitionKey ONLY without the need of Id which I don't know ?
When selecting documents you could try to use QueryDefinition + QueryAsync:
var query = new QueryDefinition("select top 1 * from c");
var partitionKey = "PARTITIONKEY";
var resultSet = container.GetItemQueryIterator<ModelObject>(query, null, new QueryRequestOptions { PartitionKey = new PartitionKey(partitionKey) });
var result = new List<ModelObject>();
while (resultSet.HasMoreResults)
{
var item = await resultSet.ReadNextAsync(ct /* CancellationToken */).ConfigureAwait(false);
var itemList = item.ToList();
result.AddRange(itemList);
}
Instead of a top 1 select you could also do a select * (for example)

Get multiple records from a dynamo db table using Global Secondary Index

I have a dynamo db table CustomerOrders with following fields
Primary partition key: CustomerId (Number)
Primary sort key: DepartmentId (Number)
Order (Serialized Json String)
I would like to do a query on multiple customers in one request without using Sort Key (DepartmentId). So I created a Global Secondary Index on CustomerId and would like to use that to query just using the CustomerId. I see documentation only related to BatchGetItemAsync for running batch queries. I don't see a way to set the IndexName on a BatchGetItemRequest. How can that be done?
Below is my code segment so far:
public async Task<List<CustomerOrder>> GetOrdersAsync(List<int> customerIds)
{
var orders = new List<CustomerOrder>();
var tableKeys = new List<Dictionary<string, AttributeValue>>();
foreach (var x in customerIds)
{
tableKeys.Add(new Dictionary<string, AttributeValue> { { "CustomerId", new AttributeValue { N = x.ToString() } } });
}
var dynamoTable = $"CustomerOrders";
var keysAndAttributes = new KeysAndAttributes
{
AttributesToGet = new List<string> { "CustomerId", "DepartmentId", "Order" },
Keys = tableKeys
};
var request = new BatchGetItemRequest
{
ReturnConsumedCapacity = ReturnConsumedCapacity.INDEXES, // Not sure what this does
RequestItems = new Dictionary<string, KeysAndAttributes> { { dynamoTable, keysAndAttributes } }
};
BatchGetItemResponse result;
do
{
result = await dynamoDbClient.BatchGetItemAsync(request); // Exception gets thrown from here
var responses = result.Responses;
foreach (var tableName in responses.Keys)
{
var tableItems = responses[tableName];
foreach (var item in tableItems)
{
orders.Add(new CustomerOrder
{
CustomerId = int.Parse(item["CustomerId"].N),
DepartmentId = int.Parse(item["DepartmentId"].N),
Order = JsonConvert.DeserializeObject<Order>(item["Order"].S)
});
}
}
//  Set RequestItems to the result's UnprocessedKeys and reissue request
request.RequestItems = result.UnprocessedKeys;
} while (result.UnprocessedKeys.Count > 0);
return orders;
}
I am getting The provided key element does not match the schema error with the above code. Please help!
You can't "set the IndexName on a BatchGetItemRequest"
In fact, you can't GetItem() on a GSI/LSI either. GetItem() only works on the table.
And GetItem() always requires the full primary key.
With a partial key, you'd need to perform multiple Query(), one for each hash key.
The GSI isn't doing anything for you. Department as a sort key really isn't doing anything for you either since I assume customerId is unique.
A better structure might have been to have the table defined with only hash key for the primary key;

use isolation level snapshot entity framework 4

I'm try to using a TransactionScope with isolation level snapshot on Entity framework 4 in asp.net web proyect and sql server 2012 standard edition. I'm getting this error Transactions with IsolationLevel Snapshot cannot be promoted.
using (var scope = new TransactionScope(TransactionScopeOption.RequiresNew,
new TransactionOptions { IsolationLevel = IsolationLevel.Snapshot })) {
using (var db = new Datos.TestDBDataContext(System.Configuration
.ConfigurationManager.ConnectionStrings["ApplicationServices"].ConnectionString))
{
Datos.Contacto newUser = new Datos.Contacto
{
name = user.name,
lastName = user.lastName,
type = user.type,
userId = user.userId,
email = user.Email,
password = Password(),
jobCode = user.JobCode,
DateCreated = user.DateCreated,
cityCode = user.cityCode,
numberPass = user.numberPass,
place = user.place,
estate = false
};
db.Contacts.InsertOnSubmit(newUser);
db.SubmitChanges();
}
scope.Complete();
}
What I'm doing wrong ?
Please try as shown below.Set the IsolationLevel.Serializable.
Serializable : Volatile data can be read but not modified, and no new
data can be added during the transaction.
IsolationLevel Enumeration
var scope = new TransactionScope(TransactionScopeOption.RequiresNew,
new TransactionOptions {
IsolationLevel = IsolationLevel.Snapshot,
IsolationLevel = IsolationLevel.Serializable,
})

DIfficulty adding a Global Secondary Index to an existing table in DynamoDB

Table: EmailMessages
EntityId - HashKey : String
Id - RangeKey : String
Name - String
Status - String
I would like to make a GlobalSecondaryIndex for Status
var request = new UpdateTableRequest
{
TableName = "EmailMessages",
GlobalSecondaryIndexUpdates = new List<GlobalSecondaryIndexUpdate>
{
new GlobalSecondaryIndexUpdate
{
Create = new CreateGlobalSecondaryIndexAction
{
IndexName = "GSI_EmailMessages_Status",
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("Status", KeyType.HASH)
}
}
}
}
};
var client = DynamoDBManager.DBFactory.GetClient();
client.UpdateTable(request);
However the return error I get is a 500 error with no return text so I'm unsure on what I need to correct to make this work. I've dug through the documentation and I can't seem to find much help on creating a GSI for an existing table. Any help would be much appreciated
I managed to solve my issue. Turns out I needed a lot more code. I'm going to post my solution in case someone else runs into this similar issue since there doesn't seem to be many resources out there for Dynamo. Please note my read and write capacities are very low because this is for a dev environment. You might want to consider upping yours depending on your needs
var request = new UpdateTableRequest
{
TableName = "EmailMessage",
AttributeDefinitions = new List<AttributeDefinition>
{
new AttributeDefinition
{
AttributeName = "Status",
AttributeType = ScalarAttributeType.S
}
},
GlobalSecondaryIndexUpdates = new List<GlobalSecondaryIndexUpdate>
{
new GlobalSecondaryIndexUpdate
{
Create = new CreateGlobalSecondaryIndexAction
{
IndexName = "GSI_EmailMessage_Status",
KeySchema = new List<KeySchemaElement>
{
new KeySchemaElement("Status", KeyType.HASH)
},
Projection = new Projection
{
ProjectionType = ProjectionType.ALL
},
ProvisionedThroughput = new ProvisionedThroughput
{
ReadCapacityUnits = 4,
WriteCapacityUnits = 1,
}
}
}
}
};
var client = DynamoDBManager.DBFactory.GetClient();
client.UpdateTable(request);

Retrieving CRM 4 entities with custom fields in custom workflow activity C#

I'm trying to retrieve all phone calls related to opportunity, which statecode isn't equal 1. Tried QueryByAttribute, QueryExpression and RetrieveMultipleRequest, but still has no solution.
Here some code i wrote.
IContextService contextService = (IContextService)executionContext.GetService(typeof(IContextService));
IWorkflowContext context = contextService.Context;
ICrmService crmService = context.CreateCrmService(true);
if (crmService != null)
{
QueryByAttribute query = new Microsoft.Crm.Sdk.Query.QueryByAttribute();
query.ColumnSet = new Microsoft.Crm.Sdk.Query.AllColumns();
query.EntityName = EntityName.phonecall.ToString();
query.Attributes = new string[] { "regardingobjectid" };
query.Values = new string[] { context.PrimaryEntityId.ToString() };
RetrieveMultipleRequest retrieve = new RetrieveMultipleRequest();
retrieve.Query = query;
retrieve.ReturnDynamicEntities = true;
RetrieveMultipleResponse retrieved = (RetrieveMultipleResponse)crmService.Execute(retrieve);
}
return ActivityExecutionStatus.Closed;
}
And almost same for QueryExpression
QueryExpression phCallsQuery = new QueryExpression();
ColumnSet cols = new ColumnSet(new string[] { "activityid", "regardingobjectid" });
phCallsQuery.EntityName = EntityName.phonecall.ToString();
phCallsQuery.ColumnSet = cols;
phCallsQuery.Criteria = new FilterExpression();
phCallsQuery.Criteria.FilterOperator = LogicalOperator.And;
phCallsQuery.Criteria.AddCondition("statuscode", ConditionOperator.NotEqual, "1");
phCallsQuery.Criteria.AddCondition("regardingobjectid", ConditionOperator.Equal, context.PrimaryEntityId.ToString();
I always get something like Soap exception or "Server was unable to proceed the request" when debugging.
To get exception details try to use following code:
RetrieveMultipleResponse retrieved = null;
try
{
retrieved = (RetrieveMultipleResponse)crmService.Execute(retrieve);
}
catch(SoapException se)
{
throw new Exception(se.Detail.InnerXml);
}

Resources