KeyFactory keyFactory = datastore.newKeyFactory().setKind("keyKind");
Key key = keyFactory.newKey("keyName");
Entity entity = datastore.get(key);
if (entity != null) {
System.out.println("Updating access_time for " + entity.getString("name"));
entity = Entity.newBuilder(entity)
.set("access_time", DateTime.now())
.build();
datastore.update(entity);
}
https://github.com/googleapis/java-datastore#updating-data , as per this there has to be a read before write.
Can an update happen without read in google datastore? As in certain cases, I would not care if certain attributes are overwritten.
If you don't need to know if an entity exists and can overwrite the data, then consider using upsert
Related
How can I get items count for a particular partition key using .net core preferably using Object Persistence Interface or Document Interfaces?
Since I do not see any docs any where, currently I get the number of items count by retrieve all the item and get its count, but it is very expensive to do the reads.
What is the best practices for such item count request? Thank you.
dynamodb is mostly a document oriented key-value db; so its not optimized for functionality of the common relation db functions (like item count).
to minimize the data that is transmitted and to improve speed you may want to do the following:
Create Lambda Function that returns Item Count
To avoid transmitting data outside of AWS; which is slow and expensive.
query options
use only keys in your projection-expression,
reducing the data that is transmitted from db
max page-size, reducing number of calls needed
Stream Option
Streams could also be used for keeping counts; e.g. as described in
https://medium.com/signiant-engineering/real-time-aggregation-with-dynamodb-streams-f93547cfb244
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-aggregation.html
Related SO Question
Complexity of finding total records count with partition key in nosql dynamodb table?
I just realized that using low level interface in QueryRequest one can set Select = "COUNT" then when calling QueryAsync() orQuery() will return the count only as a integer only. Please refer to code sample below.
private static QueryRequest getStockRecordCountQueryRequest(string tickerSymbol, string prefix)
{
string partitionName = ":v_PartitionKeyName";
string sortKeyPrefix = ":v_sortKeyPrefix";
var request = new QueryRequest
{
TableName = Constants.TableName,
ReturnConsumedCapacity = ReturnConsumedCapacity.TOTAL,
Select = "COUNT",
KeyConditionExpression = $"{Constants.PartitionKeyName} = {partitionName} and begins_with({Constants.SortKeyName},{sortKeyPrefix})",
ExpressionAttributeValues = new Dictionary<string, AttributeValue>
{
{ $"{partitionName}", new AttributeValue {
S = tickerSymbol
}},
{ $"{sortKeyPrefix}", new AttributeValue {
S = prefix
}}
},
// Optional parameter.
ConsistentRead = false,
ExclusiveStartKey = null,
};
return request;
}
but I would like to point out that this still will consumed the same read units as retrieving all the item and get its count by yourself. but since it is only returning the count as an integer, it is a lot more efficient then transmitting the entire items list cross the wire.
I think using DynamoDB Streams in a more proper way to get the counts for large project. It is just a lot more complicated to implement.
We have an existing user database with SHA1-encoded passwords. We upload them to the Google Federated Database (through the GitKitClient java lib), but then these uploaded users can't log in The verifyPassword always returns "Incorrect password" ! The call to the uploadUsers looks like gitkitClient.uploadUsers('SHA1', new byte[0], gitkitUsers)
(We must provide an empty byte array as second param (hash key), since we get NPEs if we provide a null value)
The method that creates the GitkitUsers that are in the list is as follows:
private GitkitUser createGitkitUserFromUser(User user) {
GitkitUser gitkitUser = new GitkitUser()
gitkitUser.email = user.email
gitkitUser.localId = getLocalId(user)
gitkitUser.name = user.displayName
gitkitUser.hash = user.password?.bytes
if (user.pictureFileName) {
gitkitUser.photoUrl = user.getPictureUrl()
}
return gitkitUser
}
We see no way to further investigate. Did someone successfully use it ?
Make sure that the hashKey you use in setPassword() is the same one used in uploadUsers().
I am using the php SDK so I can't share code for you, but when I did NOT use the same hashKey for both places, I had the same problem.
I am using ydn.db from local storage. Which function can be used to remove the store using YDB.DB library ? The stores i have added is as follows
var schema = {
stores: [{
name:'consignments',
keyPath:"timeStamp"
}];
var db = new ydn.db.Storage('localhost', schema);
I wish to check whether the store exist in localstorage and if it exists delete the store and if not exist add the store.
If store exist or not can be known only after opening the database. If a schema does not a store name, but exit in the connected database, the existing store will be removed.
To delete the database
var db = new ydn.db.Storage('localhost', schema);
To delete above the database,
ydn.db.deleteDatabase(db.getName(), db.getType());
If type of database is not known
ydn.db.deleteDatabase('localhost');
I use the deleteDatabase function like this:
indexedDB.deleteDatabase(storage_name);
see the doc here: https://dev.yathit.com/api/ydn/db/index.html
How to query range key programmatically in DynamoDB, I am using .Net AWSSDK ,I am able to query on Hash key with below code :
GetItemRequest request = new GetItemRequest
{
TableName = tableName
};
request.Key = new Dictionary<string,AttributeValue>();
request.Key.Add("ID",new AttributeValue { S = PKValue });
GetItemResponse response = client.GetItem(request);
Please suggest,
Thanks in advance.
There are two kinds of primary key in DynamoDB: Hash-only or Hash-Range.
In the above code I guess your table is Hash-only and you use the hash key to retrieve an element with hashkey equals to PKValue.
If your table is in H-R schema and you want to retrieve a specific element with a hashKey and rangeKey, you can reuse the above code and in addition, add the {"RangeKey", new AttributeValue } into your your request.KEY
On the other hand, query means a different thing in DynamoDB. Query will return you a list of rows sorted in some order.
In RavenDB I can store objects of type Products and Categories and they will automatically be located in different collections. This is fine.
But what if I have 2 logically completely different types of products but they use the same class? Or instead of 2 I could have a generic number of different types of products. Would it then be possible to tell Raven to split the product documents up in collections, lets say based on a string property available on the Product class?
Thankyou in advance.
EDIT:
I Have created and registered the following StoreListener that changes the collection for the documents to be stored on runtime. This results in the documents correctly being stored in different collections and thus making a nice, logically grouping of the documents.
public class DynamicCollectionDefinerStoreListener : IDocumentStoreListener
{
public bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
{
var entity = entityInstance as EntityData;
if(entity == null)
throw new Exception("Cannot handle object of type " + EntityInstance.GetType());
metadata["Raven-Entity-Name"] = RavenJToken.FromObject(entity.TypeId);
return true;
}
public void AfterStore(string key, object entityInstance, RavenJObject metadata)
{
}
}
However, it seems I have to adjust my queries too in order to be able to get the objects back. My typical query of mine used to look like this:
session => session.Query<EntityData>().Where(e => e.TypeId == typeId)
With the 'typeId' being the name of the new raven collections (and the name of the entity type saved as a seperate field on the EntityData-object too).
How would I go about quering back my objects? I can't find the spot where I can define my collection at runtime prioring to executing my query.
Do I have to execute some raw lucene queries? Or can I maybe implement a query listener?
EDIT:
I found a way of storing, querying and deleting objects using dynamically defined collections, but I'm not sure this is the right way to do it:
Document store listener:
(I use the class defined above)
Method resolving index names:
private string GetIndexName(string typeId)
{
return "dynamic/" + typeId;
}
Store/Query/Delete:
// Storing
session.Store(entity);
// Query
var someResults = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e => e.EntityId == entity.EntityId)
var someMoreResults = session.Advanced.LuceneQuery<EntityData>(GetIndexName(entityTypeId)).Where("TypeId:Colors AND Range.Basic.ColorCode:Yellow)
// Deleting
var loadedEntity = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e =>
e.EntityId == entity.EntityId).SingleOrDefault();
if (loadedEntity != null)
{
session.Delete<EntityData>(loadedEntity);
}
I have the feeling its getting a little dirty, but is this the way to store/query/delete when specifying the collection names runtime? Or do I trap myself this way?
Stephan,
You can provide the logic for deciding on the collection name using:
store.Conventions.FindTypeTagName
This is handled statically, using the generic type.
If you want to make that decision at runtime, you can provide it using a DocumentStoreListner