ydn-db: how to make any query more complex than getting a single record by id? - ydn-db

Using the packaged ydn-db library from here: http://git.yathit.com/ydn-db/downloads/ydn.db-jquery-0.7.12.js . Same happens with https://storage.cloud.google.com/download.yathit.com/ydn-db/build/zsk-ydn.db-dev-0.7.15.js .
var db = new ydn.db.Storage("mydb");
var clog = function(r) { console.log(r); }
db.put({name: "store1", keyPath: "id"}, {id: "id1", value: "value1"});
db.put({name: "store1", keyPath: "id"}, {id: "id2", value: "value2"});
db.get("store1", "id1").done(clog); // works fine
// Example as mentioned in the docs here: http://dev.yathit.com/ydn-db/getting-started.html
var reverse = false, limit = 10;
db.values(new ydn.db.Cursors("store1", "id", null, reverse), limit).done(clog)
// TypeError: Cannot call method 'getName' of null
// Also adapted from docs:
db.values(ydn.db.IndexValueCursors.where("store1", "id", "=", "id1")).done(clog)
// TypeError: Cannot call method 'getName' of null
// Also adapted from docs:
var iter = new ydn.db.ValueCursors("store1", ydn.db.KeyRange.starts("a"))
db.open(iter, clog, "readwrite")
// ydn.error.ArgumentException: Second argument must be cursor range iterator.
The functionality of the packaged library just doesn't seem to align with the docs. Any tips?

Actually, your schema don't have index name id, so the correct code should be as follow.
var db = new ydn.db.Storage("mydb");
var clog = function(r) { console.log(r); }
db.put({name: "store1", keyPath: "id"}, {id: "id1", value: "value1"});
db.put({name: "store1", keyPath: "id"}, {id: "id2", value: "value2"});
db.get("store1", "id1").done(clog); // works fine
// Example as mentioned in the docs here: http://dev.yathit.com/ydn-db/getting-started.html
var reverse = false, limit = 10;
db.values(new ydn.db.KeyCursors("store1", null, reverse), limit).done(clog)
// TypeError: Cannot call method 'getName' of null
// Also adapted from docs:
db.values(ydn.db.ValueCursors.where("store1", "=", "id1")).done(clog)
// TypeError: Cannot call method 'getName' of null
// Also adapted from docs:
var iter = new ydn.db.ValueCursors("store1", ydn.db.KeyRange.starts("i"))
var pnt = function(r) {console.log(r.getValue())}
db.open(pnt, iter, "readwrite")
Error message is definitely not informative. I push a patch, so next time, it gives index not found error.
There is api changes in open method, as well. Documentation need to be update.
Edit: more complex query can be found in http://dev.yathit.com/ydn-db/nosql-query.html

Ok, finally got it from the link in the previous answer:
// Equivalent to: SELECT * FROM store1 where value = 'value1':
var keyRange = ydn.db.KeyRange.only("value1");
var cursor = new ydn.db.IndexValueCursors("store1", "value", keyRange);
db.get(cursor, keyRange).then(function(record) {
console.log("record:", record);
}, function(err) {
throw err;
});

Related

Create an empty array in json object with AngularFire

I am a Firebase newby and I need some guidance please:
I have an array of objects: 'discussions'.
Each object contains another array of objects: 'messages'.
So far I have this code:
vm.discussions = $firebaseArray(discussionsRef.orderByChild("end_time"));
And when a user clicks on starting new discussion:
function createNewDiscussion(){
var newDiscussion = {
'category':'politics',
'isActive': true,
'start_time': Date.now(),
'messages': [
],
};
var newMessage = {
upload_time: Date.now(),
user: "fhenderson",
text: 'xxx',
.......
};
vm.discussions.$add(newDiscussion).then(function(ref) {
var id = ref.key;
var test1 = $firebaseObject(discussionsRef.child(id));
var test2 = vm.discussions.$getRecord(id); // record with $id === "foo" or null
//// need help here
}
I am having 2 issues:
When 'newDiscussion' object is created in Firebase, I cant see 'messages' array being created at all (maybe because it's empty?). Therefore I cant get 'messages' reference.
How to add new 'newMessage' to 'messages' array once a new discussion is created?
thanks.

Meteor mongodb insert document of an array

This code has the server insert some documents in a collection for the client to find it later.
I need to return the array for a given task
But the page is saying
No data received
Why is that and how to fix it? Thanks
//Both
FooterButtons2 = new Mongo.Collection('footerButtons2');
//Server
Meteor.publish('footerButtons2', function(){
return FooterButtons2.find();
});
FooterButtons2.insert(
{ "task1": ["submit"]},
{ "task2": ["cancel","continue"]}
);
//client
Meteor.subscribe('footerButtons2');
var res = FooterButtons2.findOne("task1");
When you search like this:
var res = FooterButtons2.findOne("task1");
you are searching an object that has the "_id" key equal to "task1", this is not correct.
You want the object that has the key "task1" in it. The correct way would be:
var res = FooterButtons2.findOne({
task1: { $exists: true }
});
But ideally, you should be doing searches based on values and not keys. Something like this:
FooterButtons2.insert({
task: "task1",
buttons: ["submit"]
}, {
task: "task2",
buttons: ["cancel", "continue"]
});
var res = FooterButtons2.findOne({
task: "task1"
});

How to set a DynamoDB Map property value, when the map doesn't exist yet

How do you "upsert" a property to a DynamoDB row. E.g. SET address.state = "MA" for some item, when address does not yet exist?
I feel like I'm having a chicken-and-egg problem because DynamoDB doesn't let you define a sloppy schema in advance.
If address DID already exist on that item, of type M (for Map), the internet tells me I could issue an UpdateExpression like:
SET #address.#state = :value
with #address, #state, and :value appropriately mapped to address, state, and MA, respectively.
But if the address property does not already exist, this gives an error:
'''
ValidationException: The document path provided in the update expression is invalid for update
'''
So.. it appears I either need to:
Figure out a way to "upsert" address.state (e.g., SET address = {}; SET address.state = 'MA' in a single command)
or
Issue three (!!!) roundtrips in which I try it, SET address = {}; on failure, and then try it again.
If the latter.... how do I set a blank map?!?
Ugh.. I like Dynamo, but unless I'm missing something obvious this is a bit crazy..
You can do it with two round trips, the first conditionally sets an empty map for address if it doesn't already exist, and the second sets the state:
db.update({
UpdateExpression: 'SET #a = :value',
ConditionExpression: 'attribute_not_exists(#a)',
ExpressionAttributeValues: {
":value": {},
},
ExpressionAttributeNames: {
'#a': 'address'
}
}, ...);
Then:
db.update({
UpdateExpression: 'SET #a.#b = :v',
ExpressionAttributeNames: {
'#a': 'address',
'#b': 'state'
},
ExpressionAttributeValues: {
':v': 'whatever'
}
}, ...);
You cannot set nested attributes if the parent document does not exist. Since address does not exist you cannot set the attribute province inside it. You can achieve your goal if you set address to an empty map when you create the item. Then, you can use the following parameters to condition an update on an attribute address.province not existing yet.
var params = {
TableName: 'Image',
Key: {
Id: 'dynamodb.png'
},
UpdateExpression: 'SET address.province = :ma',
ConditionExpression: 'attribute_not_exists(address.province)',
ExpressionAttributeValues: {
':ma': 'MA'
},
ReturnValues: 'ALL_NEW'
};
docClient.update(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
By the way, I had to replace state with province as state is a reserved word.
Another totally different method is to simply create the address node when creating the parent document in the first place. For example assuming you have a hash key of id, you might do:
db.put({
Item: {
id: 42,
address: {}
}
}, ...);
This will allow you to simply set the address.state value as the address map already exists:
db.update({
UpdateExpression: 'SET #a.#b = :v',
AttributeExpressionNames: {
'#a': 'address',
'#b': 'state'
},
AttributeExpressionValues: {
':v': 'whatever'
}
}, ...);
Some kotlin code to do this recursively regardless how deep it goes. It sets existence of parent paths as condition and if condition check fails, recursively creates those paths first. It has to be in the library's package so it can access those package private fields/classes.
package com.amazonaws.services.dynamodbv2.xspec
import com.amazonaws.services.dynamodbv2.document.Table
import com.amazonaws.services.dynamodbv2.model.ConditionalCheckFailedException
import com.amazonaws.services.dynamodbv2.xspec.ExpressionSpecBuilder.attribute_exists
fun Table.updateItemByPaths(hashKeyName: String, hashKeyValue: Any, updateActions: List<UpdateAction>) {
val parentPaths = updateActions.map { it.pathOperand.path.parent() }
.filter { it.isNotEmpty() }
.toSet() // to remove duplicates
try {
val builder = ExpressionSpecBuilder()
updateActions.forEach { builder.addUpdate(it) }
if (parentPaths.isNotEmpty()) {
var condition: Condition = ComparatorCondition("=", LiteralOperand(true), LiteralOperand(true))
parentPaths.forEach { condition = condition.and(attribute_exists<Any>(it)) }
builder.withCondition(condition)
}
this.updateItem(hashKeyName, hashKeyValue, builder.buildForUpdate())
} catch (e: ConditionalCheckFailedException) {
this.updateItemByPaths(hashKeyName, hashKeyValue, parentPaths.map { M(it).set(mapOf<String, Any>()) })
this.updateItemByPaths(hashKeyName, hashKeyValue, updateActions)
}
}
private fun String.parent() = this.substringBeforeLast('.', "")
Here is a helper function I wrote in Typescript that works for this a single level of nesting using a recursive method.
I refer to the top-level attribute as a column.
//usage
await setKeyInColumn('customerA', 'address', 'state', "MA")
// Updates a map value to hold a new key value pair. It will create a top-level address if it doesn't exist.
static async setKeyInColumn(primaryKeyValue: string, colName: string, key: string, value: any, _doNotCreateColumn?:boolean) {
const obj = {};
obj[key] = value; // creates a nested value like {address:value}
// Some conditions depending on whether the column already exists or not
const ConditionExpression = _doNotCreateColumn ? undefined:`attribute_not_exists(${colName})`
const AttributeValue = _doNotCreateColumn? value : obj;
const UpdateExpression = _doNotCreateColumn? `SET ${colName}.${key} = :keyval `: `SET ${colName} = :keyval ` ;
try{
const updateParams = {
TableName: TABLE_NAME,
Key: {key:primaryKeyValue},
UpdateExpression,
ExpressionAttributeValues: {
":keyval": AttributeValue
},
ConditionExpression,
ReturnValues: "ALL_NEW",
}
const resp = await docClient.update(updateParams).promise()
if (resp && resp[colName]) {
return resp[colName];
}
}catch(ex){
//if the column already exists, then rerun and do not create it
if(ex.code === 'ConditionalCheckFailedException'){
return this.setKeyInColumn(primaryKeyValue,colName,key, value, true)
}
console.log("Failed to Update Column in DynamoDB")
console.log(ex);
return undefined
}
}
I've got quite similar situation. I can think of only a one way to do this in 1 query/atomically.
Extract map values to top level attributes.
Example
Given I have this post item in DynamoDB:
{
"PK": "123",
"SK": "post",
"title": "Hello World!"
}
And I want to later add an analytics entry to same partition:
{
"PK": "123",
"SK": "analytics#december",
"views": {
// <day of month>: <views>
"1": "12",
"2": "457463",
// etc
}
}
Like in your case, it's not possible to increment/decrement views days counters in single query if analytics item nor views map might not exist (could be later feature or don't want to put empty items).
Proposed solution:
{
"PK": "123",
"SK": "analytics#december",
// <day of month>: <views>
"1": "12", // or "day1" if "1" seems too generic
"2": "457463",
// etc
}
}
Then you could do something like this (increment +1 example):
{
UpdateExpression: "SET #day = if_not_exists(#day, 0) + 1",
AttributeExpressionNames: {
'#day': "1"
}
}
if day attribute value doesn't exist, set default value to 0
if item in database doesn't exist, update API adds a new one

How would you find a collection dynamically in Meteor with a value?

Given I have 3 types of collections and a dynamic value, how would I specify what collection to search for based on that dynamic value?
E.g,
array = [
{id: 'one', type: 'profile'},
{id: 'something', type: 'post'},
{id: 'askjdaksj', type: 'comment']
]
How would I isolate the type and turn it into a collection? Basically turning type into Collection.find
array[0].type.find({_id: id});
=> Profiles.find({_id: id});
Is this possible?
Here's a complete working example:
Posts = new Mongo.Collection('posts');
Comments = new Mongo.Collection('comments');
var capitalize = function(string) {
return string.charAt(0).toUpperCase() + string.slice(1);
};
var nameToCollection = function(name) {
// pluralize and capitalize name, then find it on the global object
// 'post' -> global['Posts'] (server)
// 'post' -> window['Posts'] (client)
return this[capitalize(name) + 's'];
};
Meteor.startup(function() {
// ensure all old documents are removed
Posts.remove({});
Comments.remove({});
// insert some new documents
var pid = Posts.insert({text: 'I am a post'});
var cid = Comments.insert({text: 'I am a comment'});
var items = [
{id: pid, type: 'post'},
{id: cid, type: 'comment'}
];
_.each(items, function(item) {
// find the collection object based on the type (name)
var collection = nameToCollection(item.type);
// retrieve the document from the dynamically found collection
var doc = collection.findOne(item.id);
console.log(doc);
});
});
Recommended reading: collections by reference.

Backbone.js: correct way of filtering a collection?

The current method I'm using is to filter a collection, which returns an array, and use
collection.reset(array)
to re-populate it. However, this modifies the original collection, so I added an array called "originalCollectionArray" which keeps track of the initial array state of the collection. When no filtering is active I simply use
collection.reset(originalCollectionArray)
But then, I need to keep track of adding and removing models from the real collection, so I did this:
// inside collection
initialize: function(params){
this.originalCollectionArray = params;
this.on('add', this.addInOriginal, this);
this.on('remove', this.removeInOriginal, this);
},
addInOriginal: function(model){
this.originalCollectionArray.push(model.attributes);
},
removeInOriginal: function(model){
this.originalTasks = _(this.originalTasks).reject(function(val){
return val.id == model.get('id');
});
},
filterBy: function(params){
this.reset(this.originalCollectionArray, {silent: true});
var filteredColl = this.filter(function(item){
// filter code...
});
this.reset(filteredColl);
}
This is quickly becoming cumbersome as I try to implement other tricks related to the manipulation of the collection, such as sorting. And frankly, my code looks a bit hacky. Is there an elegant way of doing this?
Thanks
You could create a collection as a property of the main collection reflecting the state of the filters:
var C = Backbone.Collection.extend({
initialize: function (models) {
this.filtered = new Backbone.Collection(models);
this.on('add', this.refilter);
this.on('remove', this.refilter);
},
filterBy: function (params){
var filteredColl = this.filter(function(item){
// ...
});
this.filtered.params = params;
this.filtered.reset(filteredColl);
},
refilter: function() {
this.filterBy(this.filtered.params);
}
});
The parent collection keeps its models whatever filters you applied, and you bind to the filtered collection to know when a change has occurred. Binding internally on the add and remove events lets you reapply the filter. See
http://jsfiddle.net/dQr7X/ for a demo.
The major problem on your code is that you are using a raw array as original, instead of a Collection. My code is close to the yours but use only Collections, so methods like add, remove and filter works on the original:
var OriginalCollection = Backbone.Collection.extend({
});
var FilteredCollection = Backbone.Collection.extend({
initialize: function(originalCol){
this.originalCol = originalCol;
this.on('add', this.addInOriginal, this);
this.on('remove', this.removeInOriginal, this);
},
addInOriginal: function(model){
this.originalCol.add(model);
},
removeInOriginal: function(model){
this.originalCol.remove(model);
},
filterBy: function(params){
var filteredColl = this.originalCol.filter(function(item){
// filter code...
});
this.reset(filteredColl);
}
});

Resources