How to parse nested keys using Firebase Cloud Functions - firebase

I have a data structure which have multiple nested keys, key1 and key2.
The value of key1 and key2 is unknown and will typically be some GUID's.
How do I get the data out of this structure?
"key1":
{
"key2": {
"name1":"value1",
"name2":"value2",
"sub1":
{
"sub1_name1":"sub1_value1",
"sub2_name2":"sub2_value2"
},
"name3":"value3"
}
}, .... more records following
I have the following code that extracts some data but after that I am unable to query the data:
admin.database().ref('table_x/').once('value', (snapshot) => {
console.log(snapshot.key); //table_x
snapshot.forEach(function(child) {
var key1 = child.key; // this gives key1
// get key 2
// get key 2's values, children

Something like this?
for (key in child) {
for (innerkey in child[key]) {
console.log(innerkey);
}
}
https://jsfiddle.net/chickenbeef/ycers98g

Use the library Defiantjs(http://defiantjs.com/) they have a nice implementation of an xslt template to pull in your data set
once you able to loop through your data structure, use the xpath tool provided by the library to run specific queries.
Example of JSON.search
data = [{
"key1": {
"key2": {
"name1": "value1",
"name2": "value2",
"sub1": {
"sub1_name1": "sub1_value1",
"sub2_name2": "sub2_value2"
},
"name3": "value3"
}
}}],
res1 = JSON.search(data, "//*[contains(name(), 'key')]/."),
res2 = JSON.search(data, "//*[contains(text(), 'sub')]/..");
https://jsfiddle.net/edsonvanwyk/amanp3g0/4/

Related

Looping through current item values

I have what is essentially a questionnaire in the form of a SQL model.
User answers the questions, creates the item. With the item loaded, how can I loop through the values? I'm new to JS and GAM, but I've tried the below and can only seem to get the name of the fields, not its value.
function generateScore(){
ds = app.datasources.Checklist.item;
for (var x in ds){
if (ds.x === 'Safe'){
console.log("Passed");
} else {
console.log("Failed");
}
}
}
Output will be 'Fail' as 'ds.x' is only returning the name of the field and not its value.
It's probably really simple, but can somebody guide me in the right direction?
Thanks
Short answer: In your function change ds.x to ds[x]:
function generateScore(){
ds = app.datasources.Checklist.item;
for (var x in ds){
if (ds[x] === 'Safe'){
console.log("Passed");
} else {
console.log("Failed");
}
}
}
TL;DR
There are Other ways of looping through the values of an object.
Let's assume the following object:
const obj = {
"key1": "value1",
"key2": "value2",
"key3": "value3"
};
You can use the Object.keys syntax.
JS ES6 answer:
Object.keys(obj).map(key => obj[key]) // returns an array of values --> ["value1", "value2", "value3"]
Object.keys(obj).map(key => {
console.log(obj[key])
})
// logs all values one by one --> "value1" "value2" "value3"
JS ES5 answer:
Object.keys(obj).map(function(key) {
console.log(obj[key])
});

Querying Cosmos Nested JSON documents

I would like to turn this resultset
[
{
"Document": {
"JsonData": "{\"key\":\"value1\"}"
}
},
{
"Document": {
"JsonData": "{\"key\":\"value2\"}"
}
}
]
into this
[
{
"key": "value1"
},
{
"key": "value2"
}
]
I can get close by using a query like
select value c.Document.JsonData from c
however, I end up with
[
"{\"key\":\"value1\"}",
"{\"key\":\"value2\"}"
]
How can I cast each value to an individual JSON fragment using the SQL API?
As David Makogon said above, we need to transform such data within our app. We can do as below:
string data = "[{\"key\":\"value1\"},{\"key\":\"value2\"}]";
List<Object> t = JsonConvert.DeserializeObject<List<Object>>(data);
string jsonData = JsonConvert.SerializeObject(t);
Screenshot of result:

DyanamoDB SCAN with nested attribute

Can I scan DynamoDB by 'order.shortCode', in the given example. The console is indicating I can't with dot notation, and I can't find any documentation on it.
{
"key2": "cj11b1ygp0000jcgubpe5mso3",
"order": {
"amount": 74.22,
"dateCreated": "2017-04-02T19:15:33-04:00",
"orderNumber": "cj11b1ygp0000jcgubpe5mso3",
"shortCode": "SJLLDE"
},
"skey2": "SJLLDE"
}
To scan by a nested attribute, you should use ExpressionAttributeNames parameter to pass each path component (i.e. order and shortCode) separately into FilterExpression like shown below:
var params = {
TableName: 'YOUR_TABLE_NAME',
FilterExpression: "#order.#shortCode = :shortCodeValue",
ExpressionAttributeNames: {
'#order': 'order',
"#shortCode": "shortCode"
},
ExpressionAttributeValues: {
':shortCodeValue': 'SJLLDE'
}
};
dynamodbDoc.scan(params, function(err, data) {
});
Here is a link to documentation explaining this:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ExpressionAttributeNames.html#Expressions.ExpressionAttributeNames.NestedAttributes

How to set a DynamoDB Map property value, when the map doesn't exist yet

How do you "upsert" a property to a DynamoDB row. E.g. SET address.state = "MA" for some item, when address does not yet exist?
I feel like I'm having a chicken-and-egg problem because DynamoDB doesn't let you define a sloppy schema in advance.
If address DID already exist on that item, of type M (for Map), the internet tells me I could issue an UpdateExpression like:
SET #address.#state = :value
with #address, #state, and :value appropriately mapped to address, state, and MA, respectively.
But if the address property does not already exist, this gives an error:
'''
ValidationException: The document path provided in the update expression is invalid for update
'''
So.. it appears I either need to:
Figure out a way to "upsert" address.state (e.g., SET address = {}; SET address.state = 'MA' in a single command)
or
Issue three (!!!) roundtrips in which I try it, SET address = {}; on failure, and then try it again.
If the latter.... how do I set a blank map?!?
Ugh.. I like Dynamo, but unless I'm missing something obvious this is a bit crazy..
You can do it with two round trips, the first conditionally sets an empty map for address if it doesn't already exist, and the second sets the state:
db.update({
UpdateExpression: 'SET #a = :value',
ConditionExpression: 'attribute_not_exists(#a)',
ExpressionAttributeValues: {
":value": {},
},
ExpressionAttributeNames: {
'#a': 'address'
}
}, ...);
Then:
db.update({
UpdateExpression: 'SET #a.#b = :v',
ExpressionAttributeNames: {
'#a': 'address',
'#b': 'state'
},
ExpressionAttributeValues: {
':v': 'whatever'
}
}, ...);
You cannot set nested attributes if the parent document does not exist. Since address does not exist you cannot set the attribute province inside it. You can achieve your goal if you set address to an empty map when you create the item. Then, you can use the following parameters to condition an update on an attribute address.province not existing yet.
var params = {
TableName: 'Image',
Key: {
Id: 'dynamodb.png'
},
UpdateExpression: 'SET address.province = :ma',
ConditionExpression: 'attribute_not_exists(address.province)',
ExpressionAttributeValues: {
':ma': 'MA'
},
ReturnValues: 'ALL_NEW'
};
docClient.update(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
By the way, I had to replace state with province as state is a reserved word.
Another totally different method is to simply create the address node when creating the parent document in the first place. For example assuming you have a hash key of id, you might do:
db.put({
Item: {
id: 42,
address: {}
}
}, ...);
This will allow you to simply set the address.state value as the address map already exists:
db.update({
UpdateExpression: 'SET #a.#b = :v',
AttributeExpressionNames: {
'#a': 'address',
'#b': 'state'
},
AttributeExpressionValues: {
':v': 'whatever'
}
}, ...);
Some kotlin code to do this recursively regardless how deep it goes. It sets existence of parent paths as condition and if condition check fails, recursively creates those paths first. It has to be in the library's package so it can access those package private fields/classes.
package com.amazonaws.services.dynamodbv2.xspec
import com.amazonaws.services.dynamodbv2.document.Table
import com.amazonaws.services.dynamodbv2.model.ConditionalCheckFailedException
import com.amazonaws.services.dynamodbv2.xspec.ExpressionSpecBuilder.attribute_exists
fun Table.updateItemByPaths(hashKeyName: String, hashKeyValue: Any, updateActions: List<UpdateAction>) {
val parentPaths = updateActions.map { it.pathOperand.path.parent() }
.filter { it.isNotEmpty() }
.toSet() // to remove duplicates
try {
val builder = ExpressionSpecBuilder()
updateActions.forEach { builder.addUpdate(it) }
if (parentPaths.isNotEmpty()) {
var condition: Condition = ComparatorCondition("=", LiteralOperand(true), LiteralOperand(true))
parentPaths.forEach { condition = condition.and(attribute_exists<Any>(it)) }
builder.withCondition(condition)
}
this.updateItem(hashKeyName, hashKeyValue, builder.buildForUpdate())
} catch (e: ConditionalCheckFailedException) {
this.updateItemByPaths(hashKeyName, hashKeyValue, parentPaths.map { M(it).set(mapOf<String, Any>()) })
this.updateItemByPaths(hashKeyName, hashKeyValue, updateActions)
}
}
private fun String.parent() = this.substringBeforeLast('.', "")
Here is a helper function I wrote in Typescript that works for this a single level of nesting using a recursive method.
I refer to the top-level attribute as a column.
//usage
await setKeyInColumn('customerA', 'address', 'state', "MA")
// Updates a map value to hold a new key value pair. It will create a top-level address if it doesn't exist.
static async setKeyInColumn(primaryKeyValue: string, colName: string, key: string, value: any, _doNotCreateColumn?:boolean) {
const obj = {};
obj[key] = value; // creates a nested value like {address:value}
// Some conditions depending on whether the column already exists or not
const ConditionExpression = _doNotCreateColumn ? undefined:`attribute_not_exists(${colName})`
const AttributeValue = _doNotCreateColumn? value : obj;
const UpdateExpression = _doNotCreateColumn? `SET ${colName}.${key} = :keyval `: `SET ${colName} = :keyval ` ;
try{
const updateParams = {
TableName: TABLE_NAME,
Key: {key:primaryKeyValue},
UpdateExpression,
ExpressionAttributeValues: {
":keyval": AttributeValue
},
ConditionExpression,
ReturnValues: "ALL_NEW",
}
const resp = await docClient.update(updateParams).promise()
if (resp && resp[colName]) {
return resp[colName];
}
}catch(ex){
//if the column already exists, then rerun and do not create it
if(ex.code === 'ConditionalCheckFailedException'){
return this.setKeyInColumn(primaryKeyValue,colName,key, value, true)
}
console.log("Failed to Update Column in DynamoDB")
console.log(ex);
return undefined
}
}
I've got quite similar situation. I can think of only a one way to do this in 1 query/atomically.
Extract map values to top level attributes.
Example
Given I have this post item in DynamoDB:
{
"PK": "123",
"SK": "post",
"title": "Hello World!"
}
And I want to later add an analytics entry to same partition:
{
"PK": "123",
"SK": "analytics#december",
"views": {
// <day of month>: <views>
"1": "12",
"2": "457463",
// etc
}
}
Like in your case, it's not possible to increment/decrement views days counters in single query if analytics item nor views map might not exist (could be later feature or don't want to put empty items).
Proposed solution:
{
"PK": "123",
"SK": "analytics#december",
// <day of month>: <views>
"1": "12", // or "day1" if "1" seems too generic
"2": "457463",
// etc
}
}
Then you could do something like this (increment +1 example):
{
UpdateExpression: "SET #day = if_not_exists(#day, 0) + 1",
AttributeExpressionNames: {
'#day': "1"
}
}
if day attribute value doesn't exist, set default value to 0
if item in database doesn't exist, update API adds a new one

How to retrieve a subset of array within an object in a meteor collection?

Hope someone can help! I have a collection in meteor which has objects which contain arrays of temperature readings in the following format:
"temp_readings": [
{
"reading_time": {
"$date": "2015-01-18T11:54:00.700Z"
},
"temp_F": 181.76
},
{
"reading_time": {
"$date": "2015-01-18T11:55:00.700Z"
},
"temp_F": 187.16
},
{
"reading_time": {
"$date": "2015-01-18T11:56:00.700Z"
},
"temp_F": 190.76
},
{
"reading_time": {
"$date": "2015-01-18T11:57:00.700Z"
},
"temp_F": 196.16
}
]
I can retrieve this complete array in my client side meteor code but I now want to read just a subset of this array based on a date/time which is being set by the user. So for example retrieve the subset of the array which has only entries equal or later than "2015-01-18T11:56:00.700Z"... I know I could probably do something with selective publish/subscribe methods but for now is there a simple way on the client side to retrieve this subset of data? Maybe some javascript methods can help?
Thanks in advance,
Rick
// or however you fetch your document
var doc = MyCollection.findOne();
if (!doc)
return [];
// ensure temps is an array
var temps = doc.temp_readings || [];
// return an array of temp readings where the date is > someOtherDate
return _.filter(temps, function(temp) {
return temp.reading_time.$date > someOtherDate;
});

Resources