I'm trying to design a performant NoSQL schema for my Firebase application, but how would I query multiple checkpoints which have a unique tag?
Consider the following rules:
Tiers can have multiple tags
Tags can have multiple Checkpoints
Tag keys (names) are unique for each tier
Checkpoints can have multiple tags, but only 1 tag per tier
Here's what I have so far:
{
"tiers": {
"1": {
"web": true,
"android": true
},
"2": {
"basics": true,
"angular2": true,
"aurelia": true
},
"3": {
"basics": true,
"components": true
}
},
"tags" : {
"1": {
"web": {
"name": "Web Development",
"children": {
"front-end": true,
"angular2": true,
"aurelia": true
}
},
"android": {
"name": "Android Development",
"children": {
"front-end": true
}
}
},
"2": {
"front-end": {
"name": "Basics",
"parents": {
"web": true,
"android": true
},
"children": {
"angular2": true,
"aurelia": true,
"android-studio": true
}
}
},
"3": {
"angular2": {
"name": "Angular 2.x",
"parents": {...},
"children": {...}
},
"aurelia": {
"name": "Aurelia 1.x"
}
}
},
"checkpoints": {
"<randomKey>" : {
"name": "Angular 2 Quick Start",
"tags": {
"1": "web",
"2": "front-end",
"3": "angular2"
}
}
}
}
Right now I can query for all checkpoints under the Tier 1 web tag with:
ref.orderByChild('tags/1').equalTo("web").once('value', snap => console.log(snap.val()));
but since you can only define one indexOn rule, it's not optimized. At least if I can set the indexOn rule I can at least filter out most of the checkpoints then filter the rest in my code.
What can I do to efficiently query my checkpoints based on multiple tags AND the tiers?
Eventually I'd need to do a query for checkpoints with "tags": {"1": "web" AND "2": "front-end"} which I can't for the life of me figure out how to execute efficiently. I was thinking of doing another table with composite keys (e.g. each tier/tag contains references to ALL child checkpoints), but that would lead to requiring me to add and delete references within each tier.
There must be a better way.
I was vastly over complicating the solution - I ended up:
Removing Tier from the equation all together
Adding each checkpoint within each tag it was related to
Querying all checkpoints related to each selected tag and removing the ones which did not appear in each tag selected
Here's my new schema:
allTags = [
{"abc": true, "def": true, "hij": true},
{"abc": true, "def": true}
];
tags: [
{ "0": [
{"software-development": {name: "Software Development", checkpoints: [ {"abc": true}, {"def": true}, {"hij": true}]}}
]},
{"1": [
{"web": {name: "Web", checkpoints: [ {"abc": true}, {"def": true}]}},
{"android": {name: "Android", checkpoints: [{"hij": true}]}}
]}
];
checkpoints: [
{"abc": { name: "Angular2 Quick Start"}},
{"def": { name: "Building global directives in Angular2"}},
{"hij": { name: "Android Quick Start"}},
];
Creating a new Checkpoint:
public createCheckpoint(tags: any[], checkpoint: any) {
// push checkpoint
let checkpointRef = this.firebase.child('test/checkpoints');
let checkpointKey = checkpointRef.push(checkpoint).key(); // might have to do separate call
// Add checkpoint under each related tag
tags.forEach(tag => {
let tagRef = this.firebase.child('test/tags/' + tag.tier + '/' + tag.key + '/checkpoints/' + checkpointKey);
tagRef.set(true)
});
}
Retrieving all checkpoints based on selected tags:
public getCheckpointss(tags: any[]) {
// tag.tier, tag.key
let checkpointKeysToGet: string[] = [];
tags.forEach(tag => {
let ref = this.firebase.child('test/tags/' + tag.tier + '/' + tag.key + '/checkpoints');
ref.once('value', snap => {
let tagKeys:string[] = this.convertToArray(snap.val());
if (checkpointKeysToGet.length == 0){
checkpointKeysToGet = checkpointKeysToGet.concat(tagKeys);
}
else {
checkpointKeysToGet.forEach(existingTagKey => {
let tagKeysInBothTiers = tagKeys.filter(tagKey => {
return checkpointKeysToGet.includes(tagKey, 0)
});
checkpointKeysToGet = tagKeysInBothTiers;
console.log(checkpointKeysToGet);
});
}
});
});
}
All valid criticisms are welcome as long as you propose a solution to go along with it :)
Related
I have defined a state machine in AWS step functions and one of my states is storing an item to DynamoDB
...
"Store item": {
"End": true,
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Parameters": {
"Item": {
"foo": {
"S.$": "$.data.foo"
},
"bar": {
"S.$": "$.data.bar"
},
"baz": {
"S.$": "$.data.baz"
},
},
"TableName": "nrp_items"
}
},
...
The problem starts from the fact that baz property is optional, ie not exist in some cases.
On those cases, the putItem task fails:
An error occurred while executing the state 'Store item' (entered at the event id #71). > The JSONPath '$.data.baz' specified for the field 'S.$' could not be found in the input
My backup plan is to use a lambda to perform that type of operation, but can I do it directly using the putItem task in steps function?
I was wondering if:
Is possible to somehow inject via JSONPath my whole $.data item to the "Item" property, something like:
...
"Store item": {
"End": true,
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Parameters": {
"Item": "$.data",
"TableName": "nrp_items"
}
},
...
OR
2) Define that the baz property is optional
TL;DR We can deal with optional variables with a "Variable": "$.baz", "IsPresent": true Choice condition to handle no-baz cases.
The Amazon States Language spec does not have optional properties: Step Functions will throw an error if $.baz does not exist in the input. We can avoid undefined paths by inserting a two-branch Choice State, one branch of which handles baz-exists cases, the other no-baz cases. Each branch continues with a Pass State that reworks the data input into dynamo-format Item syntax, using Parameters. The put-item task's "Item.$": "$.data" (as in your #1) contains only foo-bar when baz is not defined, but all three otherwise.
{
"StartAt": "HasBazChoice",
"States": {
"HasBazChoice": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.baz",
"IsPresent": true,
"Next": "MakeHasBazItem"
}
],
"Default": "MakeNoBazItem"
},
"MakeHasBazItem": {
"Type": "Pass",
"Parameters": {
"data": {
"foo": { "S.$": "$.foo"},
"bar": { "S.$": "$.bar"},
"baz": { "S.$": "$.baz"}
}
},
"Next": "PutItemTask"
},
"MakeNoBazItem": {
"Type": "Pass",
"Parameters": {
"data": {
"foo": {"S.$": "$.foo"},
"bar": {"S.$": "$.bar"}
}
},
"Next": "PutItemTask"
},
"PutItemTask": {
...
"Parameters": {
"TableName": "my-table",
"Item.$": "$.data"
}
},
}
}
If you have more than one optional field, your lambda backup plan is the better option - the above workaround would become unwieldy.
I have started Alexa development very recently. Today I have suddenly started getting
which I did not encounter before.
The lambda function (index.js):
"use strict";
const Alexa = require("ask-sdk-core");
const http = require("http");
exports.handler = async (event, context, callback) => {
try {
if (event.request.type === "LaunchRequest") {
var welcomeMessage = '<speak>Hi</speak>';
callback(null, buildResponse(welcomeMessage, false));
}
else if (event.request.type === "AMAZON.CancelIntent") {
var msg2 = "<speak>Stopped!</speak>";
callback(null, buildResponse(msg2, true));
}
} catch (e) {
context.fail("Exception: ${e}");
}
};
function buildResponse(response, shouldEndSession) {
return {
version: "1.0",
response: {
outputSpeech: {
type: "SSML",
ssml: response
},
shouldEndSession: shouldEndSession
},
sessionAttributes: {}
};
}
package.json file:
{
"name": "third-test-skill",
"version": "1.0.0",
"description": "...",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Subrata Sarkar",
"license": "ISC",
"dependencies": {
"ask-sdk-core": "^2.5.2",
"ask-sdk-model": "^1.15.1"
}
}
Steps I followed to create the skill:
Crated a skill from AWS Alexa console
Created skill
Added sample utterances
Selected Lamda as end point Created a function called movieFacts
Uploaded .zip file containing the following file structure.
node_modules
|- ask-sdk-core
|- ask-sdk-model
|- ask-sdk-runtime
index.js
package.json
package.json.lock
When I say movie facts, I am getting the following message:
The requested skill did not provide a valid response
And this is the JSON output I am receiving:
{
"version": "1.0",
"session": {
"new": false,
"sessionId": "amzn1.echo-api.session.beca8832-50fe-4d17-96a4-30c855b18a4f",
"application": {
"applicationId": "amzn1.ask.skill.bdb88b1b-5a4a-4b37-9b63-71e78337bbca"
},
"user": {
"userId": "amzn1.ask.account.AEG2YALM6KQANVKR3YSUWKVN5DCKE66NJKN23SZIKRKZCVTU67E2JBZ5STPFIN325WNGGO5Z73FMVVL5X2SVEM27YEPD5VFNMPVDQSQK5XYW3NXOXSEIK6YPHE5HTZLGLCWW4VVQHLYECL6YBLG4XOTM2HTV5VCCQMPLVCIATFRSNS4DLHJFLY32JHD5N5MAPFBNRVN3YV7B53A"
}
},
"context": {
"System": {
"application": {
"applicationId": "amzn1.ask.skill.bdb88b1b-5a4a-4b37-9b63-71e78337bbca"
},
"user": {
"userId": "amzn1.ask.account.AEG2YALM6KQANVKR3YSUWKVN5DCKE66NJKN23SZIKRKZCVTU67E2JBZ5STPFIN325WNGGO5Z73FMVVL5X2SVEM27YEPD5VFNMPVDQSQK5XYW3NXOXSEIK6YPHE5HTZLGLCWW4VVQHLYECL6YBLG4XOTM2HTV5VCCQMPLVCIATFRSNS4DLHJFLY32JHD5N5MAPFBNRVN3YV7B53A"
},
"device": {
"deviceId": "amzn1.ask.device.AFXLD474IMHMD5V35NT2ZNUD5YLK2LTEJZUMO6DS2MY7ANONMZDZ67C3MU44OBJ6B5N4TPOXIJ64FBEFEOVOB2K4SSYEN3VTRSIHZETNTBNCDYUG6RGFIOKH7S7OBID6CG3WIHB774LNO4CFKWFUXYSNHD5HIAAXCEDKZ3U4EN7QB6EN4RRHQ",
"supportedInterfaces": {}
},
"apiEndpoint": "https://api.amazonalexa.com",
"apiAccessToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjEifQ.eyJhdWQiOiJodHRwczovL2FwaS5hbWF6b25hbGV4YS5jb20iLCJpc3MiOiJBbGV4YVNraWxsS2l0Iiwic3ViIjoiYW16bjEuYXNrLnNraWxsLmJkYjg4YjFiLTVhNGEtNGIzNy05YjYzLTcxZTc4MzM3YmJjYSIsImV4cCI6MTU1NzI0MTIxMCwiaWF0IjoxNTU3MjQwOTEwLCJuYmYiOjE1NTcyNDA5MTAsInByaXZhdGVDbGFpbXMiOnsiY29uc2VudFRva2VuIjpudWxsLCJkZXZpY2VJZCI6ImFtem4xLmFzay5kZXZpY2UuQUZYTEQ0NzRJTUhNRDVWMzVOVDJaTlVENVlMSzJMVEVKWlVNTzZEUzJNWTdBTk9OTVpEWjY3QzNNVTQ0T0JKNkI1TjRUUE9YSUo2NEZCRUZFT1ZPQjJLNFNTWUVOM1ZUUlNJSFpFVE5UQk5DRFlVRzZSR0ZJT0tIN1M3T0JJRDZDRzNXSUhCNzc0TE5PNENGS1dGVVhZU05IRDVISUFBWENFREtaM1U0RU43UUI2RU40UlJIUSIsInVzZXJJZCI6ImFtem4xLmFzay5hY2NvdW50LkFFRzJZQUxNNktRQU5WS1IzWVNVV0tWTjVEQ0tFNjZOSktOMjNTWklLUktaQ1ZUVTY3RTJKQlo1U1RQRklOMzI1V05HR081WjczRk1WVkw1WDJTVkVNMjdZRVBENVZGTk1QVkRRU1FLNVhZVzNOWE9YU0VJSzZZUEhFNUhUWkxHTENXVzRWVlFITFlFQ0w2WUJMRzRYT1RNMkhUVjVWQ0NRTVBMVkNJQVRGUlNOUzRETEhKRkxZMzJKSEQ1TjVNQVBGQk5SVk4zWVY3QjUzQSJ9fQ.UyCg4MXlOe16SlOyJnjAIiHzVpdLkRjd-izoKkUnGqiyZ0L_5eUpg8tKvVrCvTLNMtJS6ElksxgVfuLcNeOIwSbXtYCOXcSLRYbpcpgFI6oeamOZ2Yo-UMDEjzYi75fABuJyUJyZxp-Pieer8PMZO4G9-5zJXCVY2x3M_dmlpX23UBJDpW0DKddvAOzConmwgdaf3v_EWfc2q8BaCQIM950rEUbejOa08_AwE5CsqjNA9sD22QduE5hs09RV4-F-kU1zKvwwyDVDKyOkdFZQFEmCTC11_jI64re9c22e-hYR4leIE5XntNApMgtwaL-tHyjsJzVDVDfZd2q3w6wxYA"
},
"Viewport": {
"experiences": [
{
"arcMinuteWidth": 246,
"arcMinuteHeight": 144,
"canRotate": false,
"canResize": false
}
],
"shape": "RECTANGLE",
"pixelWidth": 1024,
"pixelHeight": 600,
"dpi": 160,
"currentPixelWidth": 1024,
"currentPixelHeight": 600,
"touch": [
"SINGLE"
],
"video": {
"codecs": [
"H_264_42",
"H_264_41"
]
}
}
},
"request": {
"type": "SessionEndedRequest",
"requestId": "amzn1.echo-api.request.c7b1b910-6309-48aa-af35-10ac0a20b5da",
"timestamp": "2019-05-07T14:55:10Z",
"locale": "en-US",
"reason": "ERROR",
"error": {
"type": "INVALID_RESPONSE",
"message": "An exception occurred while dispatching the request to the skill."
}
}
}
I think removing the <speak> and </speak> tags around the welcome message should do. They are by default generated by Alexa, so you don't need to provide them in your response.
What happened in my case (and definitely worth to check) is that I was using Serverless Framework to deploy Lambda and I changed the configuration in the serverless.yml but forgot to update the Lambda ARN in the Alexa Endpoint.
Updating the Endpoint to the correct ARN and rebuilding solved the issue.
I have my schema set up almost exactly how I want it in redux state, except I want to add an array of form ids to the formTemplate object.
It would look like this:
// Normalized Form Templates
{
1: {
id: '1',
isGlobal: true,
name: 'Form Template Name',
forms: [1, 2], // This is the line I want to add...but how?
},
}
// Normalized Forms
{
1: {
id: '1',
createdAt: '2016-12-28T23:30:13.547Z',
name: 'Form 1',
parentTemplate: '1',
pdfs: [1, 2],
},
2: {
id: '2',
createdAt: '2016-12-28T23:30:13.547Z',
name: 'Form 2',
parentTemplate: '1',
pdfs: [],
},
}
Here is my schema
import { schema } from 'normalizr'
const formTemplate = new schema.Entity('formTemplates', {}, {
processStrategy: value => ({
id: value.id,
name: value.attributes.title,
isGlobal: value.attributes.is_global,
}),
})
const form = new schema.Entity('forms', {
pdfs: [pdf],
}, {
processStrategy: value => ({
id: value.id,
createdAt: value.attributes.created_at,
name: value.attributes.title,
parentTemplate: value.attributes.form_template_id,
pdfs: [...value.relationships.documents.data],
}),
})
const pdf = new schema.Entity('pdfs')
export default {
data: [form],
included: [formTemplate],
}
This is an example of the API response that I'm normalizing
{
"data": [
{
"id": "5",
"type": "provider_forms",
"attributes": {
"title": "Form 1",
"created_at": "2017-01-02T06:00:42.518Z",
"form_template_id": 1
},
"relationships": {
"form_template": {
"data": {
"id": "1",
"type": "form_templates"
}
},
"documents": {
"data": [ // some pdf data here ]
}
}
}
],
"included": [
{
"id": "1",
"type": "form_templates",
"attributes": {
"title": "Form Template",
"created_at": "2016-12-29T22:24:36.201Z",
"updated_at": "2017-01-02T06:00:20.205Z",
"is_global": true
},
}
]
}
You won't be able to do this with Normalizr because the form template entity has no context back to the forms entities. Your actions/reducer need to handle this.
Okay I figured out a way to do this. I changed my formTemplate entity to map them in manually like so:
const formTemplate = new schema.Entity('formTemplates', {}, {
processStrategy: (value, parent) => {
// eslint-disable-next-line eqeqeq
const childrenForms = parent.data.filter(form => form.attributes.form_template_id == value.id)
return {
id: value.id,
name: value.attributes.title,
isGlobal: value.attributes.is_global,
forms: childrenForms.map(form => form.id),
}
},
})
I used the answer provided here:
google-services.json for different productFlavors to add a google-services.json files to my app/src/dev flavor folder and one to my app/src/production flavor folder. My dev debug variant will generate Events in my dashboard but my production release variant does not. The tracking_id is correct so not sure what I am missing.
*I DO see a real Time Event for productionDebug variant, but why not productionRelease?
Also I have an res/xml/app_tracker.xml in each of my flavor folders
with each having its respective correct ga_trackingId
Here is my google-services-json file for my production flavor:
{
"project_info": {
"project_id": "shopper-64392",
"project_number": "475366498847",
"name": "Shopper"
},
"client": [
{
"client_info": {
"mobilesdk_app_id": "1:475366498847:android:847eba250b1fe1a7",
"client_id": "android:ts.kiosk.app.shoppers",
"client_type": 1,
"android_client_info": {
"package_name": "ts.kiosk.app.shoppers",
"certificate_hash": []
}
},
"oauth_client": [],
"api_key": [],
"services": {
"analytics_service": {
"status": 2,
"analytics_property": {
"tracking_id": "UA-76684740-1"
}
},
"cloud_messaging_service": {
"status": 1,
"apns_config": []
},
"appinvite_service": {
"status": 1,
"other_platform_oauth_client": []
},
"google_signin_service": {
"status": 1
},
"ads_service": {
"status": 1
}
}
}
],
"client_info": [],
"ARTIFACT_VERSION": "1"
}
I'm trying my darnedest to do an ETL import from a large dataset that I've been keeping in MongoDB. I've successfully imported the vertices, and I feel like I'm one little syntax misunderstanding away from importing the edges too.
I am pretty sure that the error is in this transformer:
{"edge":{"class":"Friend", "joinFieldName":"id",
"lookup": "select from Character WHERE $input.id IN character_friends",
"unresolvedLinkAction":"CREATE"}},
So what I'm trying to do is to make an edge from a document with id = FOO to all other documents that contain FOO in their character_friends array.
If I execute
select from Character WHERE FOO IN character_friends
in the browser, I get a ton of documents, so my guess is that my problem is with $input.id either not returning the id I'm expecting, or maybe not being recognized as a variable at all.
Documents look like this:
{
id: FOO,
character_friends: [BAR, BAZ, QUX]
(and a bunch of other junk)
}
Seems you're inserting a property "id", but it's reserved in Blueprints standard. You can rename it (with "field" transformers) or set this in Orient Loader:
standardElementConstraints: false,
Then I've created the file /temp/datasets/charles.json with this content:
[
{
name: "Joe",
id: 1,
friends: [2,4,5],
enemies: [6]
},
{
name: "Suzie",
id: 2,
friends: [1,4,6],
enemies: [5,2]
}
]
And this pipeline:
{
config: {
log: "debug",
parallel: false
},
source : {
file: { path: "/temp/datasets/charles.json", lock : true }
},
extractor : {
json: {}
},
transformers : [
{ merge: { joinFieldName:"id", lookup:"Account.id" } },
{ vertex: { class: "Account"} },
{ edge: {
"class": "Friend",
"joinFieldName": "friends",
"lookup": "Account.id",
"unresolvedLinkAction": "CREATE"
} },
{ edge: {
"class": "Enemy",
"joinFieldName": "enemies",
"lookup": "Account.id",
"unresolvedLinkAction": "CREATE"
} }
],
loader : {
orientdb: {
dbURL: "plocal:/temp/databases/charles",
dbUser: "admin",
dbPassword: "admin",
dbAutoDropIfExists: true,
dbAutoCreate: true,
standardElementConstraints: false,
tx: false,
wal: false,
batchCommit: 1000,
dbType: "graph",
classes: [{name: 'Account', extends:"V"}, {name: 'Friend', extends:"E"}, {name: 'Enemy', extends:"E"}],
indexes: [{class:"Account", fields:["id:integer"], type:"UNIQUE_HASH_INDEX" }]
}
}
}
Assure to use last version of ETL jar (replace it in $ORIENTDB/lib) with default version. Last version is downloadable from:
https://oss.sonatype.org/content/repositories/snapshots/com/orientechnologies/orientdb-etl/2.0.2-SNAPSHOT/orientdb-etl-2.0.2-20150208.225903-1.jar
Or get OrientDB ETL 2.0.2 of major.