I need to store a hash of uuids under an specific object key, something like:
{
myHash: {
"20948-d32d2-d2d2..": "some value 1",
"20778-d7322-j5j5..": "some value 2",
...
}
...
}
For that I've used the blackbox key of meteor-simple-schema:
myHash: {
type: Object,
blackbox: true,
},
That's working: the hash keys are saving to the db.
Problems is, I can't figure out how to add schema verifications to the value of the blackbox keys: "some value 1" and "some value 2".
For example, assuming I need to enforce a max length. How would I implement it?
You could use custom function to do the validation here:
myHash: {
type: Object,
optional: true,
blackbox: true,
custom() {
const value = this.value;
if(/* value is not validated */) {
return 'notAllowed';
}
},
},
Related
I am trying to run a simple update query, but got an error when I tried to update the key of a map to the same value. Is there a technical reason this would be disallowed? or some kind of best-practice that I am violating by trying to do this?
Error:
ValidationException: Invalid UpdateExpression: Two document paths overlap with each other;
must remove or rewrite one of these paths; path one: [questions, What is xx?], path two: [questions, What is xx?]
Query object:
{
TableName: 'notesTable',
Key: { topic: 'My tooic' },
ExpressionAttributeNames: { '#qq': 'What is xx?', '#updq': 'What is xx?' },
ExpressionAttributeValues: { ':updans': 'new answer' },
UpdateExpression: 'REMOVE questions.#qq SET questions.#updq = :updans'
}
Multiple ways to deal with scenario when same key needs to be updated. Instead of removing and updating the same key, we can simply SET the key , which replaced the value anyhow.
So, simple way is to send different updateExpression each time.
const qq = "What is xx2?";
const updq = "What is xx?";
let expressionAttributeNames;
let UpdateExpression;
if (qq === updq) {
expressionAttributeNames = { "#updq": "What is xx?" };
UpdateExpression = "SET questions.#updq = :updans";
} else {
expressionAttributeNames = { "#qq": "What is xx1?", "#updq": "What is xx?" };
UpdateExpression = "REMOVE questions.#qq SET questions.#updq = :updans";
}
docClient.update(
{
TableName: "test",
Key: {
id: "My tooic",
},
ExpressionAttributeNames: expressionAttributeNames,
ExpressionAttributeValues: { ":updans": "new answer1" },
UpdateExpression: UpdateExpression,
},
function (error, result) {
console.log("error", error, "result", result);
}
);
I'm trying to insert an array into an object and I'm not having any luck. I think the schema is rejecting it based on validation but I'm not sure why. If I console.log(this.state.typeOfWork) and check typeof it states its an Object which contains:
(2) ["Audit - internal", "Audit - external"]
0: "Audit - internal"
1: "Audit - external"
My collection after an update contains:
"roleAndSkills": {
"typeOfWork": []
}
Example: Schema
roleAndSkills: { type: Object, optional: true },
'roleAndSkills.typeOfWork': { type: Array, optional: true },
'roleAndSkills.typeOfWork.$': { type: String, optional: true }
Example: update
ProfileCandidate.update(this.state.profileCandidateCollectionId, {
$set: {
roleAndSkills: {
typeOfWork: [this.state.typeOfWork]
}
}
});
Simple schema has some problems with validation on Objects or Arrays, i had the same problem in a recent app i developed.
What can you do?
well, what i did, on the Collections.js file, when you are saying:
typeOfWork:{
type: Array
}
Try adding the property blackbox:true, like this:
typeOfWork:{
blackbox: true,
type: Array
}
This will tell your Schema that this field is taking an Array, but ignore further validation.
The validation i made was on main.js, just to be sure i had no empty array and the data was plain text.
As requested here is my update method, im my case i used objects not arrays but it works the same way.
editUser: function (editedUserVars, uid) {
console.log(uid);
return Utilizadores.update(
{_id: uid},
{$set:{
username: editedUserVars.username,
usernim: editedUserVars.usernim,
userrank: {short: editedUserVars.userrank.short,
long: editedUserVars.userrank.long},
userspec: {short: editedUserVars.userspec.short,
long: editedUserVars.userspec.long},
usertype: editedUserVars.usertype}},
{upsert: true})
},
here it the collection schema
UtilizadoresSchema = new SimpleSchema({
username:{
type: String
},
usernim:{
type: String
},
userrank:{
blackbox: true,
type: Object
},
userspec:{
blackbox: true,
type: Object
},
usertype:{
type: String
}
});
Utilizadores.attachSchema(UtilizadoresSchema);
Hope it helps
Rob
typeOfWork is an Array. You should push your value in it :
$push: {
"roleAndSkills.typeOfWork": this.state.typeOfWork
}
for multiple values :
$push: {
"roleAndSkills.typeOfWork": { $each: [ "val1", "val2" ] }
}
mongo $push operator
mongo dot notation
You state that this.state.typeOfWork is an array (of strings) but then when you .update() your document you are enclosing it in square brackets:
ProfileCandidate.update(this.state.profileCandidateCollectionId, {
$set: {
roleAndSkills: {
typeOfWork: [this.state.typeOfWork]
}
}
});
Simply remove the redundant square brackets:
ProfileCandidate.update(this.state.profileCandidateCollectionId, {
$set: {
roleAndSkills: {
typeOfWork: this.state.typeOfWork
}
}
});
Also since your array is just an array of strings you can simplify your schema a bit by declaring it as such with [String] for the type:
'roleAndSkills.typeOfWork': { type: [String] }
Note furthermore that objects and arrays are by default optional so you can even omit the optional flag.
How do you "upsert" a property to a DynamoDB row. E.g. SET address.state = "MA" for some item, when address does not yet exist?
I feel like I'm having a chicken-and-egg problem because DynamoDB doesn't let you define a sloppy schema in advance.
If address DID already exist on that item, of type M (for Map), the internet tells me I could issue an UpdateExpression like:
SET #address.#state = :value
with #address, #state, and :value appropriately mapped to address, state, and MA, respectively.
But if the address property does not already exist, this gives an error:
'''
ValidationException: The document path provided in the update expression is invalid for update
'''
So.. it appears I either need to:
Figure out a way to "upsert" address.state (e.g., SET address = {}; SET address.state = 'MA' in a single command)
or
Issue three (!!!) roundtrips in which I try it, SET address = {}; on failure, and then try it again.
If the latter.... how do I set a blank map?!?
Ugh.. I like Dynamo, but unless I'm missing something obvious this is a bit crazy..
You can do it with two round trips, the first conditionally sets an empty map for address if it doesn't already exist, and the second sets the state:
db.update({
UpdateExpression: 'SET #a = :value',
ConditionExpression: 'attribute_not_exists(#a)',
ExpressionAttributeValues: {
":value": {},
},
ExpressionAttributeNames: {
'#a': 'address'
}
}, ...);
Then:
db.update({
UpdateExpression: 'SET #a.#b = :v',
ExpressionAttributeNames: {
'#a': 'address',
'#b': 'state'
},
ExpressionAttributeValues: {
':v': 'whatever'
}
}, ...);
You cannot set nested attributes if the parent document does not exist. Since address does not exist you cannot set the attribute province inside it. You can achieve your goal if you set address to an empty map when you create the item. Then, you can use the following parameters to condition an update on an attribute address.province not existing yet.
var params = {
TableName: 'Image',
Key: {
Id: 'dynamodb.png'
},
UpdateExpression: 'SET address.province = :ma',
ConditionExpression: 'attribute_not_exists(address.province)',
ExpressionAttributeValues: {
':ma': 'MA'
},
ReturnValues: 'ALL_NEW'
};
docClient.update(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
By the way, I had to replace state with province as state is a reserved word.
Another totally different method is to simply create the address node when creating the parent document in the first place. For example assuming you have a hash key of id, you might do:
db.put({
Item: {
id: 42,
address: {}
}
}, ...);
This will allow you to simply set the address.state value as the address map already exists:
db.update({
UpdateExpression: 'SET #a.#b = :v',
AttributeExpressionNames: {
'#a': 'address',
'#b': 'state'
},
AttributeExpressionValues: {
':v': 'whatever'
}
}, ...);
Some kotlin code to do this recursively regardless how deep it goes. It sets existence of parent paths as condition and if condition check fails, recursively creates those paths first. It has to be in the library's package so it can access those package private fields/classes.
package com.amazonaws.services.dynamodbv2.xspec
import com.amazonaws.services.dynamodbv2.document.Table
import com.amazonaws.services.dynamodbv2.model.ConditionalCheckFailedException
import com.amazonaws.services.dynamodbv2.xspec.ExpressionSpecBuilder.attribute_exists
fun Table.updateItemByPaths(hashKeyName: String, hashKeyValue: Any, updateActions: List<UpdateAction>) {
val parentPaths = updateActions.map { it.pathOperand.path.parent() }
.filter { it.isNotEmpty() }
.toSet() // to remove duplicates
try {
val builder = ExpressionSpecBuilder()
updateActions.forEach { builder.addUpdate(it) }
if (parentPaths.isNotEmpty()) {
var condition: Condition = ComparatorCondition("=", LiteralOperand(true), LiteralOperand(true))
parentPaths.forEach { condition = condition.and(attribute_exists<Any>(it)) }
builder.withCondition(condition)
}
this.updateItem(hashKeyName, hashKeyValue, builder.buildForUpdate())
} catch (e: ConditionalCheckFailedException) {
this.updateItemByPaths(hashKeyName, hashKeyValue, parentPaths.map { M(it).set(mapOf<String, Any>()) })
this.updateItemByPaths(hashKeyName, hashKeyValue, updateActions)
}
}
private fun String.parent() = this.substringBeforeLast('.', "")
Here is a helper function I wrote in Typescript that works for this a single level of nesting using a recursive method.
I refer to the top-level attribute as a column.
//usage
await setKeyInColumn('customerA', 'address', 'state', "MA")
// Updates a map value to hold a new key value pair. It will create a top-level address if it doesn't exist.
static async setKeyInColumn(primaryKeyValue: string, colName: string, key: string, value: any, _doNotCreateColumn?:boolean) {
const obj = {};
obj[key] = value; // creates a nested value like {address:value}
// Some conditions depending on whether the column already exists or not
const ConditionExpression = _doNotCreateColumn ? undefined:`attribute_not_exists(${colName})`
const AttributeValue = _doNotCreateColumn? value : obj;
const UpdateExpression = _doNotCreateColumn? `SET ${colName}.${key} = :keyval `: `SET ${colName} = :keyval ` ;
try{
const updateParams = {
TableName: TABLE_NAME,
Key: {key:primaryKeyValue},
UpdateExpression,
ExpressionAttributeValues: {
":keyval": AttributeValue
},
ConditionExpression,
ReturnValues: "ALL_NEW",
}
const resp = await docClient.update(updateParams).promise()
if (resp && resp[colName]) {
return resp[colName];
}
}catch(ex){
//if the column already exists, then rerun and do not create it
if(ex.code === 'ConditionalCheckFailedException'){
return this.setKeyInColumn(primaryKeyValue,colName,key, value, true)
}
console.log("Failed to Update Column in DynamoDB")
console.log(ex);
return undefined
}
}
I've got quite similar situation. I can think of only a one way to do this in 1 query/atomically.
Extract map values to top level attributes.
Example
Given I have this post item in DynamoDB:
{
"PK": "123",
"SK": "post",
"title": "Hello World!"
}
And I want to later add an analytics entry to same partition:
{
"PK": "123",
"SK": "analytics#december",
"views": {
// <day of month>: <views>
"1": "12",
"2": "457463",
// etc
}
}
Like in your case, it's not possible to increment/decrement views days counters in single query if analytics item nor views map might not exist (could be later feature or don't want to put empty items).
Proposed solution:
{
"PK": "123",
"SK": "analytics#december",
// <day of month>: <views>
"1": "12", // or "day1" if "1" seems too generic
"2": "457463",
// etc
}
}
Then you could do something like this (increment +1 example):
{
UpdateExpression: "SET #day = if_not_exists(#day, 0) + 1",
AttributeExpressionNames: {
'#day': "1"
}
}
if day attribute value doesn't exist, set default value to 0
if item in database doesn't exist, update API adds a new one
Let's say, we have this schema :
Schemas.MyCollection = new SimpleSchema({
something: {
type: Object
}
});
I want to insert something into MyCollection. For example :
var myobj = {
aaaaaa: 11111,
bbbbbb: 22222
};
MyCollection.insert({something: myobj});
We end up with this :
{
_id: "someId",
something: {}
}
When I disable simple schema checking (collection2), everything works as one expected.
Simple-schema did not report an error (collection2) so why it is invalid?
#Seraph your schema is wrong
Schemas.MyCollection = new SimpleSchema({
something: {
type: Object
},
'something.aaaaa': {
type: String
}
});
and so on you have to write every property the object has or you can do blackbox: true if you don't want to validate the object:
something: {
type: Object,
blackbox: true
}
Also if it's server-side operation you can do myCollection.insert(doc, {validate: false});
just read the docs https://atmospherejs.com/aldeed/collection2 :)
Here is the reference to help you understand more:
https://github.com/aldeed/meteor-simple-schema#blackbox
From what I understand in the docs, you can define your schema like this:
MySchema = new SimpleSchema({
//This says that the addresses key is going to contain an array
addresses: {
type: [Object],
},
// To indicate the presence of an array, use a $:
"addresses.$.street": {
type: String,
},
"addresses.$.city": {
type: String,
}
});
Ok, I get this part. But what if I wanted to validate the contents in a specific array index? I want something like this:
MySchema = new SimpleSchema({
//This says that the itemsOrdered key is going to contain an array
itemsOrdered: {
type: [Object],
},
// Here I want to validate certain indexes in the array.
"itemsOrdered.0.sku": {
type: String
},
"itemsOrdered.0.price": {
type: Number
},
"itemsOrdered.1.sku": {
type: String
},
"itemsOrdered.1.price": {
type: Number
},
"itemsOrdered.1.quantity": {
type: Number
},
"itemsOrdered.2.sku": {
type: String
},
"itemsOrdered.2.price": {
type: Number
},
"itemsOrdered.2.customerNotes": {
type: String
optional: true
}
});
So here I'm trying to validate the values inside array index 0, 1, and 2. Each array index has a different item that has been ordered.
Normally I would use a hash table data structure, but for this purpose I need to preserve order which is why I'm using an array.
When I try to run this code I get an error Cannot read property 'blackbox' of undefined
Have you considered custom validation?
https://github.com/aldeed/meteor-simple-schema/blob/master/README.md#custom-validation
According to the doc within the function the key property of this will provide the information you want. So you could have something like:
MySchema = new SimpleSchema({
//This says that the itemsOrdered key is going to contain an array
itemsOrdered: {
type: [Object],
},
// Here I want to validate certain indexes in the array.
"itemsOrdered.$.sku": {
type: String,
custom: function () {
var key = this.key,
re = /\d+/;
var index = Number(key.match(re)[0]);
// Do some custom validation
}
},
"itemsOrdered.$.price": {
type: Number
},
"itemsOrdered.$.quantity": {
type: Number,
optional: true
},
"itemsOrdered.$.customerNotes": {
type: String,
optional: true
}
});
Here I put the validation logic in the sku field since it's required.