Sequelize returning `null` object for hasOne association's linked model after bulk inserting pre-existed data - sqlite

I have a pre-existing data in the form of JSON files which I am trying to insert into Sequelize SQLite table. I have 2 tables: Master_Vessel and Requests. There is a Request.hasOne(Vessel) association, meaning that every request has a vessel associated with it.
The key connecting both the models is vessel.Vessel_Name, which is a string. The data has been generated from a system which validates the keys so it is consistent (every request.Vessel_Name exists in vessel.Vessel_Name, vessel.Vessel_Name is unique).
I want to load this data into Sequelize managed table. I am creating the models as follows:
var Request = sequelize.define('request', {
// other fields
Vessel_Name: {
type: Sequelize.STRING
}
}, {
tableName: 'Request'
});
var Vessel = sequelize.define('vessel', {
// other fields
Vessel_Name: {
type: Sequelize.STRING,
primaryKey: true
}
}, {
tableName: 'Master_Vessel'
});
And, the association is as follows:
Request.hasOne(Vessel, {foreignKey: 'Vessel_Name', allowNull: true});
Now, I am loading the data in following steps:
Create and bulk load the Vessel model
Create the Request model, the association and bulk load the Request model
Everything runs fine without any issues. However when I try to query using the association as follows:
Request.findOne()
.then(request => {
console.log(request.get())
request.getVessel()
.then(vessel => console.log(vessel))
});
request instance contains the Vessel_Name foreign key, however the vessel instance is null.
Can anybody please help?

Okay this is happenning because I got the Sequelize 'vocabulary' mixed up, Request.hasOne(Vessel) does NOT mean Request will have one Vessel_Name that will point to a Vessel. It is to be read R->L for correct meaning, not L->R.
The solution to the problem was changing the association from hasOne to belongsTo as follows: Request.belongsTo(Vessel), so now the FK will be created on Request.
Quite a confusing choice of association names and semantics!

Related

How to manually set the Firestore document ID saving a new document calling Firestore API?

I am finding the following problem trying to save a new document into Firestore database by calling the related POST API and manually setting the document ID.
I am using Python but I suppose that the problem is not related to the language.
I try to explain what I have done and what is not working. My first attempt (that works but automatically set the document ID on Firestore) was:
First of all, I created this JSON document that will be the payload of my API:
# Convert the record to a dictionary
doc = {
'fields': {
'surname': {'stringValue':record[2]},
'firstName': {'stringValue':record[1]},
'socialSecurityCode': {'stringValue':codici_fiscali_list_as_string},
'city': {'stringValue':record[4]},
'personalPhone': {'stringValue':record[5]},
'personalPhone2': {'stringValue':record[6]},
'personalEmail': {'stringValue':emails_list_as_string},
'pazPres': {'stringValue':record[7]},
'pazNotes': {'stringValue':record[8]},
'pazMemo': {'stringValue':record[9]},
'isArchived': {'booleanValue':isArchived},
'isMigrated': {'booleanValue':True},
#'decomposition_keyword_search_list':{'arrayValue':{'values':decomposition_keyword_search_list}}
"decomposition_keyword_search_list":{
"arrayValue":{
"values":[
]
}
}
}
}
then I perform the API call by these lines:
api_endpoint = 'https://firestore.googleapis.com/v1/projects/MY-PROJECT-NAME/databases/(default)/documents/test/'
response = requests.post(api_endpoint, json=doc)
It works fine and it put the expected document into my test collection. But in this way, the ID was automatically generated by Firestore. For some reason, I have to use the content of a variable as ID (my ID must be the value of my record[0] that is a unique string)
So I tried to change the previous API endpoint in the following way:
api_endpoint = 'https://firestore.googleapis.com/v1/projects/MY-PROJECT-NAME/databases/(default)/documents/test/'+ record[0]
I expected that it creates a document using the record[0] as a document ID but it seems that I am wrong since I am obtaining the following error message:
Error saving document: {
"error": {
"code": 400,
"message": "Document parent name \"projects/MY-PROJECT-NAME/databases/(default)/documents/test\" lacks \"/\" at index 71.",
"status": "INVALID_ARGUMENT"
}
}
So, what is wrong? What am I missing? How can correctly manually set the ID of the document that I am creating calling the previous API?
Take a look at the documentation for creating documents. If you want to specify a document ID, it says you should pass that as a query parameter called documentId:
The client-assigned document ID to use for this document.
Optional. If not specified, an ID will be assigned by the service.

findAll() returns empty with WHERE option

First question on StackOverflow, long time reader first time poster or whatever people say.
I'm developing a Discord bot in my free time using Discord.js, and I'm using Sequelize to interface with a local SQLite database. I can insert data into it just fine-- however, I can't seem to delete any of the records I add. Relevant piece of code is below, which I believe to be self-contradictory:
const query3 = await Towers.findAll({
attributes: ['channelID']
});
console.log(JSON.stringify(query3)); //returns the one Tower
console.log(query3[0].channelID === channel); //returns true(!)
const query2 = await Towers.findAll({
attributes: ['channelID'],
where: {channelID: channel}
});
console.log(JSON.stringify(query2)); //returns empty
//DELETE FROM Towers WHERE channelID = channel;
const query = await Towers.destroy({
where: {channelID: channel}
});
console.log(query); //returns 0, expected behavior given query2 returns empty
I'm attempting to delete a record from a table named Towers by passing a channel ID to it, which is expected to be unique. However, when I make any query on the database with a WHERE clause, the query returns an empty set-- even when, in this example, I sanity-checked and verified that the value I'm attempting to remove is present in the table. This occurs for both findAll() and findOne() as long as a WHERE clause is present.
(For posterity, I've double and triple checked that channelID was spelled correctly and with the correct capitalization in all instances.)
I'm happy to provide any more information if needed!
EDIT: As requested, the model definition...
const Towers = sequelize.define('Towers', {
serverID: {
type: Sequelize.INTEGER,
allowNull: false,
},
channelID: {
type: Sequelize.INTEGER,
unique: true,
allowNull: false,
},
pattern: Sequelize.STRING,
height: Sequelize.INTEGER,
delay: Sequelize.BOOLEAN,
});
channel in the snippet in the original post is defined as parseInt(interaction.options.getChannel('channel').id).
To anyone who happens to have the same issue I did, the answer is a doozy.
I wanted to store Discord server and channel ID's as integers, even though they're returned to you as strings when calling the API. As it turns out, Discord snowflakes are higher than float64 precision, which JS uses. When parsing the strings into integers to insert them into my table, the value changed from the intended number, and I was creating erroneous records.
In my case (with the actual numbers obfuscated) interaction.options.getChannel('channel').id returned "837512533934092340", while parseInt(interaction.options.getChannel('channel').id returned 837512533934092300. The number I was adding to the table was somehow 40 less!
I'm not sure if this could be fixed by using BigInt, but since it's going into a different structure anyway, I just shrugged and changed the serverId and channelId types to Sequelize.STRING in the model definition and removed the parseInt calls. Works like a charm now.
Good opportunity to shake my fist at JS though.

DynamoDB - Get Item by Global Secondary Index

I have an existing table which has 10 fields. Fields are like this:
AuthID, UserID, Age, Job, .etc
The table stores data of my users. "AuthID" is primary key and "UserID" is a Global Secondary Index.
When I get item by AuthID, everything is fine. But I can't get item by UserID. I tried GetItem, Query and Scan methods but I failed in all three method.
I need to be able to get data with these 3 methods I wrote below :
1 - Get user data by AuthID (It's already works fine)
2 - Get user data by UserID
3 - Get user data by AuthID and UserID both
AuthID and UserID is unique. Can someone point me right way as to what to do?
I have searched a lot in the documentation and found that if you need to get a single item even then you cant use the get or getItem method when using a global secondary index. One can use the query method. a sample of query method with global secondary index is
let params = {
TableName: "Users",
IndexName: "your-index",
ExpressionAttributeValues: {
":v1": "myid"
},
KeyConditionExpression: "my_partition_key_in_gsi = :v1",
};
dynamodb.query(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});

How to access an object into another object in a query database from firebase functions?

I am trying to access to a object into another object in my firebase database, i have a structure like this:
I want to get all the objects that have the email that i send by parameters, i am using .child to access to the childs into my object but i am not success with the query, this is my code
$ ref_db.child("/groups").child("members").orderByChild("email").equalTo(email).once("value", (snapshot)=>{
console.log(snapshot.val());
});
The snapshot.val() always is undefined.
could you help me with the query?
One efficient way to get "all the groups that have that email inside the members object" would be to denormalize you data and have another "main node" in your database where you store all "members" (i.e. their email) and the "groups" they belong to.
This means that each time you add a "member" node under a "group" (including its email) you will also add the group as a child of the member email, in this other "main node".
More concretely, here is how would be the database structure:
Your current structure:
- groups
- -LB9o....
...
- members
- -LB9qbd....
-email: xxxx#zzz.com
- -LBA7R....
-email: yyyyy#aaaa.com
And the extra structure:
- groupsByMembers
- xxxxxx#zzzcom
- Grupo1: true
- yyyyy#aaaacom
- Grupo1: true
- Grupo2: true
- bbbcccc#dddcom
- Grupo6: true
- Grupo8: true
Note that in the "extra structure" the dots within an email address are removed, since you cannot include a point in a node id. You will have to remove them accordingly when writing and querying.
This way you can easily query for the list of groups a member is belonging to, as shown below. Without the need to loop several times over several items. This dernomalization technique is quite classic in NoSQL databases.
const mailToSearchFor = xxxx.xx#zzz.com;
const ref = database.ref('/groupsByMembers/' + mailToSearchFor.replace(/\./g, ''));
ref.once('value', snapshot => {
const val = snapshot.val();
for (let key in val) {
if (val.hasOwnProperty(key)) {
console.log(key);
}
}
});
In order to write to the two database nodes simultaneously, use the update method as explained here https://firebase.google.com/docs/database/web/read-and-write#update_specific_fields
This is because you have a random key before members, you need to go through the path and not skip a node, to be able to access the values:
ref_db.child("groups").child("-LB9oWcnE0wXx8PbH4D").child("members").orderByChild("email").equalTo(email).once("value", (snapshot)=>{
console.log(snapshot.val());
});

Meteor Framework Subscribe/Publish according to document variables

I have a game built on Meteor framework. One game document is something like this:
{
...
participants : [
{
"name":"a",
"character":"fighter",
"weapon" : "sword"
},
{
"name":"b",
"character":"wizard",
"weapon" : "book"
},
...
],
...
}
I want Fighter character not to see the character of the "b" user. (and b character not to see the a's) There are about 10 fields like character and weapon and their value can change during the game so as the restrictions.
Right now I am using Session variables not to display that information. However, it is not a very safe idea. How can I subscribe/publish documents according to the values based on characters?
There are 2 possible solutions that come to mind:
1. Publishing all combinations for different field values and subscribing according to the current state of the user. However, I am using Iron Router's waitOn feature to load subscriptions before rendering the page. So I am not very confident that I can change subscriptions during the game. Also because it is a time-sensitive game, I guess changing subscriptions would take time during the game and corrupt the game pleasure.
My problem right now is the user typing
Collection.find({})
to the console and see fields of other users. If I change my collection name into something difficult to find, can somebody discover the collection name? I could not find a command to find collections on the client side.
The way this is usually solved in Meteor is by using two publications. If your game state is represented by a single document you may have problem implementing this easily, so for the sake of an example I will temporarily assume that you have a Participants collection in which you're storing the corresponding data.
So anyway, you should have one subscription with data available to all the players, e.g.
Meteor.publish('players', function (gameId) {
return Participants.find({ gameId: gameId }, { fields: {
// exclude the "character" field from the result
character: 0
}});
});
and another subscription for private player data:
Meteor.publish('myPrivateData', function (gameId) {
// NOTE: not excluding anything, because we are only
// publishing a single document here, whose owner
// is the current user ...
return Participants.find({
userId: this.userId,
gameId: gameId,
});
});
Now, on the client side, the only thing you need to do is subscribe to both datasets, so:
Meteor.subscribe('players', myGameId);
Meteor.subscribe('myPrivateData', myGameId);
Meteor will be clever enough to merge the incoming data into a single Participants collection, in which other players' documents will not contain the character field.
EDIT
If your fields visibility is going to change dynamically I suggest the following approach:
put all the restricted properties in a separated collection that tracks exactly who can view which field
on client side use observe to integrate that collection into your local player representation for easier access to the data
Data model
For example, the collection may look like this:
PlayerProperties = new Mongo.Collection('playerProperties');
/* schema:
userId : String
gameId : String
key : String
value : *
whoCanSee : [String]
*/
Publishing data
First you will need to expose own properties to each player
Meteor.publish('myProperties', function (gameId) {
return PlayerProperties.find({
userId: this.userId,
gameId: gameId
});
});
then the other players properties:
Meteor.publish('otherPlayersProperties', function (gameId) {
if (!this.userId) return [];
return PlayerProperties.find({
gameId: gameId,
whoCanSee: this.userId,
});
});
Now the only thing you need to do during the game is to make sure you add corresponding userId to the whoCanSee array as soon as the user gets ability to see that property.
Improvements
In order to keep your data in order I suggest having a client-side-only collection, e.g. IntegratedPlayerData, which you can use to arrange the player properties into some manageable structure:
var IntegratedPlayerData = new Mongo.Collection(null);
var cache = {};
PlayerProperties.find().observe({
added: function (doc) {
IntegratedPlayerData.upsert({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
changed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
removed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$unset: _.object([ doc.key ], [ true ])
});
}
});
This data "integration" is only a draft and can be refined in many different ways. It could potentially be done on server-side with a custom publish method.

Resources