I'm trying to add something to rooms/users using a data model that looks like this:
rooms: {
name: roomname
users: {
0: email#email.com
}
}
My question is if there is any way to append a new item to the users array. I would normally do this using update(), but update() requires a key for the data to be set to when I just want to set it to the next array index. I figure that I can do this by getting the current rooms/users array, appending to it locally, and using set() to overwrite it, but I was wondering if there was a better (built in) way to go about this.
Using arrays in a potentially massively distributed system such as Firebase are in general a bad idea. And from what you've described, your use-case falls under the "in general" category.
From the Firebase documentation on arrays:
Why not just provide full array support? Since array indices are not permanent, unique IDs, concurrent real-time editing will always be problematic.
Consider, for example, if three users simultaneously updated an array on a remote service. If user A attempts to change the value at key 2, user B attempts to move it, and user C attempts to change it, the results could be disastrous. For example, among many other ways this could fail, here's one:
// starting data
['a', 'b', 'c', 'd', 'e']
// record at key 2 moved to position 5 by user A
// record at key 2 is removed by user B
// record at key 2 is updated by user C to foo
// what ideally should have happened
['a', 'b', 'd', 'e']
// what actually happened
['a', 'c', 'foo', 'b']
Instead of using arrays, Firebase uses a concept called "push ids". These are consistently increasing (like array indices), but (unlike array indices) you don't have to know the current count to add a new push id.
With push ids, you can add a new user with:
var ref = new Firebase('https://yours.firebaseio.com/rooms/users');
ref.push('email#email.com');
Note that the Firebase documentation is in general considered to be pretty good. I highly recommend that you follow at least the programming guide for JavaScript, from which I copied the above.
Related
I'm creating a small game where people are able to join a room using a six-digit pin. Every room is represented by a document in a Firestore collection, where the room pin is the id of a room document.
My initial idea was to randomly generate a six-digit pin and check if a document with that id exists. If it does, create a new document with the generated pin, if not, generate a new pin and check if that id is free. This method will work, however, with a bit of bad luck it might cause a lot of unnecessary requests to the database.
My question is, therefore: is it possible to specify a format of the autogenerated id's? Or, if it is not possible, is there a way to fetch all documents id's to locally check whether or not the id exists?
You cannot specify a format for the auto-generated IDs but you can check if a room with same ID exists. If yes, then try new ID else create a room with same ID.
async function addRoom(roomId) {
const roomRef = admin.firestore().collection("rooms").doc(roomId)
if ((await roomRef.get()).exists) {
return addRoom(generatePin())
} else {
await roomRef.set({ ... })
}
return `${roomId} created`
}
function generatePin() {
return String(Math.floor(Math.random() * 999999) + 100000)
}
return addRoom(generatePin())
.then(() => console.log("Room created"))
.catch((e) => console.error(e))
PS: This might end up in some recursive state so I'd recommend using Firestore's IDs or uuid-int is you need numerical IDs only.
There is no way for you to specify the format of automatic IDs generated by the Firestore SDKs. They are purely random, and have an amount of entropy that statistically guarantees there won't be collisions (the chance where two clients generate the same ID is infinitesimally small), and minimizes the chance of hotspots (created when writing lots of documents or index values to the same area on a disk).
You can generate whatever auto-ID format you want however. You'll just have to accept a higher chance of collisions, as you already did, and the fact that you may experience hotspots when the documents are in the same area on a disk.
I have a button which runs the following code when it's clicked:
let dataReference = await db.collection("dog").doc("1").get()
let HashMap = dataReference.data().Annotations
console.log(HashMap)
My firestore database looks like this:
Whenever this function is run, it returns the proper dictionary, however, the ordering of the keys seems to change randomly. Here's a screenshot of my console logs when I pressed the button a bunch of times:
Why does the ordering of the key-value pairs change and is there a way to fix it?
The Firestore SDK does not guarantee an order of iteration of document fields. What you see in the console is always lexically sorted by the code of the console itself. If you require a stable ordering, you should sort them yourself before iteration.
One workaround you can do is to have the order of the keys you want in an array. Because arrays are ordered it will maintain the order you desired. Then you take that key and use it in the dictionary. While the dictionary is out of order, you will be accessing each value in order by key.
I'm currently trying to learn Symfony and a big part of it is Doctrine. I've been reading the official documentation for Doctrine and in the part about Collections library I stumbled upon this thing called "ordered map". I tried to search it on google, but I wasn't able to find any satisfying answer. There were just answers for specific languages (mostly Java and C++), but I want to understand it in general. How it works and what it is, because in the Doctrine documentation they are comparing it to the ArrayCollection, so I hope if I can understand what it is, it will be easier for me to understand ArrayCollection as well.
I tried to search for things like "what is an ordered map" or "ordered map explained", but as I said earlier, I didn't find what I was looking for.
A map is sometimes called ordered when the entries remain in the same sequence in which they were inserted.
For example, arrays in PHP are ordered (preserve insertion order). So creating/modifying an array like this:
$array = [2 => 'a', 1 => 'b'];
$array[0] = 'c';
will indeed result in the PHP array [2 => 'a', 1 => 'b', 0 => 'c'] - it preserves the insertion order - while in some other languages it will be turned into [0 => 'c', 1 => 'b', 2 => 'a'].
This affects a few operations. Iterating over an array with foreach will return the entries in insertion order. You can do key-wise or value-wise sorting on PHP arrays, the default sorting function sort will drop the original keys and reindex numerically. Serialization and deserialization with numeric keys may have unintended consequences. And some other effects that sometimes are beneficial and sometimes are surprising or annoying (or both). You can read lots of it on PHP's array doc page and the array function pages.
In the context of Doctrine (since it's written in PHP) this means, that a collection where values are the entity objects can be sorted in any manner you want (including id of course), and if you iterate over that collection, you get the entity objects in the order they were added by doctrine (the order of the SQL/DQL query). Doctrine also allows to set the keys to the entities' ids, while still preserving the SQL/DQL query order. This can simplify code since Doctrine's Collection implements PHP's ArrayAccess.
As a counter example, maps can also be unordered or sorted, where the first means when you retrieve the pairs the order can be random (in golang, the starting index used to be random when iterating over maps, don't know if this is still true) or automatically sorted (like SortedMap in Java).
I'm designing a chat app much like Facebook Messenger. My two current root nodes are chats and users. A user has an associated list of chats users/user/chats, and the chats are added by autoID in the chats node chats/a151jl1j6. That node stores information such as a list of the messages, time of the last message, if someone is typing, etc.
What I'm struggling with is where to make the definition of which two users are in the chat. Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea incase I ever wanted group chats.
What seems more logical is to have a chats/chat/members node in which I define userId: true, user2id: true. My issue with this is how to efficiently query it. For example, if the user is going to create a new chat with a user, we want to check if a chat already exists between them. I'm not sure how to do the query of "Find chat where members contains currentUserId and friendUserId" or if this is an efficient denormalized way of doing things.
Any hints?
Although the idea of having ids in the format id1---||---id2 definitely gets the job done, it may not scale if you expect to have large groups and you have to account for id2---||---id1 comparisons which also gets more complicated when you have more people in a conversation. You should go with that if you don't need to worry about large groups.
I'd actually go with using the autoId chats/a151jl1j6 since you get it for free. The recommended way to structure the data is to make the autoId the key in the other nodes with related child objects. So chats/a151jl1j6 would contain the conversation metadata, members/a151jl1j6 would contain the members in that conversation, messages/a151jl1j6 would contain the messages and so on.
"chats":{
"a151jl1j6":{}}
"members":{
"a151jl1j6":{
"user1": true,
"user2": true
}
}
"messages":{
"a151jl1j6":{}}
The part where this gets is little "inefficient" is the querying for conversations that include both user1 and user2. The recommended way is to create an index of conversations for each user and then query the members data.
"user1":{
"chats":{
"a151jl1j6":true
}
}
This is a trade-off when it comes to querying relationships with a flattened data structure. The queries are fast since you are only dealing with a subset of the data, but you end up with a lot of duplicate data that need to be accounted for when you are modifying/deleting i.e. when the user leaves the chat conversation, you have to update multiple structures.
Reference: https://firebase.google.com/docs/database/ios/structure-data#flatten_data_structures
I remember I had similar issue some time ago. The way how I solved it:
user 1 has an unique ID id1
user 2 has an unique ID id2
Instead of adding a new chat by autoId chats/a151jl1j6 the ID of the chat was id1---||---id2 (superoriginal human-readable delimeter)
(which is exactly what you've originally suggested)
Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea in case I ever wanted group chats.
There is a saying: https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it
There might a limitation of how many userIDs can live in the path - you can always hash the value...
i'm migrating data from a rails system, and it would be really convenient to assign the migrated objects IDs like post0000000000001, etc.
i've read here
Creating Meteor-friendly id's in Mongo?
that Meteor creates random 17 character strings from
23456789ABCDEFGHJKLMNPQRSTWXYZabcdefghijkmnopqrstuvwxyz
which looks to be chosen to avoid possibly ambiguous characters (omits 1 and I, etc.)
do the IDs need to be random for some reason? are there security implications to being able to guess a Meteor document's ID?! or it is just an easy way of generating unique IDs?
Mongo seems fine with sequential ids:
http://docs.mongodb.org/manual/core/document/#the-id-field
http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/
so i would guess this would have to be a Meteor constraint if it exists.
The IDs just need to be unique.
Typically there is an element of order: Such as using integers, or timestamps, or something with sequentiality.
This can't work in Meteor since inserts can come from the client, they may be disconnected for a period, or clients clocks may be off/have varying latency. Also its not possible to know the previous _id (in the case of a sequential _id) at the time an _id is written owing to latency compensation (instant inserts).
The consequence of the lack of order in the DDP protocol is the decision to use entirely random ids. That is not to say you can't use your own _ids.
while there is a risk of a collision with this strategy it is minimal on the order of [number of docs in your collection]/[55^17] * 100 % or nearly impossible. In the event this occurs the client will temporarily insert it and cancel it once the server confirms the error with a Mongo Duplicate Key error.
Also when it comes to security with the other answer. It is not too much of an issue if the _id of the user is known. It is not possible to log in without a valid hashed login token or retrieve any information with it. This applies to the user collection only of course. If you have your own collection an easily guessable URL containing an id as a reference without publish method checks on the eligibility to read the data is a risk the high entropy random ids generated by Meteor can mitigate.
As long as they are unique it should be ok to use your own ids.
I am not an expert, but I suppose Mongo needs a unique ID so when it updates the document, it in fact creates a new version of the document of that same ID.
The real question is - I too whish to know - if we can change the ID without screwing Mongo mechanism and reliability, or we need to create a secondary attribute? (It can make a smaller index too I suppose)?
But me too, I can imagine that security wise, it is better if document IDs are difficult to guess, especially user IDs! Otherwise, could it be easy or possible to fake a user, knowing the ID? Anybody, correct me if I am wrong.
I don't think it's possible and desirable to change ID from Mongo.
But you can easily create a autoincrement ID with http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/
function getNextSequence(name) {
var ret = db.counters.findAndModify(
{
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
}
);
return ret.seq;
}
I have created a package that does just that and that is configurable.
https://atmospherejs.com/stivaugoin/fluid-refno
var refNo = generateRefNo({
name: 'invoices', // default: 'counter'
prefix: 'I-', // default: ''
size: 5, // default: 5
filling: '0' // default: '0'
});
console.log(refNo); // output: "I-00001"
you now can use refNo to add in your document on Insert
maybe it will help you