Simple version:
When I register to onSnapshot in Firestore, is there a way to know if I am the client that pushed the update?
Detailed Version:
When a webpage is opened, it registers an onSnapshot callback to a Firestore collection called notes. The collection is then downloaded and stored in a local array var notes = [].
var window.notes = []
let notesRef = refs.db.collection('notes')
notesRef.onSnapshot(querySnapshot => {
querySnapshot.docChanges.forEach((docChange) => {
switch (docChange.type) {
case 'added':
notes.push(docChange.doc.data())
break
}
})
}
We then update a note locally and push it to the database.
notes[0] = 'changed contents'
firestore.notesRef.doc(0).set(notes[0])
This will send the note out to firestore, update it in the remote database, then because we're registered to onSnapshot for the notes collection we'll trigger that event and get a modified change.
let notesRef = refs.db.collection('notes')
notesRef.onSnapshot(querySnapshot => {
querySnapshot.docChanges.forEach((docChange) => {
switch (docChange.type) {
case 'added':
notes.push(docChange.doc.data())
break
case 'modified':
// ********** We end up here!!! ***********
// Probably want to do:
notes[docChange.doc.id] = docChange.doc.data()
break
}
})
}
Is there a way to detect if this same client called the set that triggered the modified docChange?
The point is this: If the user made the change themself then they don't care about the change ad we should ignore it. If a different user made the change then this user should be given a choice of whether or not they want to update their local version.
In my experience, when firestore.notesRef.doc(0).set(notes[0]) is called it takes some time to receive the update from the server. If the local user modifies the note again in this time but hasn't pushed it to the server yet then their changes are lost (overridden by the servers version - the version they last sent to the server). It's essentially a race condition.
So, when I register to onSnapshot in Firestore, is there a way to know if I am the client that pushed the update?
You can see if it was a local or remote change with:
docChange.doc.metadata.hasPendingWrites
If hasPendingWrites is true, then origin of the update was local. The local cache has been updated and the write to the server is still pending.
If hasPendingWrites is false, then the origin of the update was remote.
See how it can be used with onSnapshot in this snippet:
let notesRef = refs.db.collection('notes')
notesRef.onSnapshot(querySnapshot => {
querySnapshot.docChanges.forEach((docChange) => {
switch (docChange.type) {
case 'added':
notes.push(docChange.doc.data())
break
case 'modified':
// ********** New Code ***********
let source = docChange.doc.metadata.hasPendingWrites ? 'Local' : 'Server'
if (source === 'Server') {
notes[docChange.doc.id] = docChange.doc.data()
} else {
// Do nothing, it's a local update so ignore it
}
// ********** New Code End ***********
break
}
})
}
Related
Is there any way to pause firestore listener without removing it?
I have multiple firebase listeners, some are dependent on other, that changes or start other listeners on data change. Lets say my first listener starts a second listener its onSnapshot. First listener started on useEffect. For certain condition I may not want to change the second listener, so I need to discard data change update from first listener.
If condition met (button click), I discard data changes on first listener for a few moments. Currently I'm doing this using a boolean with useRef. My react app is working fine, with dependant listeners like this. I could remove the listener but I do not want to remove and recreate the listener.
I was wondering if there is a pausing mechanism or method available for any listener. I think it will save a tiny read cost if there was such a method because I'm not using that data sent onSnapshot.
Code example:
useEffect(() => {
let firstListener, secondListener;
//console.log("useEffect...");
function ListenerFunc(p) {
secondListener = await firestore
.collection("test")
.doc(p)
.onSnapshot((doc) => {
//console.log("Current data: ", doc.data());
//Need to discard unwanted change here.
//Changing it on button click for a 2 seconds then it changes back to : pauser.current = false.
if (pauser.current) {
console.log("paused for a moment.");
//pauser.current = false;
return;
}
else {
//update.
}
})
}
firstListener = firestore
.collection("test")
.doc("tab")
.onSnapshot((doc) => {
//console.log("Current data: ", doc.data());
var p = doc.data().p; //get variable p
ListenerFunc(p);
});
// cleanup.
}
Unfortunately this is not possible. If you need to stop listening for changes, even temporarily, you have to detach your listener and attach a new one when you want to start listening again, there is no pause mechanism for listeners.
You could open a Feature Request in Google's Issue Tracker if you'd like so that the product team can consider this, but given that this has already been proposed in this GitHub Feature Request for the IOS SDK and it was rejected I don't see this changing anytime soon.
I use a Cloud Function to generate a short unique URL on a record on the 'onWrite' event, and save it. This works well, but when I save a record from my Ember app using EmberFire, I do get a model back as an argument to a callback, but the URL of this model is undefined. Is there a way to return this back to the client? Or do I need to query the record to get the generated URL?
This is how my Cloud Function code looks:
exports.generateUrl = functions.database.ref('/tournaments/{tid}')
.onWrite(event => {
if (event.data.previous.exists()) {
return;
}
if (!event.data.exists()) {
return;
}
const url = shortid.generate();
return event.data.ref.update({ url });
});
Here is my component that saves data through form submission. I'm using an add-on called ember-changeset to handle some validations, but this shouldn't be related to the issue.
export default Ember.Component.extend({
submit(e) {
e.preventDefault();
let snapshot = this.changeset.snapshot();
return this.changeset
.cast(Object.keys(this.get('schema')))
.validate()
.then(() => {
if (this.changeset.get('isValid')) {
return this.changeset
.save()
.then((result) => {
// Here, result.get('url') is undefined.
})
}
})
}
});
If you have a function that writes new data back to a location in the database after a write, you'll have to keep listening to that location on the client in order to get that data back. Don't use a one-time read (once()), use a persistent listener (on()), and in that listener, make sure you're getting the URL or whatever you expect to be generated by the function. Then remove that listener if you don't need it any more.
(Sorry, I don't know Ember or what abstractions it provides around Realtime Database - I'm giving you the plain JavaScript API methods you'd use on a reference.)
Let's say that two users do changes to the same document while offline, but in different sections of the document. If user 2 goes back online after user 1, will the changes made by user 1 be lost?
In my database, each row contains a JS object, and one property of this object is an array. This array is bound to a series of check-boxes on the interface. What I would like is that if two users do changes to those check-boxes, the latest change is kept for each check-box individually, based on the time the when the change was made, not the time when the syncing occurred. Is GroundDB the appropriate tool to achieve this? Is there any mean to add an event handler in which I can add some logic that would be triggered when syncing occurs, and that would take care of the merging ?
The short answer is "yes" none of the ground db versions have conflict resolution since the logic is custom depending on the behaviour of conflict resolution eg. if you want to automate or involve the user.
The old Ground DB simply relied on Meteor's conflict resolution (latest data to the server wins) I'm guessing you can see some issues with that depending on the order of when which client comes online.
Ground db II doesn't have method resume it's more or less just a way to cache data offline. It's observing on an observable source.
I guess you could create a middleware observer for GDB II - one that checks the local data before doing the update and update the client or/and call the server to update the server data. This way you would have a way to handle conflicts.
I think to remember writing some code that supported "deletedAt"/"updatedAt" for some types of conflict handling, but again a conflict handler should be custom for the most part. (opening the door for reusable conflict handlers might be useful)
Especially knowing when data is removed can be tricky if you don't "soft" delete via something like using a "deletedAt" entity.
The "rc" branch is currently grounddb-caching-2016 version "2.0.0-rc.4",
I was thinking about something like:
(mind it's not tested, written directly in SO)
// Create the grounded collection
foo = new Ground.Collection('test');
// Make it observe a source (it's aware of createdAt/updatedAt and
// removedAt entities)
foo.observeSource(bar.find());
bar.find() returns a cursor with a function observe our middleware should do the same. Let's create a createMiddleWare helper for it:
function createMiddleWare(source, middleware) {
const cursor = (typeof (source||{}).observe === 'function') ? source : source.find();
return {
observe: function(observerHandle) {
const sourceObserverHandle = cursor.observe({
added: doc => {
middleware.added.call(observerHandle, doc);
},
updated: (doc, oldDoc) => {
middleware.updated.call(observerHandle, doc, oldDoc);
},
removed: doc => {
middleware.removed.call(observerHandle, doc);
},
});
// Return stop handle
return sourceObserverHandle;
}
};
}
Usage:
foo = new Ground.Collection('test');
foo.observeSource(createMiddleware(bar.find(), {
added: function(doc) {
// just pass it through
this.added(doc);
},
updated: function(doc, oldDoc) {
const fooDoc = foo.findOne(doc._id);
// Example of a simple conflict handler:
if (fooDoc && doc.updatedAt < fooDoc.updatedAt) {
// Seems like the foo doc is newer? lets update the server...
// (we'll just use the regular bar, since thats the meteor
// collection and foo is the grounded data
bar.update(doc._id, fooDoc);
} else {
// pass through
this.updated(doc, oldDoc);
}
},
removed: function(doc) {
// again just pass through for now
this.removed(doc);
}
}));
This Meteor client public method needs to re run when the Meteor.user().profile.propA changes which is does fine, but it also runs when profile.propB changes or added. How can I stop it from re running when any other child property of profile has changed or added but only for profile.propA? Thanks
myListener: () => {
Tracker.autorun(() => {
if (Meteor.userId()) {
const indexes = Meteor.user().profile.propA;
if (!indexes || indexes.length <= 0) return;
dict.set('myStuff', indexes);
console.log('auto has run');
}
});
},
on the mongodb terminal:
db.users.update({'_id':'123abc'}, {$set: {'profile.propB':'B'}})
triggers the autorun. even though the reactive data source is Meteor.user().profile.propA;
Mongo.Collection.findOne allows you to specify which fields are retrieved from the local database using the fields option. Only changes to the fields specified there will trigger the autorun again.
Since Meteor.user() is just a shorthand for Meteor.users.findOne(Meteor.userId()), you can do the following to get updates for propA only:
const indexes = Meteor.users.findOne(Meteor.userId(), {
fields: {
'profile.propA': 1
}
});
Note that indexes will only contain profile.propA and the document's _id. If you need more data from the user document but still want to receive reactive updates separately, you have to fetch that data in a second autorun.
I am seeing a repeatable issue where a user authenticates ("logs in") with a Meteor server, and then a client subscription that depends on userId is updated (and dependent UI templates reactively update) before Meteor.userId() registers the successful login.
For example, in this code snippet, the assert will throw:
var coll = new Meteor.Collection("test");
if (Meteor.isServer) {
Meteor.publish('mineOrPublic', function () {
// Publish public records and those owned by subscribing user
return coll.find({owner: { $in: [ this.userId, null ]}});
});
}
if (Meteor.isClient) {
var sub = Meteor.subscribe('mineOrPublic');
var cursor = coll.find({});
cursor.observe({
added: function (doc) {
if (doc.owner) {
// This should always be true?!
assert(doc.owner === Meteor.userId());
}
}
});
}
Analogous to the added function above, if I write a template helper that checks Meteor.userId(), it will see a value of null, even when it is invoked with a data context of a document with an owner.
There is apparently a race condition between Meteor collection Pub/Sub and the Account userId update mechanisms. It seems to me that Meteor.userId() should always be updated before any subscriptions update based on a change in this.userId in a server publish function, but for some reason the opposite usually seems to be true (that is, the assert in the code above will usually throw).
The reason I care is because I have packages that depend on obtaining a valid Meteor Authentication token (using Accounts._storedLoginToken()) on the client for use in securing HTTP requests for files stored on the Meteor server. And the authentication token isn't correct until Meteor.userId() is. So the flow of events usually goes something like this:
User logs in
Publish function on server reruns based on the change in this.userId.
Client begins receiving new documents corresponding to the change in userId.
UI Template reactively updates to add DOM elements driven by new documents
Some of the DOM elements are <img> tags with src= values that depend on the data context.
HTTP requests are triggered and ultimately fail with 403 (forbidden) errors because the required authentication cookie hasn't been set yet.
Meteor.userId() finally updates on the client, and code reactively runs to set the authentication cookie
Helpers in the template that depend on a session variable set in the cookie update code are rerun, but the DOM doesn't change, because the URLs in the <img> tags don't change.
Because the DOM doesn't change, the tags don't retry their failed attempts to load the images.
Everything settles down, and the user has to manually reload the page to get their images to appear.
I've come up with two possible approaches to work around this issue:
In the template helper that generates the URL for the <img> tag, always append a dummy query string such as: "?time=" + new Date().getTime(). This causes the DOM to change every time the helper is called and fixes the problem, but it screws-up browser caching and if not coordinated will cause some assets to unnecessarily load multiple times, etc.
In every template helper that touches document data add a test of:
if (this.owner && this.owner !== Meteor.userId()) {
// Perhaps Meteor.loggingIn() could be used above?
// Invalid state, output placeholder
} else {
// Valid state, output proper value for template
}
I really hope someone knows of a less kludgy way to work around this. Alternatively, if consensus arises that this is a bug and Meteor's behavior is incorrect in this respect. I will happily file an issue on Github. I mostly really enjoy working with Meteor, but this is the kind of gritty annoyance that grinds in the gears of "it just works".
Thanks for any and all insights.
After trying lots of things, this variation on the example code in the OP seems to consistently solve the race condition, and I find this an acceptable resolution, unlike my initial attempted workarounds.
I still feel that this kind of logic should be unnecessary and welcome other approaches or opinions on whether Meteor's behavior in the OP sample code is correct or erroneous. If consensus emerges in the comments that Meteor's behavior is wrong, I will create an issue on Github for this.
Thanks for any additional feedback or alternative solutions.
var coll = new Meteor.Collection("test");
if (Meteor.isServer) {
Meteor.publish('mineOrPublic', function (clientUserId) {
if (this.userId === clientUserId) {
// Publish public records and those owned by subscribing user
return coll.find({owner: { $in: [ this.userId, null ]}});
} else {
// Don't return user owned docs unless client sub matches
return coll.find({owner: null});
}
});
}
if (Meteor.isClient) {
Deps.autorun(function () {
// Resubscribe anytime userId changes
var sub = Meteor.subscribe('mineOrPublic', Meteor.userId());
});
var cursor = coll.find({});
cursor.observe({
added: function (doc) {
if (doc.owner) {
// This should always be true?!
assert(doc.owner === Meteor.userId());
}
}
});
}
This code works by giving the server publish function the information it needs to recognize when it is running ahead of the client's own login state, thereby breaking the race condition.
I think this is something that Meteor should do automatically: clients should not see documents based on changes to this.userId in a publish function until after the client Meteor.userId() has been updated.
Do others agree?
I tried with this code that works on server too. In association with FileCollection package.
if (Meteor.isServer) {
CurrentUserId = null;
Meteor.publish(null, function() {
CurrentUserId = this.userId;
});
}
....
OrgFiles.allow({
read: function (userId, file) {
if (CurrentUserId !== file.metadata.owner) {
return false;
} else {
return true;
}
}
...