When retrieving data from Firestore one has the option of forcing retrieval from the server. The default option is cache and server, as determined by Firestore.
I have a certain usage where a command and control node is issuing real-time commands to remote nodes backed by Firestore. This requires the updates to be done on the server (or fail) so that the C&C node has certainty on the execution (or failure) in real-time. What I would like to do is to disable use of cache with these updates. I have not found a way to do that. Is this possible with current capabilities of Firestore?
Note that it is not desirable to disable Firestore caching at a global level as the cache is beneficial in other situations.
----EDIT-----
Based on the responses I have created this update method that attempts to force updating the server using a transaction.
A couple of notes:
This is dart code.
Utils.xyz is an internal library and in this case it is being used to log.
I have reduced the network speed for the test to simulate a bad network connection.
The timeout is set to 5 seconds.
Here is the output of my log:
I/flutter (22601): [2021-06-06 22:35:30] [LogLevel.DEBUG] [FirestoreModel] [update] [We are here!]
I/flutter (22601): [2021-06-06 22:35:47] [LogLevel.DEBUG] [FirestoreModel] [update] [We are here!]
I/flutter (22601): [2021-06-06 22:36:02] [LogLevel.DEBUG] [FirestoreModel] [update] [We are here!]
I/flutter (22601): [2021-06-06 22:37:18] [LogLevel.DEBUG] [FirestoreModel] [update] [We are here!]
I/flutter (22601): [2021-06-06 22:37:20] [LogLevel.INFO] [FirestoreModel] [update] [Transaction successful in 110929ms.]
Firebase completely ignores the timeout of 5 seconds; tries to update 4 times each time ~15 seconds apart and is finally successful after 110 seconds. I am after a real-time response within seconds (5 sec) or failure.
Future<void> update(
Map<String, dynamic> data, {
WriteBatch batch,
Transaction transaction,
bool forceServer = false,
}) async {
// If updating there must be an id.
assert(this.id != null);
// Only one of batch or transaction can be non-null.
assert(batch == null || transaction == null);
// When forcing to update on server no transaction or batch is allowed.
assert(!forceServer || (batch == null && transaction == null));
try {
if (forceServer) {
DateTime start = DateTime.now();
await FirebaseFirestore.instance.runTransaction(
(transaction) async {
await update(data, transaction: transaction);
Utils.logDebug('We are here!');
},
timeout: Duration(seconds: 5),
);
Utils.logDebug('Transaction successful in ${DateTime.now().difference(start).inMilliseconds}ms.');
} else {
DocumentReference ref =
FirebaseFirestore.instance.collection(collection).doc(this.id);
if (batch != null)
batch.update(ref, data);
else if (transaction != null)
transaction.update(ref, data);
else
await ref.update(data);
}
} catch (e, s) {
Utils.logException('Error updating document $id in $collection.', e, s);
// Propagate the error.
rethrow;
}
}
This requires the updates to be done on the server (or fail)
For that you could use Transactions and batched writes.
Transactions will fail when the client is offline.
Check out doc
To get live data from the server once, you would use:
firebase.firestore()
.doc("somecollection/docId")
.get({ source: "server" })
.then((snapshot) => {
// if here, snapshot.data() is from the server
// TODO: do something with data
})
.catch((err) => {
// if here, get() encountered an error (insufficient permissions, server not available, etc)
// TODO: handle the error
});
To get realtime live data from only the server (ignoring the cache), you would use:
const unsubscribe = firebase.firestore()
.doc("somecollection/docId")
.onSnapshot({ includeMetadataChanges: true }, {
next(snapshot) {
// ignore cache data
if (snapshot.metadata.fromCache) return;
// if here, snapshot.data() is from the server
// TODO: do something with data
},
error(err) {
// if here, onSnapshot() encountered an error (insufficient permissions, etc)
// TODO: handle the error
}
});
To write to the server, you would use the normal write operations - delete(), set(), and update(); as they all return Promises that will not resolve while the client is offline. If they have resolved, the data stored on the server has been updated.
To test if you are online or not, you can try and pull a non existant document down from the server like so:
/**
* Attempts to fetch the non-existant document `/.info/connected` to determine
* if a connection to the server is available.
* #return {Promise<boolean>} promise that resolves to a boolean indicating
* whether a server connection is available
*/
function isCurrentlyOnline() {
// unlike RTDB, this data doesn't exist and has no function
// must be made readable in security rules
return firebase.firestore()
.doc(".info/connected")
.get({ source: "server" })
.then(
() => {
// read data successfully, we must be online
return true;
}, (err) => {
// failed to read data, if code is unavailable, we are offline
// for any other error, rethrow it
if (err.code === "unavailable")
return false;
throw err;
}
);
}
/**
* A function that attaches a listener to when a connection to Firestore has
* been established or when is disconnected.
*
* This function listens to the non-existant `/.info/connected` document and
* uses it's `fromCache` metadata to **estimate** whether a connection to
* Firestore is currently available.
* **Note:** This callback will only be invoked after the first successful
* connection to Firestore
*
* #param {((error: unknown | null, isOnline: boolean) => unknown)} callback the
* callback to invoke when the isOnline state changes
* #return {(() => void)} a function that unsubscribes this listener when
* invoked
*/
function onOnline(callback) {
let hasConnected = false;
// unlike RTDB, this data doesn't exist and has no function
// must be made readable in security rules
return firebase.firestore()
.doc(".info/connected")
.onSnapshot(
{ includeMetadataChanges: "server" },
{
next(snapshot) {
const { fromCache } = snapshot.metadata;
if (!hasConnected) {
if (fromCache) return; // ignore this event
hasConnected = true;
}
callback(null, !fromCache);
},
error(err) {
callback(err);
}
}
);
}
Related
How do i trigger the sending of an email when a document data is modified ?
Trigger Email only composes and sends an email based on the contents of a document written to a Cloud Firestore collection but not modified
I can’t figure this one out…
From checking the code we can see that the extension already processes all writes to the document:
export const processQueue = functions.handler.firestore.document.onWrite(
...
From looking a bit further it seems that the action the extension takes upon a write depends on the value of the delivery.state field in the document.
async function processWrite(change) {
...
const payload = change.after.data() as QueuePayload;
...
switch (payload.delivery.state) {
case "SUCCESS":
case "ERROR":
return null;
case "PROCESSING":
if (payload.delivery.leaseExpireTime.toMillis() < Date.now()) {
// Wrapping in transaction to allow for automatic retries (#48)
return admin.firestore().runTransaction((transaction) => {
transaction.update(change.after.ref, {
"delivery.state": "ERROR",
error: "Message processing lease expired.",
});
return Promise.resolve();
});
}
return null;
case "PENDING":
case "RETRY":
// Wrapping in transaction to allow for automatic retries (#48)
await admin.firestore().runTransaction((transaction) => {
transaction.update(change.after.ref, {
"delivery.state": "PROCESSING",
"delivery.leaseExpireTime": admin.firestore.Timestamp.fromMillis(
Date.now() + 60000
),
});
return Promise.resolve();
});
return deliver(payload, change.after.ref);
}
}
My guess is that if you clear that field, the extension will pick up on that change and try to mail the document again.
I've followed the Firestore documentation with relation to transactions, and I think I have it all sorted correctly, but in testing I am noticing issues with my documents not getting updated properly sometimes. It is possible that multiple versions of the document could be submitted to the function in a very short interval, but I am only interested in only ever keeping the most recent version.
My general logic is this:
New/Updated document is sent to cloud function
Check if document already exists in Firestore, and if not, add it.
If it does exist, check that it is "newer" than the instance in firestore, if it is, update it.
Otherwise, don't do anything.
Here is the code from my function that attempts to accomplish this...I would love some feedback if this is correct/best way to do this:
const ocsFlight = req.body;
const procFlight = processOcsFlightEvent(ocsFlight);
try {
const ocsFlightRef = db.collection(collection).doc(procFlight.fltId);
const originalFlight = await ocsFlightRef.get();
if (!originalFlight.exists) {
const response = await ocsFlightRef.set(procFlight);
console.log("Record Added: ", JSON.stringify(procFlight));
res.status(201).json(response); // 201 - Created
return;
}
await db.runTransaction(async (t) => {
const doc = await t.get(ocsFlightRef);
const flightDoc = doc.data();
if (flightDoc.recordModified <= procFlight.recordModified) {
t.update(ocsFlightRef, procFlight);
console.log("Record Updated: ", JSON.stringify(procFlight));
res.status(200).json("Record Updated");
return;
}
console.log("Record isn't newer, nothing changed.");
console.log("Record:", JSON.stringify("Same Flight:", JSON.stringify(procFlight)));
res.status(200).json("Record isn't newer, nothing done.");
return;
});
} catch (error) {
console.log("Error:", JSON.stringify(error));
res.status(500).json(error.message);
}
The Bugs
First, you are trusting the value of req.body to be of the correct shape. If you don't already have type assertions that mirror your security rules for /collection/someFlightId in processOcsFlightEvent, you should add them. This is important because any database operations from the Admin SDKs will bypass your security rules.
The next bug is sending a response to your function inside the transaction. Once you send a response back the client, your function is marked inactive - resources are severely throttled and any network requests may not complete or crash. As a transaction may be retried a handful of times if a database collision is detected, you should make sure to only respond to the client once the transaction has properly completed.
You use set to write the new flight to Firestore, this can lead to trouble when working with transactions as a set operation will cancel all pending transactions at that location. If two function instances are fighting over the same flight ID, this will lead to the problem where the wrong data can be written to the database.
In your current code, you return the result of the ocsFlightRef.set() operation to the client as the body of the HTTP 201 Created response. As the result of the DocumentReference#set() is a WriteResult object, you'll need to properly serialize it if you want to return it to the client and even then, I don't think it will be useful as you don't seem to use it for the other response types. Instead, a HTTP 201 Created response normally includes where the resource was written to as the Location header with no body, but here we'll pass the path in the body. If you start using multiple database instances, including the relevant database may also be useful.
Fixing
The correct way to achieve the desired result would be to do the entire read->check->write process inside of a transaction and only once the transaction has completed, then respond to the client.
So we can send the appropriate response to the client, we can use the return value of the transaction to pass data out of it. We'll pass the type of the change we made ("created" | "updated" | "aborted") and the recordModified value of what was stored in the database. We'll return these along with the resource's path and an appropriate message.
In the case of an error, we'll return a message to show the user as message and the error's Firebase error code (if available) or general message as the error property.
// if not using express to wrangle requests, assert the correct method
if (req.method !== "POST") {
console.log(`Denied ${req.method} request`);
res.status(405) // 405 - Method Not Allowed
.set("Allow", "POST")
.end();
return;
}
const ocsFlight = req.body;
try {
// process AND type check `ocsFlight`
const procFlight = processOcsFlightEvent(ocsFlight);
const ocsFlightRef = db.collection(collection).doc(procFlight.fltId);
const { changeType, recordModified } = await db.runTransaction(async (t) => {
const flightDoc = await t.get(ocsFlightRef);
if (!flightDoc.exists) {
t.set(ocsFlightRef, procFlight);
return {
changeType: "created",
recordModified: procFlight.recordModified
};
}
// only parse the field we need rather than everything
const storedRecordModified = flightDoc.get('recordModified');
if (storedRecordModified <= procFlight.recordModified) {
t.update(ocsFlightRef, procFlight);
return {
changeType: "updated",
recordModified: procFlight.recordModified
};
}
return {
changeType: "aborted",
recordModified: storedRecordModified
};
});
switch (changeType) {
case "updated":
console.log("Record updated: ", JSON.stringify(procFlight));
res.status(200).json({ // 200 - OK
path: ocsFlightRef.path,
message: "Updated",
recordModified,
changeType
});
return;
case "created":
console.log("Record added: ", JSON.stringify(procFlight));
res.status(201).json({ // 201 - Created
path: ocsFlightRef.path,
message: "Created",
recordModified,
changeType
});
return;
case "aborted":
console.log("Outdated record discarded: ", JSON.stringify(procFlight));
res.status(200).json({ // 200 - OK
path: ocsFlightRef.path,
message: "Record isn't newer, nothing done.",
recordModified,
changeType
});
return;
default:
throw new Error("Unexpected value for 'changeType': " + changeType);
}
} catch (error) {
console.log("Error:", JSON.stringify(error));
res.status(500) // 500 - Internal Server Error
.json({
message: "Something went wrong",
// if available, prefer a Firebase error code
error: error.code || error.message
});
}
References
Cloud Firestore Transactions
Cloud Firestore Node SDK Reference
HTTP Event Cloud Functions
I'm a little bit lost on how firebase functions work with authentication,
Suppose I have a function that pulls 100 documents and sets the cache header for 24 hours.
res.set('Cache-Control', 'public, max-age=0, s-maxage=86400' // 24 * 60 * 60
By default, does that apply to all users or is it cached per user? There's some instances where the 100 documents are unique to the user - while other functions where the 100 documents are available to any user that's authenticated.
I see in the docs that you can set a __session which implies it's for individual users data, however there isn't much documentation for how to set that (or where). Is it set by default?
My goal is to have a function that requires the user be authenticated, then return 100 documents from a non-user specific collection - aka not have to read 100 documents per user. However, I don't think thats feasible because it would need to check if each user is authorized (not cacheable). So is there a way to just make a publicly available cache?
Any light that can be shared on this is greatly appreciated!
The Cache-Control header is used to instruct a user's browser and any CDN edge server on how to cache the request.
For requests requiring authentication, making use of the CDN is not really possible for this as you should be using Cache-Control: private for these responses (the default for Cloud Functions).
While you could check that your users are authenticated and then redirect them to a publically cached resource (like https://example.com/api/docs?sig=<somesignature>), this URL would still be accessible if someone got hold of that URL/cached data.
Arguably the best approach would be to store your "cached" responses in a single Cloud Firestore document (if it is less than 1MB in size and is JSON-compatible) or store it in Cloud Storage.
The code included below is an example of how you could do this with a Cloud Firestore cache. I've used posts where the authenticated user is the author as an example, but for this specific use case, you would be better off using the Firebase SDK to make such a query (realtime updates, finer control, query API). A similar approach could be applied for "all user" resources.
If attempting to cache HTML or some other not JSON friendly format, I would recommend changing the caching layer to Cloud Storage. Instead of storing the post's data in the cache entry, store the path and bucket to the cached file in storage (like below). Then if it hasn't expired, get a stream of that file from storage and pipe it through to the client.
{
data: {
fullPath: `/_serverCache/apiCache/${uid}/posts.html`,
bucket: "myBucket"
},
/* ... */
}
Common Example Code
import functions from "firebase-functions";
import { HttpsError } from "firebase-functions/lib/providers/https";
import admin from "firebase-admin";
import hash from "object-hash";
admin.initializeApp();
interface AttachmentData {
/** May contain a URL to the resource */
url?: string;
/** May contain Base64 encoded data of resource */
data?: string;
/** Type of this resource */
type: "image" | "video" | "social" | "web";
}
interface PostData {
author: string;
title: string;
content: string;
attachments: Record<string, AttachmentData>;
postId: string;
}
interface CacheEntry<T = admin.firestore.DocumentData> {
/** Time data was cached, as a Cloud Firestore Timestamp object */
cachedAt: admin.firestore.Timestamp;
/** Time data was cached, as a Cloud Firestore Timestamp object */
expiresAt: admin.firestore.Timestamp;
/** The ETag signature of the cached resource */
eTag: string;
/** The cached resource */
data: T;
}
/**
* Returns posts authored by this user as an array, from Firestore
*/
async function getLivePostsForAuthor(uid: string) {
// fetch the data
const posts = await admin.firestore()
.collection('posts')
.where('author', '==', uid)
.limit(100)
.get();
// flatten the results into an array, including the post's document ID in the data
const results: PostData[] = [];
posts.forEach((postDoc) => {
results.push({ postId: postDoc.id, ...postDoc.data() } as PostData);
});
return results;
}
/**
* Returns posts authored by this user as an array, caching the result from Firestore
*/
async function getCachedPostsForAuthor(uid: string) {
// Get the reference to the data's location
const cachedPostsRef = admin.firestore()
.doc(`_server/apiCache/${uid}/posts`) as admin.firestore.DocumentReference<CacheEntry<PostData[]>>;
// Get the cache entry's data
const cachedPostsSnapshot = await cachedPostsRef.get();
if (cachedPostsSnapshot.exists) {
// get the expiresAt property on it's own
// this allows us to skip processing the entire document until needed
const expiresAt = cachedPostsSnapshot.get("expiresAt") as CacheEntry["expiresAt"] | undefined;
if (expiresAt !== undefined && expiresAt.toMillis() > Date.now() - 60000) {
// return the entire cache entry as-is
return cachedPostsSnapshot.data()!;
}
}
// if here, the cache entry doesn't exist or has expired
// get the live results from Firestore
const results = await getLivePostsForAuthor(uid);
// etag, cachedAt and expiresAt are used for the HTTP cache-related headers
// only expiresAt is used when determining expiry
const cacheEntry: CacheEntry<PostData[]> = {
data: results,
eTag: hash(results),
cachedAt: admin.firestore.Timestamp.now(),
// set expiry as 1 day from now
expiresAt: admin.firestore.Timestamp.fromMillis(Date.now() + 86400000),
};
// save the cached data and it's metadata for future calls
await cachedPostsRef.set(cacheEntry);
// return the cached data
return cacheEntry;
}
HTTPS Request Function
This is the request type you would use for serving Cloud Functions behind Firebase Hosting. Unfortunately the implementation details aren't as straightforward as using a Callable Function (see below) but is provided as an official project sample. You will need to insert validateFirebaseIdToken() from that example for this code to work.
import express from "express";
import cookieParserLib from "cookie-parser";
import corsLib from "cors";
interface AuthenticatedRequest extends express.Request {
user: admin.auth.DecodedIdToken
}
const cookieParser = cookieParserLib();
const cors = corsLib({origin: true});
const app = express();
// insert from https://github.com/firebase/functions-samples/blob/2531d6d1bd6b16927acbe3ec54d40369ce7488a6/authorized-https-endpoint/functions/index.js#L26-L69
const validateFirebaseIdToken = /* ... */
app.use(cors);
app.use(cookieParser);
app.use(validateFirebaseIdToken);
app.get('/', async (req, res) => {
// if here, user has already been validated, decoded and attached as req.user
const user = (req as AuthenticatedRequest).user;
try {
const cacheEntry = await getCachedPostsForAuthor(user.uid);
// set caching headers
res
.header("Cache-Control", "private")
.header("ETag", cacheEntry.eTag)
.header("Expires", cacheEntry.expiresAt.toDate().toUTCString());
if (req.header("If-None-Match") === cacheEntry.eTag) {
// cached data is the same, just return empty 304 response
res.status(304).send();
} else {
// send the data back to the client as JSON
res.json(cacheEntry.data);
}
} catch (err) {
if (err instanceof HttpsError) {
throw err;
} else {
throw new HttpsError("unknown", err && err.message, err);
}
}
});
export const getMyPosts = functions.https.onRequest(app);
Callable HTTPS Function
If you are making use of the client SDKs, you can also request the cached data using Callable Functions.
This allows you to export the function like this:
export const getMyPosts = functions.https.onCall(async (data, context) => {
if (!context.auth) {
throw new functions.https.HttpsError(
'failed-precondition',
'The function must be called while authenticated.'
);
}
try {
const cacheEntry = await getCachedPostsForAuthor(context.auth.uid);
return cacheEntry.data;
} catch (err) {
if (err instanceof HttpsError) {
throw err;
} else {
throw new HttpsError("unknown", err && err.message, err);
}
}
});
and call it from the client using:
const getMyPosts = firebase.functions().httpsCallable('getMyPosts');
getMyPosts()
.then((postsArray) => {
// do something
})
.catch((error) => {
// handle errors
})
Suppose I have 2 collections "PlanSubscriptions" and "ClientActivations". I am serially doing a insert on both the collections.
Later one depends on previous one, if any of the transaction fails then the entire operation must rollback.
How can I achieve that in Meteor 1.4?
Since MongoDB doesn't support atomicity, you will have to manage it with Method Chaining.
You can write a method, say, transaction where you will call PlanSubscriptions.insert(data, callback). Then in the callback function you will call ClientActivations.insert(data, callback1) if the first insertion is success and in callback1 return truthy if second insertion is succes, otherwise falsy. If the first insertion returns error you don't need to do anything, but if the second insertion returns error then remove the id got from the insertion in first collection.
I can suggest following structure:
'transaction'(){
PlanSubscriptions.insert(data, (error, result)=>{
if(result){
// result contains the _id
let id_plan = result;
ClientActivations.insert(data, (error, result)=>{
if(result){
// result contains the _id
return true;
}
else if(error){
PlanSubscriptions.remove(id_plan);
return false;
}
})
}
else if(error){
return false;
}
})
}
There is no way to do that in Meteor, since mongodb is not an ACID-compliant database. It has a single-document update atomicity, but not a multiple-document one, which is your case with the two collections.
From the mongo documentation:
When a single write operation modifies multiple documents, the modification of each document is atomic, but the operation as a whole is not atomic and other operations may interleave.
A way to isolate the visibility of your multi-document updates is available, but it's probably not what you need.
Using the $isolated operator, a write operation that affects multiple documents can prevent other processes from interleaving once the write operation modifies the first document. This ensures that no client sees the changes until the write operation completes or errors out.
An isolated write operation does not provide “all-or-nothing” atomicity. That is, an error during the write operation does not roll back all its changes that preceded the error.
However, there are a couple of libraries which try to tackle the problem at the app-level. I recommend taking a look at fawn
In your case, where you have exactly two dependent collections, it's possible to take advantage of the two phase commits technique. Read more about it here: two-phase-commits
Well I figured it out myself.
I added a package babrahams:transactions
At server side Meteor Method call, I called tx Object that is globally exposed by the package. The overall Server Side Meteor.method({}) looks like below.
import { Meteor } from 'meteor/meteor';
import {PlanSubscriptions} from '/imports/api/plansubscriptions/plansubscriptions.js';
import {ClientActivations} from '/imports/api/clientactivation/clientactivations.js';
Meteor.methods({
'createClientSubscription' (subscriptionData, clientActivationData) {
var txid;
try {
txid = tx.start("Adding Subscription to our database");
PlanSubscriptions.insert(subscriptionData, {tx: true})
ClientActivations.insert(activation, {tx: true});
tx.commit();
return true;
} catch(e){
tx.undo(txid);
}
return false;
}
});
With every insert I had added {tx : true}, this concluded it to be a apart of transaction.
Server Console Output:
I20170523-18:43:23.544(5.5)? Started "Adding Subscription to our database" with
transaction_id: vdJQvFgtyZuWcinyF
I20170523-18:43:23.547(5.5)? Pushed insert command to stack: vdJQvFgtyZuWcinyF
I20170523-18:43:23.549(5.5)? Pushed insert command to stack: vdJQvFgtyZuWcinyF
I20170523-18:43:23.551(5.5)? Beginning commit with transaction_id: vdJQvFgtyZuWcinyF
I20170523-18:43:23.655(5.5)? Executed insert
I20170523-18:43:23.666(5.5)? Executed insert
I20170523-18:43:23.698(5.5)? Commit reset transaction manager to clean state
For more Information you can goto link : https://github.com/JackAdams/meteor-transactions
NOTE: I am using Meteor 1.4.4.2
Just sharing this link for future readers:
https://forums.meteor.com/t/solved-transactions-with-mongodb-meteor-methods/48677
import { MongoInternals } from 'meteor/mongo';
// utility async function to wrap async raw mongo operations with a transaction
const runTransactionAsync = async asyncRawMongoOperations => {
// setup a transaction
const { client } = MongoInternals.defaultRemoteCollectionDriver().mongo;
const session = await client.startSession();
await session.startTransaction();
try {
// running the async operations
let result = await asyncRawMongoOperations(session);
await session.commitTransaction();
// transaction committed - return value to the client
return result;
} catch (err) {
await session.abortTransaction();
console.error(err.message);
// transaction aborted - report error to the client
throw new Meteor.Error('Database Transaction Failed', err.message);
} finally {
session.endSession();
}
};
import { runTransactionAsync } from '/imports/utils'; // or where you defined it
Meteor.methods({
async doSomething(arg) {
// remember to check method input first
// define the operations we want to run in transaction
const asyncRawMongoOperations = async session => {
// it's critical to receive the session parameter here
// and pass it to every raw operation as shown below
const item = await collection1.rawCollection().findOne(arg, { session: session });
const response = await collection2.rawCollection().insertOne(item, { session: session });
// if Mongo or you throw an error here runTransactionAsync(..) will catch it
// and wrap it with a Meteor.Error(..) so it will arrive to the client safely
return 'whatever you want'; // will be the result in the client
};
let result = await runTransactionAsync(asyncRawMongoOperations);
return result;
}
});
When I use transaction() to update a location, data at that location is returning null even though the location having some data.
I tried transaction() after reading data at the same location that time it is giving all data at that location.
How can I use transaction() if the case is like the above?
Transactions work in the manner of Amazon's SimpleDB or a sharded cluster of databases. That is to say, they are "eventually consistent" rather than guaranteed consistent.
So when you are using transactions, the processing function may get called more than once with a local value (in some cases null if it's never been retrieved) and then again with the synced value (whatever is on the server).
Example:
pathRef.transaction(function(curValue) {
// this part is eventually consistent and may be called several times
}, function(error, committed, ss) {
// this part is guaranteed consistent and will match the final value set
});
This is really the mindset with which you must approach transaction anyways. You should always expect multiple calls, since the first transaction may collide with another change and be rejected. You can't use a transaction's processing method to fetch the server value (although you could read it out of the success callback).
Preventing the locally triggered event
When the transaction happens, a local event is triggered before it reaches the server for latency compensation. If the transaction fails, then the local event will be reverted (a change or remove event is triggered).
You can use the applyLocally property on transactions to override this behavior, which makes the local results slower but ensures that only the server value is triggered locally.
pathRef.transaction(function(curValue) {
// this is still called multiple times
}, function(error, committed, ss) {
// this part is guaranteed consistent and will match the final value set
},
// by providing a third argument of `true`, no local event
// is generated with the locally cached value.
true);
You need to follow this pattern:
var pinRef = firebase.database().ref('vm-pin-generator');
pinRef.transaction(function(oldPin) {
// Check if the result is NOT NULL:
if (oldPin != null) {
return localPinIncrementor(oldPin);
} else {
// Return a value that is totally different
// from what is saved on the server at this address:
return 0;
}
}, function(error, committed, snapshot) {
if (error) {
console.log("error in transaction");
} else if (!committed) {
console.log("transaction not committed");
} else {
console.log("Transaction Committed");
}
}, true);
Firebase usually returns a null value while retrieving a key for the first time but while saving it checks if the new value is similar to older value or not. If not, firebase will run the whole process again, and this time the correct value is returned by the server.
Adding a null check and returning a totally unexpected value (0 in this case) will make firebase run the cycle again.
Simply showing an example implementation to elaborate on #Kato accepted answer above with a custom upsert function:
/**
* Transactional insert or update record
* #param {String} type - object type (table or index) to build lookup path
* #param {String} id - object ID that will be concat with path for lookup
* #param {Object} data - new object (or partial with just edited fields)
* #return {Object} new version of object
*/
const upsert = (type, id, data) => {
return new Promise((resolve, reject) => {
if (!type) {
log.error('Missing db object type')
reject(new TypeError('Missing db object type'))
}
if (!id) {
log.error('Missing db object id')
reject(new TypeError('Missing db object id'))
}
if (!data) {
log.error('Missing db data')
reject(new TypeError('Missing db data'))
}
// build path to resource
const path = `${type}/${id}`
log.debug(`Upserting record '${path}' to database`)
try {
const ref = service.ref(path)
ref.transaction(record => {
if (record === null) {
log.debug(`Creating new record`) // TODO: change to debug
return data
} else if (record) {
log.debug(`Updating existing record`) // TODO: change to debug
const updatedRecord = Object.assign({}, record, data)
return updatedRecord
} else {
return record
}
}, (error, committed, snapshot) => {
if (error) {
log.error(`Error upserting record in database`)
log.error(error.message)
reject(error)
} else if (committed) {
log.debug(`Saved update`)
} else {
log.debug(`Record unchanged`)
}
if (snapshot) {
resolve(snapshot.val())
} else {
log.info(`No snapshot found in transaction so returning original data object`)
resolve(data)
}
})
} catch (error) {
log.error(error)
reject(error)
}
})
}