I've got two working bits of code as follows.
This gets an existing subscription:
this.pubSubClient = new PubSub()
sub = this.pubSubClient.subscription(appConfig.pubSub.subscription)
The creates and gets a non-existing subscription:
this.pubSubClient = new PubSub()
this.topic = this.pubSubClient.topic(appConfig.pubSub.topic)
[sub] = await this.topic.createSubscription(appConfig.pubSub.subscription)
This all works great. However, the first bit of code causes problems if the subscription does NOT exist (on one environment) and the second bit of code causes problems if the subscription does exist (on another environment).
So I've tried to do this:
let sub
try {
sub = this.pubSubClient.subscription(appConfig.pubSub.subscription)
console.log('using existing subscription')
} catch (err) {
[sub] = await this.topic.createSubscription(appConfig.pubSub.subscription)
console.log('using created subscription')
}
But the above does not work because the first line of code never triggers an error. It merely fails to ever receive any messages. Is there a command to getOrCreateSubscription that I can use which will get the subscription and create it if necessary?
Best way seems to be to get subscriptions on the topic of interest and if its not listed then it does not exist so create it else just get it
const [subscriptions] = await this.topic.getSubscriptions()
const subs = subscriptions.map(subscription => last(subscription.name.split('/')))
const subExists = subs.includes(appConfig.pubSub.subscription)
let sub
if (subExists) {
sub = this.pubSubClient.subscription(appConfig.pubSub.subscription)
console.log('using existing subscription')
} else {
[sub] = await this.topic.createSubscription(appConfig.pubSub.subscription)
console.log('using created subscription')
}
FYI last is imported from lodash and returns the last item in an array
The right way
const subscription = this.topic.subscription("name");
const [exists] = await subscription.exists();
if (!exists) {
await subscription.create();
}
Related
Is there a way to retrieve the updated value of a document field updated using firestore.FieldValue.increment without asking for the document?
var countersRef = db.collection('system').doc('counters');
await countersRef.update({
nextOrderCode: firebase.firestore.FieldValue.increment(1)
});
// Get the updated nextOrderCode without asking for the document data?
This is not cost related, but for reliability. For example if I want to create a code that increases for each order, there is no guaranty that if >= 2 orders happen at the same time, will have different codes if I read the incremental value right after the doc update resolves, because if >= 2 writes happen before the first read, then at least 2 docs will have the same code even if the nextOrderCode will have proper advance increment.
Update
Possible now, check other answer.
It's not possible. You will have to read the document after the update if you want to know the value.
If you need to control the value of the number to prevent it from being invalid, you will have to use a transaction instead to make sure that the increment will not write an invalid value. FieldValue.increment() would not be a good choice for this case.
We can do it by using Firestore Transactions, like incremental worked before Field.increment feature:
try {
const orderCodesRef = admin.firestore().doc('system/counters/order/codes');
let orderCode = null;
await admin.firestore().runTransaction(async transaction => {
const orderCodesDoc = await transaction.get(orderCodesRef);
if(!orderCodesDoc.exists) {
throw { reason: 'no-order-codes-doc' };
}
let { next } = orderCodesDoc.data();
orderCode = next++;
transaction.update(orderCodesRef, { next });
});
if(orderCode !== null) {
newOrder.code = orderCode;
const orderRef = await admin.firestore().collection('orders').add(newOrder);
return success({ orderId: orderRef.id });
} else {
return fail('no-order-code-result');
}
} catch(error) {
console.error('commitOrder::ERROR', error);
throw errors.CantWriteDatabase({ error });
}
Had the same question and looks like Firestore Python client
doc_ref.update() returns WriteResult that has transform_results attribute with the updated field value
I've this situation with a circular function, having trouble finding a solution.
Have a collection where I have a flag that tells if the data has changed. Also want to log the changes.
export async function landWrite(change, context) {
const newDocument = change.after.exists ? change.after.data() : null
const oldDocument = change.before.data()
const log = {
time: FieldValue.serverTimestamp(),
oldDocument: oldDocument,
newDocument: newDocument
}
const landid = change.after.id
const batch = db.batch()
const updated = newDocument && newDocument.updated === oldDocument.updated
if (!updated) {
const landRef = db.collection('land').doc(landid)
batch.update(landRef, {'updated': true })
}
const logRef = db.collection('land').doc(landid).collection('logs').doc()
batch.set(logRef, log)
return batch.commit()
.then(success => {
return true
})
.catch(error => {
return error
})
}
The problem is that this writes the log twice when the UPDATED flag is false.
But also cannot put the log write in the ELSE statement because the flag can already be UPDATED and a new document update be made so a new log has to be written.
Trigger:
import * as landFunctions from './lands/index'
export const landWrite = functions.firestore
.document('land/{land}')
.onWrite((change, context) => {
return landFunctions.landWrite(change, context)
})
If I understand correctly, the problem here is that the updated flag does not specify which event the update is in response to (as you can't really do this with a boolean). In other words - you may have multiple simultaneous "first-stage" writes to lands, and need a way to disambiguate them.
Here are a few possible options that I would try - from (IMHO) worst to best:
The first option is not very elegant to implement
The first and second options both result in your function being
called twice.
The third option means that your function is only
called once, however you must maintain a separate parallel
document/collection alongside lands.
Option 1
Save some sort of unique identifier in the updated field (e.g. a hash of the stringified JSON event - e.g. hash(JSON.stringify(oldDocument)), or a custom event ID [if you have one]).
Option 2
Try checking the updateMask property of the incoming event, and discard any write events that only affect that property.
Option 3
Store your update status in a different document path/collection (e.g. a landUpdates collection at the same level as your lands collection), and configure your Cloud Function to not trigger on that path. (If you need to, you can always create a second Cloud Function that does trigger on the landUpdates path and add either the same logic or different logic to it.)
Hope this helps!
The main problem here is the inability of differentiating changes that are made by this server function or by a client. Whenever you are in this situation, you should try to explicitly differentiate between them. You can even consider having an extra field like fromServer: true that goes with server's updates and helps the server ignore the related trigger. Having said that, I think I have identified the issue and provided a clear solution below.
This line is misleading:
const updated = newDocument && newDocument.updated === oldDocument.updated
It should be named:
const updateStatusDidNotChange = newDocument && newDocument.updated === oldDocument.updated
I understand that you want the updated flag to be managed by this function, not the client. Let me know if this is not the case.
Therefore, the update field is only changed in this function. Since you want to log only changes made outside of this function, you want to log only when updated did not change.
Here's my attempt at fixing your code in this light:
export async function landWrite(change, context) {
const newDocument = change.after.exists ? change.after.data() : null
const oldDocument = change.before.data()
const updateStatusDidNotChange = newDocument && newDocument.updated === oldDocument.updated
if (!updateStatusDidNotChange) return true; //this was a change made by me, ignore
const batch = db.batch()
if (!oldDocument.updated) {
const landid = change.after.id
const landRef = db.collection('land').doc(landid)
batch.update(landRef, {'updated': true })
}
const log = {
time: FieldValue.serverTimestamp(),
oldDocument: oldDocument,
newDocument: newDocument
}
const logRef = db.collection('land').doc(landid).collection('logs').doc()
batch.set(logRef, log)
return batch.commit()
.then(success => {
return true
})
.catch(error => {
return error
})
}
Edit
I had the exact problem and I had to differentiate changes by the server and the client, and ignore the ones that were from the server. I hope you give my suggestion a try.
I was assuming that it was possible to do something like:
transaction.add(collectionRef,{
uid: userId,
name: name,
fsTimestamp: firebase.firestore.Timestamp.now(),
});
But apparently it is not:
transaction.add is not a function
The above message is displayed inside the chrome console.
I see that we can use the set method of the transaction to add a new document transactionally. see: https://firebase.google.com/docs/firestore/manage-data/transactions
The thing is if I use set instead of add(which is not supported anyways), the id of the document should be created by me manually, firestore won't create it.
see: https://firebase.google.com/docs/firestore/manage-data/add-data
Do you see any downside of this not having an add method that generates the id for you automatically?
For example, is it possible that the id generated by the firestore itself is somehow optimized considering various concerns including performance?
Which library/method do you use to create your document IDs in react-native while using transaction.set?
Thanks
If you want to generate a unique ID for later use in creating a document in a transaction, all you have to do is use CollectionReference.doc() with no parameters to generate a DocumentReference which you can set() later in a transaction.
(What you're proposing in your answer is way more work for the same effect.)
// Create a reference to a document that doesn't exist yet, it has a random id
const newDocRef = db.collection('coll').doc();
// Then, later in a transaction:
transaction.set(newDocRef, { ... });
after some more digging I found in the source code of the firestore itself the below class/method for id generation:
export class AutoId {
static newId(): string {
// Alphanumeric characters
const chars =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
let autoId = '';
for (let i = 0; i < 20; i++) {
autoId += chars.charAt(Math.floor(Math.random() * chars.length));
}
assert(autoId.length === 20, 'Invalid auto ID: ' + autoId);
return autoId;
}
}
see: https://github.com/firebase/firebase-js-sdk/blob/73a586c92afe3f39a844b2be86086fddb6877bb7/packages/firestore/src/util/misc.ts#L36
I extracted the method (except the assert statement) and put it inside a method in my code. Then I used the set method of the transaction as below:
generateFirestoreId(){
const chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
let autoId = '';
for (let i = 0; i < 20; i++) {
autoId += chars.charAt(Math.floor(Math.random() * chars.length));
}
//assert(autoId.length === 20, 'Invalid auto ID: ' + autoId);
return autoId;
}
then,
newDocRef = db.collection("PARENTCOLL").doc(PARENTDOCID).collection('SUBCOLL').doc(this.generateFirestoreId());
transaction.set(newDocRef,{
uid: userId,
name: name,
fsTimestamp: firebase.firestore.Timestamp.now(),
});
Since I am using the same algo for the id generation as the firestore itself I feel better.
Hope this helps/guides someone.
Cheers.
Based on the answer from Doug Stevenson, this is how I got it worked with #angular/fire:
// Create a reference to a document and provide it a random id, e.g. by using uuidv4
const newDocRef = this.db.collection('coll').doc(uuidv4()).ref;
// In the transaction:
transaction.set(newDocRef, { ... });
To complete Stefan's answer. For those using Angularfire, earlier to version 5.2 using CollectionReference.doc() results in an error "CollectionReference.doc() requires its first argument to be of type non-empty string".
This workaround worked for me:
const id = this.afs.createId();
const ref = this.afs.collection(this.collectionRef).doc(id);
transaction.set(ref, { ... });
Credit: https://github.com/angular/angularfire/issues/1974#issuecomment-448449448
I'd like to add an answer solving the id problem. There's no need to generate your own ids. The documentReference is updated after the transaction.set() is called, so in order to access the Firestore's id you need to just do the following:
const docRef = collectionRef.doc();
const result = await transaction.set(docRef, input);
const id = docRef.id;
First of all, firestore transaction object has 4 (get,set,update,delete) methods and doesnt has "add" method. However, the "set" method can be used instead.
import { collection,doc,runTransaction } from "firebase/firestore";
On the other hand documentReference must be created for "set" method.
Steps :
1-) collection method create a collectionReference object.
const collectionRef = collection(FirebaseDb,"[colpath]");
2-) doc method create a documentReference object with unique random id for specified collectionReference.
const documentRef = doc(collectionRef);
3-) add operation can be performed with the transaction set method
try {
await runTransaction(FirebaseDb,async (transaction) => {
await transaction.set(documentRef, {
uid: userId,
name: name,
fsTimestamp: firebase.firestore.Timestamp.now(),
});
})
} catch (e) {
console.error("Error : ", e);
}
I have a website for testing purposes hosted via firebase, storing client information on a realtime database which needs to be accessed later. When I do this via a single html document with a script that accesses my reatime database I am able to find information successfuly, but when I copied and pasted that same logic into a cloud function it did not work. I have tried everything I can think of and now when I run the function it executes twice (I am not sure why). The first execution finishes with a http 204 status (no content found). The second execution returns http 500 internal service error. When I checked the logs on firebase it says the error was because "accounts.getValue() is not a function". I think what is happening is on the first execution the function is unable to locate accounts and it executes again without trying to find the accounts, which might be why it can't run accounts.getValue()
I guess my main question is why is my function unable to locate accounts?
geturl is the function I am having trouble with
The structure of my realtime database is
database name
-accounts
-some data
-more data
-more account data
-ActiveQRs
-some data...
My index.js file for cloud functions is
const functions = require('firebase-functions');
const express = require('express');
const cors = require('cors')({origin: true});
var firebase = require("firebase");
var admin = require("firebase-admin");
require("firebase/auth");
require("firebase/database");
//require("firebase/firestore");
//require("firebase/messaging");
require("firebase/functions");
var serviceAccount = require("./serviceKey.json");
// Initialize the app with a service account, granting admin
//privileges
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://databaseName.firebaseio.com"
});
const displayqr = express();
const geturl = express();
displayqr.get('/displayqr', (request, response) => {
console.log("response sent");
response.send("testio/qrdisplay.html");
});
exports.displayqr = functions.https.onRequest(displayqr);
exports.geturl = functions.https.onCall((email) => {
const mail = email.toString();
var result = "";
result = result + mail;
var accounts =
admin.database().ref("livsuiteform/accounts");
result = (accounts.getValue());
accounts.orderByKey().on("value", function(snapshot) {
snapshot.forEach(function(data) {
if (data.child("Email").val() == mail) {
var firstName = data.child("FirstName").val();
var lastName = data.child("LastName").val();
result = firstname;
result = "if loop entered";
} // end if
// return "name not found";
}); // end for each
}); // end order by
return result;
});
TLDR; follow this tutorial on how to build and deploy callable functions for your mobile app.
There are multiple reasons for why your functions aren't working as you expect.
You are including the client-side version of Firebase (var firebase = require("firebase");). You shouldn't use or even require the client-side version. Instead just use Firebase Admin (docs) to access any data. If you need certain user permissions when accessing the DB from the Admin SDK, here is a good example of how to achieve that (Scroll down to "You can still perform user-authorized changes...").
You have mixed different Admin SDK references. getValue() is part of the Admin SDK for Java. You should use the JavaScript equivalent val(). Also, in your code, accounts is a Reference and not a DataSnapshot.
You aren't returning your Promise's. This can be a source of inconsistency in your function execution later SO Question.
You aren't returning anything from your initial function. If you don't return anything, then nothing will get returned to your app. The solution is the same as 3's solution: return your Promise.
You shouldn't use on in Firebase Functions. You should use once. The difference is that on doesn't return a Promise while once does. It returns a function that is used to detach the listener.
I know this is a lot of bullet points and pointing out problems in your code, but I just didn't want give a shallow answer which resulted in you asking another question and waiting another ~2 hours (at the time of writing) for an answer.
I hope this helps!
Code
exports.geturl = functions.https.onCall((email) => {
const mail = email.toString();
var result = "";
result = result + mail;
var accounts = admin.database().ref("livsuiteform/accounts");
return accounts.orderByKey().once("value")
.then(function (snapshot) {
snapshot.forEach(function (data) {
if (data.child("Email").val() == mail) {
var firstName = data.child("FirstName").val();
var lastName = data.child("LastName").val();
result = firstName;
result = "if loop entered";
} // end if
// return "name not found";
}); // end for each
return result;
}); // end order by
});
I'm trying to get an item from my DynamoDB database. The way my code is presently written, I fail to retrieve any data from DynamoDB. I must be doing something wrong, because as far as I can tell from my test, my callback is not being called.
I spent all day on this yesterday and have been tinkering with it unsuccessfully since I woke up this morning.
If anyone can provide insight into what I'm doing wrong here, I would be very grateful. Thanks to everyone in advance!
Final note: The timeout on the Lambda function itself is set to 5 minutes. So I don't think the Lambda function is timing out before the db query can return. When I run the function, it exits after only a moment.
const AWS = require('aws-sdk');
const dynamodb = new AWS.DynamoDB();
var response = null;
var test = false;
function getFromDB(callback) {
const params = {
TableName: process.env['DB_TABLE_NAME'] // evaluates to 'test-table',
Key: {
"id": {
S: postId // evaluates to a big string, pulling it in from an SNS message. Verified it with console.log(). It stores the expected value.
}
}
};
dynamodb.getItem(params, function(err, data) {
if (err) callback(data, true); // an error occurred
else callback(data, true); // successful response
});
}
getFromDB((data, isCalled) => {
response = data;
test = isCalled;
});
console.log(data); // evaluates to null
console.log(test); // evaluates to false
I Had faced similar issue.
I removed async in the statement below to resolve :
exports.handler = async (event,context)
I think what's going on is Lambda calls the function, but it's not going to wait for the call back, so it thinks it is done and exits.
I think I had a similar problem and resolved it by using Bluebird and async/await.
I can provide a snippet from my code if you need it
Have you loaded the SDK? I can't see it in your code snippet
// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'REGION'});
EDIT: Included region