How to avoid double Cloud Firestore write when uploading Storage file using the document ID and passing the DownloadURL - firebase

I am working on a Flutter app connected to Firebase Cloud Firestore and Storage, and I want to create documents in Cloud Firestore that contain the downloadURL of files in Storage. However, I also need the filename to be the unique ID of the same Firestore document. Right now I upload the file after I create the document, but it would cost me an extra write to update the Firestore document afterwards.
I want to prevent this. It might work if I could access the id of the newly created document within the add() function, so that I can pass it to the storageTask, which would return the downloadURL directly inside the add() function. Another option would be if I could create a document, get its ID, then do the upload task, and write all the data afterwards, but I''m not sure if this would count as one write or two, nor do I know how to create a document without setting any data.
What I have now is roughly like this:
CollectionReference activities = FirebaseFirestore.instance.collection('activities');
activities.add({
'postDate': DateTime.now(),
}).then((value) => _startUpload(id: value.id));
Where _startUpload is a function that uploads a file to Storage, and could potentially return a downloadURL. I want to be able to access that URL in the add function like this:
CollectionReference activities = FirebaseFirestore.instance.collection('activities');
activities.add({
'postDate': DateTime.now(),
'downloadURL': _startUpload(id: SomehowGetThisDocuments.id)
});

You can do something like described in the last paragraph of this section in the documentation:
newActivityRef = activities.document();
newActivityRef.set({
'postDate': DateTime.now(),
'downloadURL': _startUpload(id: newActivityRef.documentID)
});
This should count as one write as .document().set() is equivalent to .add() according to the documentation.

Related

Is there a way to determine once a cloud function has finished running - either through looping getDownloadURL or messaging?

I am still getting the hang of Firebase and Cloud functions, but here is what I'm trying to figure out.
The current setup
My app has a cloud function that will take a PDF that has been uploaded into a storage bucket and convert it into PNG. It doesn't destroy the original PDF, so I am left with both files.
The URL for the newly created PNG is then attached to a property on one of our documents in Firestore.
What I am trying to accomplish
I want to be able to upload a new PDF to use as a replacement image. I think I am running into a race condition where the cloud function hasn't finished executing by the time I am trying to call updateDoc() with the new PNG.
On the client side, I have the storageRef returned from the upload method:
uploadFunction(...).then((snapshot) => {
return snapshot.ref
}
I'm saving the result of this function to a variable, and I am trying to pass that into the update method that will adjust the property on my document in Firestore:
const storageRef = await functionThatUploadsPDF(file);
updateDocumentInFirestore(storageRef);
Within updateDocumentInFirestore, I'm trying to navigate to the new reference that should exist once the cloud function has finished, get a download URL, and update that property on my document:
const newImageRef = ref(storageRef.parent, "generatedImage.png");
const newDownloadURL = getDownloadURL(newImageRef).then((url) => {
updateDoc(documentRef, backgroundImage: url);
});
However, I am getting the following error - I believe due to the cloud function having not finished yet:
Firebase Storage: Object 'storage-bucket/generatedImage.png' does not exist. (storage/object-not-found)
My thoughts on potential solutions
I could try to poll the storage for the existence of generatedImage.png until the getDownloadURL call returns an actual URL, but I worry about the amount of calls this would yield.
If there is a way for the cloud function to send a message to let me know that the conversion is finished, I can send a call once for the download URL after receiving said message. However, I can't figure out how to accomplish this.
Efforts so far
I have been pursuing course 1. So far, but have not met any success yet. Scouring through Firebase documentation, I haven't been able to find any supporting resources on how to accomplish 1 or 2. Does anyone have any suggestions - either on my planned courses of action, or a new option that I haven't considered?
You can use this onFinalize trigger to send a message or update a document in Firestore to indicate that the function has finished running. This trigger is triggered whenever a file is created or updated.
onFinalize Sent when a new object (or a new generation of an existing
object) is successfully created in the bucket. This includes copying
or rewriting an existing object. A failed upload does not trigger this
event.
you can also create a promise that resolves when the downloadURL is not null, and use that promise in your updateDocumentInFirestore function. This way, the updateDoc function will only be called once the downloadURL is available.
Additionally, as was mentioned in the comments, you can consider cloud workflow.The exact implementation will depend on your specific use case
You can also check these similar cases
Firebase Storage: Object does not exist
Error: storage/object-not-found when trying to upload large image file
Firebase Storage Put could not get object

Flutter & Firebase: How to populate an array and then later, return all the contents

I have been trying to get arrays working in Firebase, and I am aware that there are a lot of references and discussions about this online, and I have read through all of these and none of it works.
First off, the Firebase side. The structure containing the array and two example strings inside it:
Firebase Structure
collection -> document -> fields
userData profileImages URLs (array)
: https://firebasestorage.googleapis.com/v0/b/app-138804.appspot.com/o/jRwscYWLs1DySLMz7jn5Yo2%2Fprofile%2Fimage_picker4459623138678.jpg?alt=media&token=ec1043b-0120-be3c-8e142417
: https://firebasestorage.googleapis.com/v0/b/app-138804.appspot.com/o/jRwscYWLs3872yhdjn5Yo2%2Fprofile%2Fimage_picker445929873mfd38678.jpg?alt=media&token=ec3213b-0120-be9c-8e112632
The first issue I am facing is writing to this array in the database:
Firestore.instance.collection('userData').document('profileImages').updateData({
'URLs': _uploadedFileURL,
});
Whenever I add data to this array, it just overwrites the existing data. I need to be able to keep all the existing data intact and simply add the current new line to the array.
Once this is working, I then need to be able to return all of the strings in this array without needing to know how many of them there will be.
For this part, I basically have nothing at this point. I could show some of the things I have tried based on suggestions from other articles on this, but none of it is even close to working correctly.
im assuming that _uploadedFileURL is a String, and you are updating the property URLs, that's why your data gets overwritten, because you are changing the URLs value to a single string which is _uploadedFileURL. to solve this issue, simply get the current data inside profileImages before commiting the update. like so
final DocumentSnapshot currentData = await Firestore.instance.collection('userData').document('profileImages').get();
Firestore.instance.collection('userData').document('profileImages').updateData({
'URLs': [
...currentData.data['URLs'],
_uploadedFileURL
],
});
and for the second part of your question, all you need is to query for the profileImages
Future<List<String>> _getProfileImages() {
final document = Firestore.instance.collection('userData').document('profileImages').get();
return document.data['profileImages]
}
the result of the get method will be a DocumentSnapshot, and inside the data property will access the profileImages which is a List<String>.
Ok guys and girls I have worked this out. Part 1: appending data to an array in Firebase.
Firestore.instance.collection('userData').document('profileImages').updateDataupdateData({
'URLs':FieldValue.arrayUnion([_uploadedFileURL]),
});
Where _uploadedFileURL is basically a string, for these purposes. Now I have read that arrayUnion, which is super groovy, is only available in Cloud Firestore, and not the Realtime Database. I use Cloud Firestore so it works for me but if you are having issues this might be why.
Now what is extra groovy about Cloud Firestore is that you can similarly remove an element from the array using:
Firestore.instance.collection('userData').document('profileImages').updateDataupdateData({
'URLs':FieldValue.arrayRemove([_uploadedFileURL]),
});
So how to get this data back out again. A simple way I have found to get that data and chuck it into a local array is like so:
List imageURLlist = [];
DocumentReference document = Firestore.instance.collection('userData').document('profileImages');
DocumentSnapshot snapshot = await document.get();
setState(() {
imageURLlist = snapshot.data['URLs'];
});
From here at least you have the data, can add to it, can remove from it and this can be a platform for you to figure out what you want to do with it.

What is the difference between getDocuments() and snapshots() in Firestore?

I am a little confused about the difference between these two. My understanding is that getDocuments is a type of Future and seems to get the entire documents according to the query. while snapshots, on the other hand, is a type of Stream and, correct me if I'm wrong, I think it represents the results of the query? I need a more specific explanation of this issue. I will include some code snippets as an example for more clarification
getDocuments()
getUserById(String userId) async {
return await _firestore.collection("users").where("userId", isEqualTo: userId).getDocuments();
}
snapshots()
getUserById(String userId) async {
return await _firestore.collection("users").where("userId", isEqualTo: userId).snapshots();
}
So what's the difference?
When you call getDocuments(), the Firestore client gets the documents matching the query from the server once. Since this may take some time it returns a Future<QuerySnapshot>.
When you call snapshots() the Firestore client gets the documents, and then keeps watching the database on the server for changes that affect your query. So if document is written in the users collection that affects your query, your code gets called again. So this returns a stream of QuerySnapshot.
In both cases the results for the entire query are in the QuerySnapshot object.
I highly recommend reading the Firestore documentation on getting data once and on listening realtime updates. While they don't contain Flutter examples, the explanation in there applies equally to the Flutter libraries.
getDocuments():
It's used to provide data once. Cloud Firestore contains collections and inside these collections, you have documents that may contain subcollections or fields mapped to a value. To retrieve any of the doc fields to used it in widget this is used.
snapshots():
It will be called on every data change in your document query. For this StreamBuilder must be used to fetch fields as modified.
In short, it will do the job of setState() where it gives you the response for every modification so that UI can be updated.

How do I setup firebase realtime DB triggers for multiple databases using cloud functions?

I've a specific use case where I have multiple realtime DBs on a single project (and this number will grow) and I want to set up cloud functions triggers on all of them, currently I'm hoping if there's a way to get the DB name in the callback on which the cloud function is triggered?
import * as functions from 'firebase-functions';
import mongoose from 'mongoose';
export const updateData = functions.database.ref('/someendpoint/{code}').onUpdate(async (change, context) => {
$dbName = getFireBaseDBName(); //some function to get the DB name - This is the step that I would like to know how
await mongoose.connect(`mongo-db-string-connection/${dbName}`, {useNewUrlParser: true});
const Code = context.params.code;
const Schema = new mongoose.Schema({}, { collection: `someendpoint`, strict: false });
const Model = mongoose.model(`someendpoint`, Schema);
const after = change.after.val();
await Model.deleteMany({code: Code});
await Model.create({after, ...{code:Code}});
});
I need the DB name so that I can save to the database with the same name on Mongo.
For example:
Given I have a firebase project 'My-Project' and I have multiple Realtime Database instances on them say:
'db1', 'db2', 'db3'
When the trigger fires, I want to save/update/delete the data in MongoDB database so that it stays in sync with my Firebase Realtime database.
So it's crucial that not only do I get the data stored in db1 but also I get the name 'db1' so that the right data can be altered in Mongo.
Please keep in mind that more databases will be added to My-Project so
somewhere down the line it'll be 'db100.
First thing - I'll say that the way you're using database shards for multi-tenancy isn't really the way they're meant to be used. The Firebase team recommends using separate projects for multi-tenancy, one for each tenant, in order to keep users and their data isolated. The reason that database shards exist is to help developers deal with the scaling limitations of Realtime Database.
All that said, the triggers for Realtime Database don't directly provide the name of the shard in the callback. You will need to write one function for each shard, as required by the API, and described in the documentation.
To control when and where your function should trigger, call ref(path)
to specify a path, and optionally specify a database instance with
instance('INSTANCE_NAME'). If you do not specify an instance, the
function deploys to the default database instance for the Firebase
project For example:
Default database instance: functions.database.ref('/foo/bar')
Instance named "my-app-db-2": functions.database.instance('my-app-db-2').ref('/foo/bar')
Since you have to hard code the name of the shard in the function code, the you might as well just type it again inside the function itself. Or put them in global variables, and use them inside each function.
If you want to see an example of how to share code between each function declared for each instance, read this other question: How to trigger firebase function on all database instances rather than default one?

Watch a collectionGroup with Firestore using Cloud Functions

I'm working on a Firestore DB that uses collectionGroups.
The collectionGroup in question is a collection of 'Fights'.
When a new fight is created I would like to use the onCreate method in a cloud function to watch for new 'Fight' entries and then add some meta data to them. Ideally It would look something like the pseudo code below
export const placeFightersInEvent = functions.firestore
.collectionGroup('fights/{fightId}')
.onCreate(async (fightSnapshot, context) => {
// get metadata and add to the newly created 'fight'
});
I'm using the most up to date versions of the firebase functions and admin sdk but I can't seem to find an available function to do this. Is it possible to watch collection groups in this way?
Currently this is not possible for fights subcollections at any depth. Please file a feature request with Firebase support if you need to do this.
However, if you are only ever working with a fights subcollections at a known depth, then this might work just as well anyway:
export const placeFightersInEvent = functions.firestore
.document('{coll}/{doc1}/fights/{doc2}')
.onCreate(async (fightSnapshot, context) => {
// get metadata and add to the newly created 'fight'
});
It should only trigger for new fights nested below documents in any top-level collection, so it is not a true collection group event handler. But you could just create new functions for each depth required.

Resources