We are using the code below in an application to execute a write to the database. However, multiple clients may be writing at the same time so we used a Transaction as this is supposed to run again if the data is modified by another Transaction during execution.
Something seemed to go wrong though: When seperate clients (two different mobile devices) are running the code below ± simultaneously, only one of them seems to update the value in the database.
What could be the cause of this and how can we prevent this in the future?
static Future< bool > voteOnUser(GroupData groupData, String voteeID) async {
await Firestore.instance.runTransaction((Transaction transaction) async {
DocumentReference groupRef = Firestore.instance
.collection("groups")
.document( groupData.getGroupCode() );
DocumentSnapshot ds = await transaction.get( groupRef );
/// Initialize lists
Map<dynamic, dynamic> dbData = ds.data['newVotes'];
Map<String, int> convertedData = new Map<String, int>();
/// Loop the database Map and add the values to the data Map
/// All data is assumed to exist and definitely be of the desired types
dbData.forEach( (key, value) {
convertedData[key.toString()] = value;
} );
if (convertedData.containsKey( voteeID ))
convertedData[voteeID] += 1;
else convertedData[voteeID] = 1;
Map<String, dynamic> incremented = new Map<String, dynamic>();
incremented['newVotes'] = convertedData;
await transaction.update(groupRef, incremented);
});
return true;
}
Edit: The purpose of this code is to vote on a user that is located in a map within a document. Several clients may vote at the same time and each vote should increment the value of one element in the map by one.
When two clients vote at the same time on for example the user with ID 'joe' who had no votes before, the expected result is that joe would have two votes. However, when testing simultaneous voting, we observed that only one vote was added to the database instead of the desired two votes.
Edit 2: At first, the document exists and contains an empty map newVotes. We were testing with two different mobile devices and casted a vote at ± the same point in time. Almost instantly, the document was updated and a user ID was added to the previously empty map with one vote while we expected two votes to be mapped to that key.
Edit 3: First, a document is created with the following fields: newVotes (map) to map a number of votes to a unique user id, totalVotes (map) to keep track of the total votes and members (List) which is a list of all user id's in that group. newVotes And totalVotes are empty at this point and members contains a few IDs (in this case, two unique Google IDs)
Once this document exists, clients can vote with the given code. In my case we both voted on the same google user ID (say: "ID1") and expected the map newVotes to contain one element: {"ID1": 2}. However, we both voted at the same time (no errors were thrown) and the newVotes field contained {"ID1":1}.
Transactions does not guarantee protection from modification of the same entity concurently - unless SERIALIZED (or "Read for update locking"). It guarantees that every operations in transaction will be applied at once upon commit or discarded if tx is rolled back
To guarantee that there are no conflicts either:
Synchronize access to entity - lock upon reading
Synchrionize method execution
Serialize transaction
Use version controll mechanism
Related
I'm working on the Flutter app where users can save multiple addresses. Previously I used a real-time database and it was easier for me to push data in any child with a unique Id but for some reason, I changed to Firestore and the same thing want to achieve with firestore. So, I generated UUID to create unique ID to append to user_address
This is how I want
and user_address looks like this
And this is how it's getting saved in firestore
So my question Is how I append data with unique id do I have to create a collection inside users field or the above is possible?
Below is my code I tried to set and update even user FieldValue.arrayUnion(userServiceAddress) but not getting the desired result
var uuid = Uuid();
var fireStoreUserRef =
await FirebaseFirestore.instance.collection('users').doc(id);
Map locationMap = {
'latitude': myPosition.latitude,
'longitude': myPosition.longitude,
};
var userServiceAddress = <String, dynamic>{
uuid.v4(): {
'complete_address': completedAddressController.text,
'floor_option': floorController.text,
'how_to_reach': howtoreachController.text,
'location_type': locationTag,
'saved_date': DateTime.now().toString(),
'user_geo_location': locationMap,
'placeId': addressId
}
};
await fireStoreUserRef.update({'user_address': userServiceAddress});
If I use set and update then whole data is replaced with new value it's not appending, so creating a collection is the only solution here and If I create a collection then is there any issue I'll face?
You won't have any issues per se by storing addresses in a separate collection with a one-to-many relationship, but depending on your usage, you may see much higher read/write requests with this approach. This can make exceeding your budget far more likely.
Fortunately, Firestore allows updating fields in nested objects via dot notation. Try this:
var userServiceAddress = {
'complete_address': completedAddressController.text,
'floor_option': floorController.text,
'how_to_reach': howtoreachController.text,
'location_type': locationTag,
'saved_date': DateTime.now().toString(),
'user_geo_location': locationMap,
'placeId': addressId
};
await fireStoreUserRef.update({'user_address.${uuid.v4()}': userServiceAddress});
In Firestore, I have a collection of fruits containing documents with automatically generated ids, and a name property.
I want to insert a new fruit document, with an automatically generated id, and only if no other with the same name exists.
Inspired by this answer, I try this:
Edit: For the record, as detailed in the accepted answer: this code is NOT transactionally safe: it does NOT prevent race conditions which could insert the same fruit name under heavy load
const query = firestore.collection(`/fruits`).where("name", "==", "banana").limit(1);
await firestore.runTransaction(async transaction => {
const querySnapshot = await transaction.get(query);
if (querySnapshot.length == 0) {
const newRef = firestore.collection(`/fruits`).doc();
await transaction.create(newRef, { name: "banana" });
}
});
But I wonder: is newRef guaranteed to be un-used?
Otherwise, does the transaction automatically retries (due to the create failing) until success?
Otherwise, how can I insert my fruit?
Note: I use the node.js admin SDK, but I think the problem is the same with the javascript API.
Edit: here is how I do it finally:
const hash = computeHash("banana"); // md5 or else
const uniqueRef = firestore.doc(`/fruitsNameUnique/${hash}`);
try {
await firestore.runTransaction(async transaction => {
transaction.create(uniqueRef, {}); // will fail if banana already exists
const newRef = firestore.collection(`/fruits`).doc();
transaction.create(newRef, { name: "banana" });
});
} catch (error) {
console.log("fruit not inserted", error.message);
}
is newRef guaranteed to be un-used?
It is virtually guaranteed to be unique. The chances of two randomly generated document IDs is astronomically small.
See also:
Firestore: Are ids unique in the collection or globally?
Are Firestore ids unique across the whole db
One thing you should be aware of is that your code is not actually transactionally safe. There is nothing stopping two clients from adding a new fruit where name=banana in a race condition between the moment of the query and the moment the transaction actually creates the new document. Under low traffic situations it's probably OK, but you are taking a chance on that.
In fact, Firestore doesn't have a built-in way to ensure uniqueness of a document's field value. It will require a fair amount of extra work to implement that yourself, perhaps by using that field value as the unique key in another collection, and making sure that collection is part of a bigger transaction that deals with the documents in your fruits collection.
I'm quite new to Firebase Flutter. I'm developing a mobile application to share books among others.
In firebase firestore,
I have 'users' collections which contain all the user data with unique id
I have 'books' collection which contain all the book data with unique id created automatically
Also I have 'global' collection with single document with one integer field called 'bookcount'.
Users can can have many books.
Now I want to create a another unique id field for book. idea is to have simple integer id.
One way of doing this is get list of books and find the length (count) and add 1 when creating a new record. I have ruled out this method as if many users using simultaneously, I think this can lead to duplicate ids.
So I have created a another collection global with single document and field name bookcount. Which hold number of books (rough count) on books collection. So idea is each time when adding a book to a collection increase bookcount and use this value as simple unique id for a book. This bookcount may not represent actual books as user can discard the book entry before saving it, which is okay as I only need a simple unique id.
class DatabaseService {
...
...
//final CollectionReference bookCollection = Firestore.instance.collection('users');
//final CollectionReference bookCollection = Firestore.instance.collection('books');
final CollectionReference globalData = Firestore.instance.collection('global');
...
...
Future<String> bookId() async
{
String uniquebookid = await globalData.document('SomeHardcodedID').updateData(
{
'bookcount': FieldValue.increment(1)
}).then((voidvalue) async
{
String cid = await globalData.getDocuments().then((bookvalue) => bookvalue.documents.single.data['bookcount'].toString());
return cid;
});
return uniquebookid;
}//future bookId
...
...
}//class
Now this works. well somewhat, Can we do this better? In here there are two parts, first increment the value bookcount, and then retrieve it.
Can we do this in one go?
If I try to call this method consecutively really fast when returning a value it might skip few numbers. I have call this from a button and try to press as fast I could. I think counter increase but it return
same number few times. and then skip some when press again. for example 1,2,3,4,8,8,8,8,9,10,... So at counter 4 I try to press the button multiple times. I wonder how this will behave when multiple users adding multiple books at the same time.
How Can I fix this?
Please Help, Thanks.
I think the problem was since await globalData.document('SomeHardcodedID').updateData is not producing a return value (void), as soon as this fired next call also execute which is okay, which okay for most scenarios.
However if bookId called few times within very short period (milliseconds) this produce number before FieldValue.increment(1) process.
My application uses dates as document Id.
Each day stores the items the user bought and their total price.
The user might buy items from his local store twice or more in a day.
How can I allow that in my flutter code?
All the questions I found, ask about how not to duplicate document ID. I guess I'm the only one who want to do this.
The code just in case:
/// Save to collection [user], document [day], the list of items in [cartMap], and the total amount [total]
static Future<void> saveToFirestore(String day, Map<GroceryItem, int> cartMap, double total) async {
bool paidStatus = false;
List<String> x = day.split('-');
assert(matches(x[2], '[0-9]{2}')); // make sure the day is two digits
assert(matches(x[1], '\\b\\w{1,9}\\b')); // make sure the month is a word between 1 and 9 length
assert(matches(x[0], '[0-9]{4}')); // make sure it's a number year of four digits
final groceries = Groceries.cartMap(cartMap).groceries;
final user = await CurrentUser.getCurrentUser();
CollectionReference ref = Firestore.instance.collection(user.email);
DocumentReference doc = ref.document(day);
final dateSnapshot = await doc.get();
Firestore.instance.runTransaction((transaction) async {
updatePaidUnpaidTotal(day, user.email, total);
if(dateSnapshot.exists) {
print('Updating old document');
List<dynamic> existingItems = dateSnapshot.data['items'];
var totalItems = existingItems + groceries;
return await transaction.update(doc, {'total': FieldValue.increment(total), 'items': totalItems, 'paidStatus': false},);
}
else {
print('New document');
return await transaction.set(doc, {'total': total, 'items': groceries, 'paidStatus': paidStatus});
}
});
}
I strongly recommend not using dates (or any actual data at all) in a document ID, for the reason that you're running into.
There is rarely a requirement in an app to use a specific format for a document ID. It might be convenient in some cases, but ultimately, it constrains the future expansion of your collection.
One flexible way of adding any document at all, is to simply use add() to accept a random ID for the new document. Then you can put the date in a field in the document, and use that in your queries as a filter on that field. This is going to give you much more freedom to change things up later if you want, so you are not bound the dates for IDs. There are no real performance penalties for this.
The only reason to use a non-random string as a document ID is if you need to absolutely enforce uniqueness on that value. Since your application no longer wants to enforce this, it's better to simply remove that constraint and use random IDs.
It is not possible to have several Firestore documents in the same (sub)collection with the same document ID.
A classical approach in your case is to create one document per day and create a subcollection which contains a document for each item the user bought during this day.
Something like:
orders (collection)
- 20200725 (doc)
- orderItems (subcollection)
- KJY651KNB9 (doc with auto generate id): {'items': totalItems, 'paidStatus': false ... }
- 20200726 (doc)
- orderItems (subcollection)
- AZEY543367 (doc with auto generate id): {'items': totalItems, 'paidStatus': false ... }
- AZEY5JKKLJ (doc with auto generate id): {'items': totalItems, 'paidStatus': false ... }
- AZEY598T36 (doc with auto generate id): {'items': totalItems, 'paidStatus': false ... }
Is there a way to fetch document after documentID like
private fun fetchCollectoionnAfterDocumentID(limit :Long){
val db = FirebaseFirestore.getInstance()
var query:Query = db.collection("questionCollection")
.startAfter("cDxXGLHlP56xnAp4RmE5") //
.orderBy("questionID", Query.Direction.DESCENDING)
.limit(limit)
query.get().addOnSuccessListener {
var questions = it.toObjects(QuestionBO::class.java)
questions.size
}
}
I want to fetch sorted questions after a given Document ID. I know I can do it using DocumentSnapShot. In order to fetch the second time or after the app is resume I have to save this DocumentSnapshot in Preference.
Can It be possible to fetch after document ID?
startAfter - > cDxXGLHlP56xnAp4RmE5
Edit
I know I can do it using lastVisible DocumentSnapshot . But I have to save lastVisible DocumentSnapshot in sharedPreference.
When app launch first time 10 question are fetched from questionCollection. Next time 10 more question have to be fetched after those lastVisible. So for fetching next 10 I have to save DocumentSnapshot object in sharedPreference. Suggest me a better approach after seeing my database structure.
And one more thing questionID is same as Document reference ID.
There is no way you can pass only the document id to the startAfter() method and simply start from that particular id, you should pass a DocumentSnapshots object, as explained in the official documentation regarding Firestore pagination:
Use the last document in a batch as the start of a cursor for the next batch.
first.get().addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
#Override
public void onSuccess(QuerySnapshot documentSnapshots) {
=// Get the last visible document
DocumentSnapshot lastVisible = documentSnapshots.getDocuments()
.get(documentSnapshots.size() -1);
// Construct a new query starting at this document,
Query next = db.collection("cities")
.orderBy("population")
.startAfter(lastVisible) //Pass the DocumentSnapshot object
.limit(25);
// Use the query for pagination
}
});
See, here the lastVisible is a DocumentSnapshot object which represents the last visible object. You cannot pass only a document id. For more information, you can check my answer from the following post:
How to paginate Firestore with Android?
It's in Java but I'm confident you can understand it and write it in Kotlin.
Edit:
Please consider defining an order of your results so that all your pages of data can exist in a predictable way. So you need to either specify a startAt()/startAfter() value to indicate where in the ordering to begin receiving ordered documents or use a DocumentSnapshot to indicate the next document to receive, as explained above.
Another solution might be to put the document id into the document itself (as a value of a property) and order on it, or you can use FieldPath.documentId() to order by the id without having to add one.
You can also check this and this out.
There is one way to let startAfter(documentID) works.
Making one more document "get", then using the result as startAfter input.
val db = FirebaseFirestore.getInstance()
// I use javascript await / async here
val afterDoc = await db.collection("questionCollection").doc("cDxXGLHlP56xnAp4RmE5").get();
var query:Query = db.collection("questionCollection")
.startAfter(afterDoc)
.orderBy("questionID", Query.Direction.DESCENDING)
.limit(limit)
A simple way to think of this: if you order on questionID you'll need to know at least the value of questionID of the document to start after. You'll often also want to know the key, to disambiguate between documents with the same values. But since it sounds like your questionID values are unique within this collection, that might not be needed here.
But just knowing the key isn't enough, as that would require Firestore to scan its entire index to find that document. Such an index scan would break the performance guarantees of Firestore, which is why it requires you to give you the information it needs to perform a direct lookup in the index.