In the doc, it says :
Using the get() and exists() functions, your security rules can
evaluate incoming requests against other documents in the database.
Thats all right to me, and the example makes sense to me:
service cloud.firestore {
match /databases/{database}/documents {
match /cities/{city} {
// Make sure a 'users' document exists for the requesting user before allowing any writes to the 'cities' collection
allow create: if exists(/databases/$(database)/documents/users/$(request.auth.uid))
// Allow the user to delete cities if their user document has the
// 'admin' field set to 'true'
allow delete: if get(/databases/$(database)/documents/users/$(request.auth.uid)).data.admin == true
}
}
}
but an then it says
For writes, you can use the getAfter() function to access the state of
a document after a transaction or batch of writes completes but before
the transaction or batch commits.
I might still don't fully understand the concept. My questions are:
Why is it specifically have to use getAfter() for the transaction or batch write, Can we just use get()?
If you have to use getAfter() for transaction or batch write, does that mean you still need get() for normal write? how do they exist at the same time?
Thanks.
First, bear in mind that that security rules for writes kick in before anything in the database has been changed by that write. That's how the security rules are able to safely and efficiently reject access, without having to roll back any writes that already happened.
The documentation you're citing suggests that getAfter is useful to examine the contents of the database after the entire transaction's state would be recorded (in a sort of "staging" environment in memory), but before the transaction actually changes the database, visible to everyone. This is different than get, because get only looks at the actual contents of the database, before the transaction is finally committed. In short, getAfter uses then entire staged write of the entire transaction or batch, while get uses the actual existing contents of the database.
You are by no means obliged to use getAfter, if get works just fine for your case.
getAfter is useful when you need to examine other documents that may have been changed in the transaction or batch, and still have a chance to reject the entire transaction or batch by failing the rule. So, for example, if two documents being written in a single transaction must have some field value in common in order to be consistent, you need to use getAfter to verify the equality between the two. get wouldn't be helpful here, because it doesn't know anything about the other document in the transaction that hadn't been written yet.
On the other hand, if your rule needs to check if a document in the transaction has not changed an a field in an existing document (that is not the current document being checked), get would be necessary to fetch that value instead, before it's written by the transaction.
Related
I've created a new project on Firebase, and created a Realtime Database in there. When asked about the security rules for my database, I selected to Start in test mode.
Now the security rules of my database in the Firebase console show up as:
{
"rules": {
".read": "now < 1622790000000", // 2021-6-4
".write": "now < 1622790000000", // 2021-6-4
}
}
What do these rules mean? And how can I change them to be more secure?
It's been a month since I created my Firebase Realtime Database, and I now got a message:
Your project's Realtime Database '' will start denying client requests unless you update your security rules
These default test mode rules are a simple catch-all that allows everyone in the world to read from and write to your database until a given date.
Let's break the rules down to see exactly how they work:
The ".read" and ".write" nodes immediately under "rules" determine who can read/write the data in the entire database.
The now variable is automatically set by Firebase to be the current time on the server. This value is in milliseconds since the epoch, which is the recommended value to also store timestamps in Firebase.
The 1622790000000 value in the rules is the timestamp of some point in the future. Let's see what this value is in a more readable date format:
console.log(new Date(1622790000000))
"2021-06-04T07:00:00.000Z"
So anyone can read of write all data in our database until June 4th, 2021. After that date nobody can access the data anymore with the client-side SDKs. The Firebase Admin SDKs bypass these rules altogether, so they are not affected.
Can I extend the time period?
You may have gotten a message like this from Firebase:
You chose to start developing in Test Mode, which leaves your Realtime Database instance completely open to the Internet. Because this choice makes your app vulnerable to attackers, your database security rules were configured to stop allowing requests after the first 30 days. In 5 day(s), all client requests to your Realtime Database instance will be denied.
This message means that access to your data is about to expire, due to timestamp that is in your security rules.
It's actually pretty easy to extend the test mode to another deadline. All you need to do is change that 1622790000000 value. For example, for extend it to July 4th, I can set the value to 1625382000000.
To determine the value to use, I run this tiny JavaScript snippet:
console.log(new Date("2021-07-04T07:00:00.000Z").getTime())
Run this snippet to get the timestamp exactly one month from now:
console.log(new Date(Date.now()+30*24*60*60*1000).getTime())
Here's another tool to calculate these values.
By using 1625382000000 we've extended test mode for a month and everyone can read/write the entire database until July 4, 2021.
How can I better protect the data?
At some point you should come up with a better way to protect your (user's) data than just opening it until a specific date. I typically do this right when I start a project, but it's also fine if you start it a bit later.
The important thing is that you should treat the server-side security rules the same as the client-side source code of your app.
I develop my code and rules in tandem. So:
I start with a fully closed off database, since there is no code yet that needs access to any data.
I add some data manually to the database, and write code to read it. At this point, I write security rules that only allow read-access to that specific data. So it may be ".read": true, but it'll be much deeper in my JSON structure. Even such simple rules will already block many bad actors.
The first time I want the app to write to the database is also when I add authentication. Typically I start with anonymous auth, since it does not require me to enter any credentials.
I then include the hard-coded UID in my security rules, to ensure only I can write data. You'll often still find this top-level ".write": "auth.uid === 'hardcodedUidOfPufsAnonymousUser'" in my rules much later, after I added proper data ownership.
When using Firestore I sometimes evolve that as explained here: User conflict when using same Auth method for Admin and Normal users | Firebase Auth
At any point when I add (typically lists of) data, I think through who "owns" this data, and who can read it. I then expand my rules to allow exactly that access, and nothing more.
This need to update my security rules as I write code slows down the pace at which I code, but I'll gladly do it anyway. Keeping the data in my database secure at every step, allows me to give people access to the app/database with confidence. I recommend you do the same.
For more information, I recommend reading:
The Firebase documentation on security rules, which contains examples of these common use-cases:
Content-owner only access
Public read, private write access
Attribute and role based access
All authenticated users can read/write all datsa
I found myself in a situation where I want to perform some operations on the database that should be handled in a single transaction. One of those operations is injecting > 500 documents, so this is throwing an error because it's hitting
maximum 500 writes allowed per request
In order to work around that, you could use batched writes, but I can't figure out how to do batched writes as part of a transaction. It seems like transaction.commit() is not a thing and in the docs transactions and batched writes appear to be two separate concepts.
Generally speaking, we are using transactions to have consistent data. The recommendation that you get:
you could use batched writes
It is for the exact same reason. Unfortunately, you cannot mix them. You have to choose one or the other. Realistic speaking, both the batch and the transaction are used for atomic updates.
A transaction is similar to batch and as the docs states:
All of the operations succeed, or none of them are applied.
The main difference between a batch write and a transaction is that a batch just writes, while a transaction reads and right after then writes.
So the solution in your case is to use Firestore batched-writes to perform 500 operation at a time.
As you have most probably read in the doc:
The Transaction object passed to a transaction's updateFunction
provides the methods to read and write data within the transaction
context.
and this object, in the Client SDKs, has only four methods: get(), set(), update() and delete() which all take a single Firestore Document as parameter.
With the Node.js Server SDK for Google Cloud Firestore, you will note that there is an additional method, getAll(), which "retrieves multiple documents from Firestore. Holds a pessimistic lock on all returned documents".
So, at the time of writing, there is no possibility, to "mix" a Transaction and a Batched Write.
According to this answer, Firestore references cannot be used for JOIN-like queries, i.e. for retrieving the referencing document and the referenced document in one database round-trip. This can be a performance issue, since network latency costs apply for each database round-trip.
Network latency is only a problem if you are not close to the datacenter, which means that if you do the join server-side, i.e. in the Google datacenter where Firestore runs, then it should not be a problem.
Can we use Firebase Functions to implement this functionality in a generic manner? I'm thinking of a service, implemented in Firebase Functions, which sits between the client and the database. Most queries are just passed to the database (where, orderBy, limit etc. must still be possible), but there should be an additional populate: true query parameter. If this parameter is present and set to true, then the referenced documents are returned as well.
Maybe it will also be necessary to indicate which documents should be populated.
Sure, sounds like you could give it a try. You can still expect to pay for all documents read by whatever queries you use, even if you don't return that many to the client.
If you really have a lot of joins to perform, it's typically better to pre-compute the joins in another collection, and have the client query that instead. Then you have the advantage of the client having a local cache the helps both speed and cost.
But there's nothing stopping you from implementing a function that does this, if that's what you need.
I am using react-native-firebase package in a react native application and am trying to understand how transactions work in offline. I am trying to write a transaction using the following code
firebase.database().ref('locations').transaction(locations => {
... my location modification logic,
return locations
})
However, if I go offline before writing the transaction and have not accessed the reference previously and therefore have no cached data, locations is null.
There is this small tidbit in Firebase's official documentation
Note: Because your update function is called multiple times, it must
be able to handle null data. Even if there is existing data in your
remote database, it may not be locally cached when the transaction
function is run, resulting in null for the initial value.
Which leads me to believe I should wrap the entire transaction logic inside
if (locations) {
... my location modification logic
}
But I still don't fully understand this. Is the following assumption correct?
Submit transaction
If offline and cached data exists, apply transaction against cached data, then apply towards current data in remote when connectivity resumes
If offline and no cached data exists, do not apply transaction. Once connectivity resumes, apply transaction to current data in remote
If online, immediately apply transaction
If these assumptions are correct, then the user will not immediately see their change in case #3, but in case #2 it will 'optimistically' update their cached data and the user will feel like their action immediately took place. Is this how offline transactions work? What am I missing?
Firebase Realtime Database (and Firestore) don't support offline transactions at all. This is because a transaction must absolutely round trip with the server at least once in order to safely commit the changes to the data, while also avoiding collisions with other clients that could be trying to change the same data.
If you're wondering why the SDK doesn't just persist the callback that handles the transaction, all that can be said is that persisting an instance of an object (and all of its dependent state, such as the values of all variables in scope) is actually very difficult, and is not even possible in all environments. So, you can expect that transaction only work while the client app is online and able to communicate with the server.
I'm using firebase's .on() event listener to listen for entries inserted into my database in real time. I've noticed that even when a data insertion is denied, a user subscribed to the ref the data was supposed to be inserted into still gets that piece of data, even though in the database the data is never inserted.
I noticed this while developing the chat module of my web app. Here is a gif of the bug: https://gfycat.com/VariableFrailBasenji
I've set a validation rule on new messages that their length has to be under 200:
"$msg": { ".validate": "newData.val().length < 200"}
So when you see me paste in a bunch of letters, the console says the write gets denied, but the user who had the .on() subscription to that part of the database still got the message, even though it didn't get added to the database.
Anyways, this isn't really a post with a question, just wanted to share this strange bug that could potentially lead to data leakage.
The Firebase SDK uses advanced techniques like latency compensation when you write to the database. This means that before the server has acknowledged a write it goes into an in-memory database cache and optimistically assumes the write will be allowed.
When the server denies the write, it will remove the incorrectly cached data and you should see a corresponding child_removed event.