I'm setting up a Firestore database and am playing around with structuring it. Is there a way to populate and change it quickly without having to add/change fields manually every single time?
Two example things I am looking to do are:
1) Populate collections with documents that have predetermined fields. Currently I have to add the fields manually every single time.
2) Edit the fields en masse for all documents within a collection (e.g. change the name of a field, delete a field entirely, add a new field)
The Firebase console doesn't seem to provide these tools, would my best bet be to write a separate app specifically for this purpose?
Since such bulk uploads and bulk edits are not part of the console, you'll have to build something yourself indeed.
A good place to start would be the Cloud Firestore API, which allows adding and updating documents in the database.
Related
Suppose I have a users collection. The users collection has a large number of documents in it. Now in my app, I have a feature request that forces me to add or remove a field in my users collection data model. How can I add a new field or remove an existing field from all my users documents? Is there any best practice that the community recommends here?
How can I add a new field or remove an existing field from all my users documents?
While #AdityaNandardhane solution might work, please note that if you have a lot of documents, then you have a lot of update operations to perform, which also means that you have to play a lot of writes.
So the best approach would be to perform the update, only when the user reads the document. When it comes to users, most likely the details of the users are displayed on a profile screen. This means that when the users want to check the profile, before displaying the data, check for the existence of the new field. If it doesn't exist, then perform the update operation, and right after that display the data, otherwise, just display the data. This means that you'll have to pay for an update operation only when needed. It doesn't make any sense to update all documents, of all users, since there may be users that will never use their accounts anymore. So there is no need to pay for them.
As I understood, You can do the following thing
1. Add New Field
If you are using Firebase Functions- you can create one function and write an update query with a new field and set one default value and Run the function. You can do the same from android also with kotlin/java.
2. Remove existing Field
If you are using Firebase Functions- you can create one function and write a query to delete one field and Run the function. You can do the same from android also with kotlin/java.
Look for a better approach If any, Its suggestion as per my knowledge.
I have just published an app that uses Firestore as a backend.
I want to change how the data is structured;
for example if some documents are stored in subcollections like 'PostsCollection/userId/SubcolletionPosts/postIdDocument' I want to move all this last postIdDocument inside the first collection 'PostsCollection'.
Obviously doing so would prevent users of the previous app version from writing and reading the right collection and all data would be lost.
Since I don't know how to approach this issue, I want to ask you what is the best approach that also big companies use when changing the data structure of their projects.
So the approach I have used is document versioning. There is an explanation here.
You basically version your documents so when you app reads them, it knows how to update those documents to get them to the desired version. So in your case, you would have no version, and need to get to version 1, which means read the sub-collections to the top collection and remove the sub collection before working with the document.
Yes it is more work, but allows an iterative approach to document changes. And sometimes, a script is written to update to the desired state and new code is deployed 😛. Which usually happens when someone wants it done yesterday. Which with many documents can have it's own issues.
I have a few music albums - basically just files in folders - that I want to upload to Firebase Storage.
One would usually run a function after a file has been uploaded to create a Document containing the metadata about the Song but that's where Im stuck.
I can get most infos I need by reading the Tracks ID3 Tags but in a NoSql Database I think im supposed to not only create a Document for the Track but also a Document for each album with an array of all tracks - or at least an array with all track ids.
But when or how do I create the Album Document? Another example is the Album Cover.. I want to save the Url inside the Track Document as well as in the corresponding Album but that means that the Artwork is the first thing I need to upload because I can't add an URL because it doesn't exist yet.
I feel like I have to get this right before I start because updating everything afterwards is a pain.
Is using upload functions really the way to go here or is there really a tool or another way im missing.
thank you very much
You mentioned Firebase Storage wich is a just a cover for Cloud Storage and it's a obejct managment system not a Database, however I think you are refering to Firebase Firestore.
On firestore since as you mentioned is a NoSQL DB and the schema structure your Db should have, There no correct way to do this and will defitetly depend on each specific use case. However you can take a look at this docs where it's expalined how to arquitecture your schema thinking from a SQL to a NoSQL format.
Among other information the main pointsa are:
In general, you can treat documents as lightweight JSON records
You have complete freedom over what fields you put in each document
After you create the first document in a collection, the collection exists. If you delete all of the documents in a collection, it no longer exists.
You can use sub collections inside of collections
Deleting a document does not delete its subcollections!
And finally to have an idea on how to structure the information, you can take a look at this repo where "NoSQL-Spotify by Luke Halley" explains a NoSQL schema based on spotify so I think it shoudl fit your need or at least give you a starting point.
I have set up all the currently needed Fields in my firestore collection documents, but I fear I may need to add new fields later in future as the usage of the app grows.
My question is, will I need to delete the whole collection just to add another new field in future? or will i simply add a new field thereby changing the structure of all the future documents that will be created in the collection.
Thanks
I fear I may need to add new fields later in future as the usage of the app grows.
It happens all the time. Is normal. As the app grows, new features are needed.
Will I need to delete the whole collection just to add another new field in future?
No and never think about that. You can simply update each document with the new properties that you need. You can easily do that using a POJO class, as explained in my anwer from this post or even simpler using a Map, as explained here.
Firestore, like most NoSQL databases, is schema-less. This means there is no structure to the data you put in it. The only structure is that which you impose by your own code. You could have millions of documents in a collection that all have completely different fields, and they will not conflict with each other in any way. You can add and remove fields at any time. You choose whatever suits your application the best.
If I have User and Profile objects. What is the best way to structure my collections in firestore given that the follow scenarios can take place?
Users have a single Profile
Users can update their Profile
Users can save other users' profiles
Users can deleted their saved profiles
The same profile can't be saved twice
If Users and Profiles are separate collections, what is the best way to store saved profiles?
One way that came to mind was that each user has a sub collection called SavedProfiles. The id of each document is the id of the profile. Each saved Profile only contains a reference to the user who's profile it belongs to.
The other option was to do the same thing but store the whole profile of each saved profile.
The benefits of the first approach is that when a user updates their own profile there's no need to update any of the their profiles that have already been saved as it's only the reference that is stored. However, attempting to read a user's saved profiles may require two read operations (which will be quite often), one to get all the references then querying for all the profiles with those reference (if that's even possible???). This seems quite expensive.
The second approach seems like the right way to go as it solves the problem of reading all the saved profiles. But updating multiple saved profiles seems like an issue as each user's saved profiles may be unique. I understand that it's possible to do a batch update but will it be necessary to query each user in the db for their saved profiles and check if that updated profile exists, if so update it? I'm not too sure which way to go. I'm not super used to NoSQL data structures and it already seems like I've done something wrong since I've used a sub collection since it's advised to keep everything as denormalized as possible so please let me know if the structure to my whole db is wrong too, which is also quite possible...
Please provide some examples of how to get and update profiles/saved profiles.
Thank you.
Welcome to the conundrum that is designing a NoSQL database. There is no right or wrong answer, here. It's whatever works best for you.
As you have identified, querying will be much easier with your second option. You can easily create a Cloud Function which updates any profiles which have been modified.
Your first option will require multiple gets to the database. It really depends how you plan to scale this and how quick you want your app to run.
Option 1 will be a slow user experience, while all of the data is fetched. Option 2 will be a much faster user experience, but will requre your Cloud Function to update every saved profile. However, this is a background task so wouldn't matter if it takes a few seconds.