So I am working on a message app, i was thinking about a few methods to make the loading of data more appealing, because now, it just loads in really "ugly". basically it just jumps when new data gets added. I was think about just at the beginning really load all the data but store that data in lists to then later manipulate those instead of building everything again. What would your approach be? Am I not thinking correct there?
I was think about just at the beginning really load all the data but store that data in lists to then later manipulate those instead of building everything again.
That is how I've implemented it, storing updates as the state updates in the app. But this only works if all data is personal so you don't have two persons changing the same data at the same time so you get overrides in firebase.
If you have multiple changes to the same dataset it is better to work with realtime data imo. Read more at Firestore realtime changes
Related
I have just published an app that uses Firestore as a backend.
I want to change how the data is structured;
for example if some documents are stored in subcollections like 'PostsCollection/userId/SubcolletionPosts/postIdDocument' I want to move all this last postIdDocument inside the first collection 'PostsCollection'.
Obviously doing so would prevent users of the previous app version from writing and reading the right collection and all data would be lost.
Since I don't know how to approach this issue, I want to ask you what is the best approach that also big companies use when changing the data structure of their projects.
So the approach I have used is document versioning. There is an explanation here.
You basically version your documents so when you app reads them, it knows how to update those documents to get them to the desired version. So in your case, you would have no version, and need to get to version 1, which means read the sub-collections to the top collection and remove the sub collection before working with the document.
Yes it is more work, but allows an iterative approach to document changes. And sometimes, a script is written to update to the desired state and new code is deployed 😛. Which usually happens when someone wants it done yesterday. Which with many documents can have it's own issues.
I have a Firebase realtime DB i am using to track user analytics. Currently there is about 11 000 users and each of them has quite a bit of entries ( from ten to few hundreds based on how long they interacted with the app ). Json file is 76MBs when i export whole DB.
I am using this data only for analytics, so i will have a look once per day or so on all of the data. Ie i need to download whole DB to get all the data.
When i do that, it takes about 3-5 minutes to actually load the data. I can imagine that if there were ten times more users, it would not be usable then anymore, because of load time.
So i am wondering if these load times are normal and if this is realy bad practice to do such thing? The reason i always download whole DB, is that i want to get overall data, ie how many users is registered and then for example how many ads were watched. To do that, i need to go into each user and see how many ads he watched and count them up. I cant do that without having access to data of all users.
This is first time i am doing something like this on a bit larger scale and those 76MBs are a bit surprising to me as well as the load times to get the data. It seems like its not feasable long term to use this setup.
If you only need this data yourself, consider using the automated backups to get access to the JSON. These backups are made out-of-band, meaning that they (unlike your current process) don't interfere with the handling of other client requests.
Additionally, if you're only using the database for gathering user analytics, consider offloading the data to a database that's more suitable for this purpose. So: use Realtime Database for the user's to send the data to you, but remove it from there to a cheaper/better place after that.
For example, it is quite common to transfer the data to BigQuery, which has much better ad-hoc querying capabilities than Realtime Database.
I have a usecase wherein I save an entity for the first time and a second after saving it I fetch it, update it a bit and save it in batch along with two other entites (different 'kinds'). In a few cases (10 out of 50K), the update to datastore is ignored.
I mean, it's there in the objectify cache but the change didn't happen in datastore.
How I am able to justify the above, is because after the save, I fetch it again after a second and I'm able to see it.
PS: I also use .now() while saving. This shouldn't happen when now() is used right?
Sounds like you are seeing eventual consistency in the datastore. There is quite a bit of Google documentation available, but this looks to be the most comprehensive:
https://cloud.google.com/datastore/docs/articles/balancing-strong-and-eventual-consistency-with-google-cloud-datastore/
There are a lot of ways to deal with eventual consistency, either by avoiding it (using get-by-key operations), changing the structure of your data (using #Parent relationships), or masking it with UI behavior (say, add the new thing to the list in UI code instead of just refreshing the whole list).
I am new to IONIC. I want to know what is the ideal choice for storage. Is it IONIC storage?
Also i need to manually enter some data for the app. Now i can’t find any way to store data beforehand, is this possible in ionic?
For example, lets say i need a database/storage filled before with some values. How can i do that? Is that possible or i need to get the data from cloud?
I have posted my query in IONIC forum, this is the best answer i got:
Super simple solution: On start you check if a special value is set, e.g. databasePreloaded. If it is missing, you open a file, read the content and write it to the storage. Then you set databasePreloaded. On next start it will be present and the data won’t be loaded again.
My question is if the data is around 5-6 MB then what the ideal way to do that?
Using a check to see if the data has been loaded is a simple choice.
It would be easy to work with.
The main issue for me is whether the data will change at all once it has been deployed? You would need to think how this would be handled.
Finally, I'm under the impression IndexedDB can only handle up to 5mb so you will need to store this data in sqlite?
Most examples of Flux use a todo or chat example. In all those examples, the data set you are storing is somewhat small and and be kept locally so not exactly sure if my planned use of stores falls in line with the flux "way".
The way I intend to use stores are somewhat like ORM repositories. A way to access data in multiple ways and persist data to the data service, whatever that might be.
Lets say I am building a project management system. I would probably have methods like these for data retrieval:
getIssueById
getIssuesByProject
getIssuesByAssignedUser
getIssueComments
getIssueCommentById
etc...
I would also have methods like this for persisting data to the data service:
addIssue
updateIssue
removeIssue
addIssueComment
etc...
The one main thing I would not do is locally store any issue data (and for that matter most store data that related to a data store). Most of the data is important to have fresh because maybe the issue status has updated since I last retrieved that issue. All my data retrieval method would probably always make an API requests to the the latest data.
Is this against the flux "way"? Are there any issue with going about flux in this way?
I wouldn't get too hung up on the term "store". You need create application state in some way if you want your components to render something. If you need to clear that state every time a different request is made, no problem. Here's how things would flow with getIssueById(), as an example:
component calls store.getIssueById(id)
returns empty object since issue isn't in store's cache
the store calls action.fetchIssue(id)
component renders empty state
server responds with issue data and calls action.receiveIssue(data)
store caches that data and dispatches a change event
component responds to event by calling store.getIssueById(id)
the issue data is returned
component renders data
Persisting changes would be similar, with only the most recent server response being held in the store.
user interaction in component triggers action.updateIssue(modifiedIssue)
store handles action, sending changes to server
server responds with updated issue and calls action.receiveIssue(data)
...and so on with the last 4 steps from above.
As you can see, it's not really about modeling your data, just controlling how it comes and goes.