redux has the principle of "single source of truth", and I'm having and hard time figuring how polling fragments from GQL will continue to keep this principle.
TL;DR : Does apollo normalize entities for the whole app, or for each apollo container?
Apollo has his own redux store.
By the architecture graph I see that there is a normalization stage before applying the data into redux.
My questions is: lets say I have a post schema and a user schema.
when polling a post, I pull the post information, and also the author (user schema) name and profile pic.
I have a posts container with apollo.
so I'm GUESSING apollo store now has this state :
{
posts: {
512 : {
title: "title"
content: "big content"
author: [32]
}
},
users: {
32 : {
name: "James"
profilePic: "YYYYYY"
}
}
}
but what happens when the user - James, change his name to Julian, and another component, the chat menu component which updates frequently will have the user id : 32 with different name, Julian.
will the posts container will be updates too because of this?
or it will keep displaying out of date data?
Great question!
Your answer can be found here: http://dev.apollodata.com/core/how-it-works.html#normalize
Going to summarize in case the link breaks:
apollo-client doesn't store things the way you request it, but rather as a flat structure (and generate an ID for that object based on id, __typename and keep track of the references between the objects.
When components request those objects it will take the objects from the flattened structure and build it into the requested format.
Related
I have a state object with the following branches (trying to adhere to "Normalizing the state shape"):
Users
An array of elements like
{
id: 1,
name: "Werner"
}
originating from some server.
User locations
An array of elements like
{
userId: 1,
latitude: 45,
longitude: 70
}
originating from some server.
The problem
The users might change depending on a number of actions: SET_USERS_ACTION, ADD_USER_ACTION, DELETE_USER_ACTION.
Every time something happens to the users, I want to update the user locations (which is an asynchronous operation, as the data needs to come from the server). The how of the matter is what I'm struggling with.
Obviously, I can't fetch the user locations in the reducer (when updating the users), as the reducer would no longer be pure in that case.
I might do it in the thunk, but that would mean I have to add user location considerations to every action creator involving user-actions, which seems like mixing concerns to me.
Additionally, once an action is added that changes the users array in some way, the developer needs the remember to also update the user locations. My experience is that stuff like this will almost always be forgotten at some point.
Further complications
To further complicate the matter, we don't always need to fetch the locations. Only if a component displaying a map with all users is active, does it make sense to fetch the user locations. Not every action is generated at a place where I know (beforehand) if that component is visible or not. One example is when we receive a notification from the server (with Web Sockets) that a user was added or removed.
What is the best way of solving this problem?
I'll suggest to use https://github.com/kolodny/immutability-helper The benefit of using the update helper is that you are able to do many changes at once without touching the state many times. For example:
import update from 'immutability-helper';
...
case SET_USERS_ACTION:
return update(
state,
{
users: {
[idx]: { status: { $set: 'ready' }}
},
locations: {
$push: [{...}]
}
}
);
break;
Almost every object in my state tree needs to have the following fields:
createdByGuid
createdAt
modifiedAt
The date fields are easy to add in the action creators. There seems to be consensus that action creators are the place where impureness are allowed, i.e. new Date().
But the currently logged in user is not easily fetched since this data lives in another slice of the state.
Example state:
profile
data
guid
firstname
...
media
items
0: guid, createdByGuid, ...
1: guid, createdByGuid, ...
drawings
items
...
When a MEDIA_ADD action is dispatched I want the action creator to augment the dispatched data with createdById. Current user needs to be fetched from profile.data.guid.
I don't want to bring the state into my action creator. Instead I can use a thunk to get hold of the state via getState(). But this seems to be a clumsy way of getting hold of a simple guid constant (constant for this session). It becomes more clumsy when essentially all of my action creators for adding data to the state needs to be thunks.
I'm inclined to create a separate global singleton object to hold this data and bring this dependency into all of my action creator modules. This seems to be a lot less disturbing dependency in terms of for example testing.
I suppose most Redux users has about the same question regarding current user. What is a good solution?
I'm working in react-native, but I suppose the question applies to most Redux applications.
If someone is interested, I went the singleton way with this. Not a true singleton per definition, but it suits my needs.
When a user logs in I handle this in an asynchronous action creator (a thunk). The thunk calls setGlobalCurrentUser() after successfully authenticating the user with the server.
To get hold of the current user I only have to import currentUserGuid() from currentUser.js and I don't end up in a dependency hell.
currentUser.js:
let _currentUserGuid;
export function currentUserGuid() {
if(!_currentUserGuid) throw new Error('currentUserGuid:: No logged in user')
return _currentUserGuid;
}
export function setGlobalCurrentUser(user) {
if(user) {
_currentUserGuid = user.get('guid')
} else {
_currentUserGuid = null
}
}
Since Firebase security rules cannot be used to filter children, what's the best way to structure data for efficient queries in a basic multi-user application? I've read through several guides, but they seem to break down when scaled past the examples given.
Say you have a basic messaging application like WhatsApp. Users can open chats with other groups of users to send private messages between themselves. Here's my initial idea of how this could be organized in Firebase (a bit similar to this example from the docs):
{
users: {
$uid: {
name: string,
chats: {
$chat_uid : true,
$chat2_uid: true
}
}
},
chats: {
$uid: {
messages: {
message1: 'first message',
message2: 'another message'
}
}
}
}
Firebase permissions could be set up to only let users read chats that are marked true in their user object (and restrict adding arbitrarily to the chats object, etc).
However this layout requires N+1 selects for several common scenarios. For example: to build the home screen, the app has to first retrieve the user's chats object, then make a get request for each thread to get its info. Same thing if a user wants to search their conversations for a specific string: the app has to run a separate request for every chat they have access to in order to see if it matches.
I'm tempted to set up a node.js server to run root-authenticated queries against the chats tree and skip the client-side firebase code altogether. But that's defeating the purpose of Firebase in the first place.
Is there a way to organize data like this using Firebase permissions and avoid the N+1 select problem?
It appears that n+1 queries do not necessarily need to be avoided and that Firebase is engineered specifically to offer good performance when doing n+1 selects, despite being counter-intuitive for developers coming from a relational database background.
An example of n+1 in the Firebase 2.4.2 documentation is followed by a reassuring message:
// List the names of all Mary's groups
var ref = new Firebase("https://docs-examples.firebaseio.com/web/org");
// fetch a list of Mary's groups
ref.child("users/mchen/groups").on('child_added', function(snapshot) {
// for each group, fetch the name and print it
String groupKey = snapshot.key();
ref.child("groups/" + groupKey + "/name").once('value', function(snapshot) {
System.out.println("Mary is a member of this group: " + snapshot.val());
});
});
Is it really okay to look up each record individually? Yes. The Firebase protocol uses web sockets, and the client libraries do a great deal of internal optimization of incoming and outgoing requests. Until we get into tens of thousands of records, this approach is perfectly reasonable. In fact, the time required to download the data (i.e. the byte count) eclipses any other concerns regarding connection overhead.
I'm doing my meteor app and it has 1 Collection: Students
In Server I made a Publish that receives 3 params: query, limit and skip; to avoid client to subscribe all data and just show the top 10.
I have also 3 Paths:
student/list -> Bring top 10, based on search input and pagination (using find);
student/:id -> Show the student (using findOne)
student/:id/edit -> Edit the student (using findOne)
Each Template subscribe to the Students collection, but every time the user change between this paths, my Template re-render and re-subscribe.
Should I make just one subscribe, and make the find based on this "global" subscription?
I see a lot of people talking about Template level subscription, but I don't know if it is the better choice.
And about making query on server to publish and not send all data, I saw people talking too, to avoid data traffic...
In this case, when I have just 1 Collection, is better making an "global" subscription?
You're following a normal pattern although it's a bit hard to tell without the code. If there many students then you don't really want to publish them all, only what is really necessary for the current route. What you should do is figure out why your pub-sub is slow. Is it the find() on the server? Do you have very large student objects? (In which case you will probably want to limit what fields are returned). Is the search you're running hitting mongo indexes?
Your publication for a list view can have different fields than for a individual document view, for example:
Meteor.publish('studentList',function(){
let fields = { field1: 1, field2: 1 }; // only include two fields
return Students.find({},fields);
});
Meteor.publish('oneStudent',function(_id){
return Students.find(_id); // here all fields will be included
});
I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => {
let newRef = ref.child('list-index').push(newItem);
resolve( newRef.key()); // ignore reject() for brevity
});
addIndexPromise.then( key => {
ref.child('list').child(key).set(newItem);
});
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?
Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users
so:209103
name: "Frank van Puffelen"
location: "San Francisco, CA"
questionCount: 12
so:3648524
name: "legolandbridge"
location: "London, Prague, Barcelona"
questionCount: 4
messages
-Jabhsay3487
message: "How to write denormalized data in Firebase"
user: so:3648524
username: "legolandbridge"
-Jabhsay3591
message: "Great question."
user: so:209103
username: "Frank van Puffelen"
-Jabhsay3595
message: "I know of three approaches, which I'll list below."
user: so:209103
username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Transactional update
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) {
var updates = {}; // all paths to be updated and their new values
updates['users/'+uid+'/name'] = name;
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
updates['messages/'+messageSnapshot.key()+'/username'] = name;
})
ref.update(updates);
});
}
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/');
var uid = "so:209103";
var nameInProfileRef = ref.child('users').child(uid).child('name');
nameInProfileRef.transaction(function(currentName) {
return "puf";
}, function(error, committed, snapshot) {
if (error) {
console.log('Transaction failed abnormally!', error);
} else if (!committed) {
console.log('Transaction aborted by our code.');
} else {
console.log('Name updated in profile, now update it in the messages');
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.on('child_added', function(messageSnapshot) {
messageSnapshot.ref().update({ username: "puf" });
});
}
console.log("Wilma's data: ", snapshot.val());
}, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.
change all username fields for messages by so:209103 to null (some magic value)
change the name of user so:209103 to 'puf'
change the username in every message by so:209103 that is null to puf.
that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.
client-side code that handles all these transitions transactionally.
This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Eventual consistency
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) {
ref.child('users').child(uid).update({ name: name });
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
messageSnapshot.update({ username: name });
})
});
}
Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
Not caring
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
Summary
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.
To add to Franks great reply, I implemented the eventual consistency approach with a set of Firebase Cloud Functions. The functions get triggered whenever a primary value (eg. users name) gets changed, and then propagate the changes to the denormalized fields.
It is not as fast as a transaction, but for many cases it does not need to be.