I use Redux Toolkit, and in particular the new listener api, to perform tasks similar to what I could do with Redux-Saga.
Unfortunately, since a few days, I'm stuck with a memory leak and I can't find the cause.
I have reproduced a minimal example of the code that produces this memory leak, link to the example : https://github.com/MrSquaare/rtk-memory-leak
To observe this memory leak :
I use Chromium, DevTools memory tool
I trigger a garbage collector
I make a heap memory snapshot
I dispatch entity/load (via the UI button)
I make several heap memory snapshots every 2-3 seconds
I use the comparison tool, I notice that I have the array allocation size growing infinitely
And after dispatch entity/unload, then make a snapshot heap memory, we can observe that the allocations disappear...
Has anyone observed similar behavior? Or does anyone have an idea of the cause? Thanks!
EDIT 1:
I made an example with only the listener middleware (only-middleware branch), and compared it with different ways of doing :
With forkApi.pause : Important leaks, especially of the generated entities
Without forkApi.pause : I use directly api.dispatch, no more leaks of the generated entities, some leaks of other kinds, but maybe normal things (I am not qualified enough to pronounce on this)
Without api.dispatch : I call directly the function that generates an entity, same result as with api.dispatch
It seems that the leak is related to forkApi.pause, but again I am not qualified enough to know the real cause...
It's probably the promises.forEach. Every 1000ms, you create a bunch of new promises and schedule things for them. You never wait for the last batch of those promises to finish, so they accumulate.
Replace the promises.forEach with a await Promise.all(promises.map and see what that does.
After reading your solution more closely, I believe you can do this with less problems by sticking more to the reducer and less to the listenerMiddleware.
I would suggest these changes:
export const entitySlice = createSlice({
name: "entity",
initialState: entityAdapter.getInitialState({ acceptingEntities: false }),
reducers: {
upsertOne: (state, action) => {
entityAdapter.upsertOne(state, action.payload);
},
removeAll: (state) => {
entityAdapter.removeAll(state);
},
load(state) { state.acceptingEntities = true },
unload(state) { state.acceptingEntities = false },
},
extraReducers: builder => {
builder.addCase(getEntity.fulfilled, (state, action) => {
if (!state.acceptingEntities) return;
const prevEntity = entitySelectors.selectById(state.entity, id);
entityAdapter.upsertOne(state,
prevEntity
? mergeEntityData(prevEntity.data, action.payload.data)
: action.payload.data
)
})
}
});
and
entityMiddleware.startListening({
actionCreator: api.actions.load,
effect: async (action, api) => {
const task = api.fork(async (forkApi) => {
while (!forkApi.signal.aborted) {
for (const id of entityIds) {
api.dispatch(getEntity(id))
}
await api.delay(1000);
}
});
await api.condition(api.actions.unload.match);
task.cancel();
},
});
Generally:
logic like calculating a new value belongs into the reducer, not outside. Doing stuff like this outside always has the risk of race conditions and in the reducer you always have all the info available (also, no risk of hogging memory by holding stale value references)
dispatching another action directly after a thunk will only add more workload - after every reducer, every selector will rerun and your UI might rerender. Just go for an extraReducer from the start
I just added a boolean acceptingEntities to indicate if updates should currently take place or not
this massively reduces complexity in your listener
It may be related to use of Promise.race(): https://github.com/nodejs/node/issues/17469 . Filed https://github.com/reduxjs/redux-toolkit/issues/3020 for us to look at this further.
Related
React application using Redux. A have a combined reducer, consisting of appStateReducer and contestReducer. Each of these two takes care of some part of the application data.
When action is performed, I want not only the respective state to be changed, but I also want to persistently save the new state, so that if the user reloads application page in the browser, the state would be preserved.
My idea is to add third reducer to take care only of save and load actions (each of the two sub-states separately).
Save and load will use IndexedDB, through localbase package. All of the db actions (add, get, update, delete) appear to be synchronous, i.e. there seems to be no real need to implement asynchronous actions. UPDATE: this is wrong, it is asynchronous, just some basic examples ignore it.
I am not sure how to handle the problem properly.
I will need a database connection object, a singleton, initialized once after page is loaded, which should be shared by all save/load actions regardless of which part of the state is to be stored or loaded. That would lead to a separate reducer working only with the db object. If I do this, the db reducer would have to have access to all the other sub-state, which is normally not the case in Redux.
Or, I could implement save and load action in each reducers separately, not a big deal, actually. But how to make the global db object accessible by the reducers?
It is as React application written in typescript and all components are implemented as classes.
You already have access to all data if you are using middleware, Example:
export const requestPost = (id) => (dispatch,getState) => {
// You can make an bank for post and check If data exist or not
const postState = getState().bank.posts.data;
const found = postState?.find((post) => post.id === id);
if (found) {
dispatch({ type: SUCCESS.POST, data: found });
} else {
dispatch({ type: REQUEST.POST });
API.get(`/post/v2?id=${id}`)
.then((res) => dispatch({ type: SUCCESS.POST, data: res.data[0] }))
.catch((err) => errorHandler(err, FAILURE.POST));
}
};
Just make and reducer for saving data on DB or somewhere and read them at the start.
I have an app that loads some images with metadata. A single folder can be quite large (~100-142Mb) once loaded into memory. Previously, we were using a plain old javascript object to manage the state of the app and everything worked fine, but i'd like to gain the benefits of ngrx's state management.
I've discovered ngrx and it seemed to be a smarter option when it comes to state management. However, when i add these items to the state, the app hangs when adding images to the store and then performance slows down when accessing individual (and unrelated) flags from the store i.e. UI flag - draw is open.
1) Here "directories" is a Map < string, Directory > () object that is saved the the Store (~100-120Mb). Directory is a complex object with many nested values. Once images are loaded, and then added to the store, it a) hangs and then b) everything else (i.e. changing a ui flag) slows down.
return {
...state,
loadedDirectories: directories,
filesLoading: false,
};
2) The directories are then later accessed from the store.
this.store
.pipe(select(fromReducer.getLoadedDirectories))
.subscribe(loadedDirectories => {
this._directoryData = loadedDirectories;
});
Selector looks like this....
export interface ImageLoaderState {
loadedDirectories: Map<string, Directory>;
filesLoading: boolean;
errorMessage: string;
}
export class AppState {
imageLoader: fromImageLoader.ImageLoaderState;
}
export const combinedReducers = {
imageLoader: fromImageLoader.imageLoaderReducer
.... More reducers here ....
}
// Select Image loader state.
export const selectImageLoaderState = (state: AppState) => state.imageLoader;
export const getLoadedDirectories = createSelector(
selectImageLoaderState,
(state: fromImageLoader.ImageLoaderState) => state.loadedDirectories
);
Using angular 8 and the following versions of ngrx.
"#ngrx/effects": "^8.4.0",
"#ngrx/store": "^8.4.0",
"#ngrx/store-devtools": "^8.4.0",
Are there any better practices? i.e. Add each image, one at a time to the store?
The ngrx store is for application state and not so good as a document store.
Please see..
https://github.com/btroncone/ngrx-store-localstorage/issues/39
One issue I see is how you create your new state. You mention that when you create your new state, you do the following
return {
...state,
loadedDirectories: directories,
filesLoading: false,
};
I think you are creating an object with tons of key-value pairs, then recreating that work when you set the loadedDirectories property again. I'm uncertain about the performance costs of using the spread operator in the context of very large objects. I would suggest you focus on creating this property once. This might help you
Does spread operator affect performance?
Let's say that two users do changes to the same document while offline, but in different sections of the document. If user 2 goes back online after user 1, will the changes made by user 1 be lost?
In my database, each row contains a JS object, and one property of this object is an array. This array is bound to a series of check-boxes on the interface. What I would like is that if two users do changes to those check-boxes, the latest change is kept for each check-box individually, based on the time the when the change was made, not the time when the syncing occurred. Is GroundDB the appropriate tool to achieve this? Is there any mean to add an event handler in which I can add some logic that would be triggered when syncing occurs, and that would take care of the merging ?
The short answer is "yes" none of the ground db versions have conflict resolution since the logic is custom depending on the behaviour of conflict resolution eg. if you want to automate or involve the user.
The old Ground DB simply relied on Meteor's conflict resolution (latest data to the server wins) I'm guessing you can see some issues with that depending on the order of when which client comes online.
Ground db II doesn't have method resume it's more or less just a way to cache data offline. It's observing on an observable source.
I guess you could create a middleware observer for GDB II - one that checks the local data before doing the update and update the client or/and call the server to update the server data. This way you would have a way to handle conflicts.
I think to remember writing some code that supported "deletedAt"/"updatedAt" for some types of conflict handling, but again a conflict handler should be custom for the most part. (opening the door for reusable conflict handlers might be useful)
Especially knowing when data is removed can be tricky if you don't "soft" delete via something like using a "deletedAt" entity.
The "rc" branch is currently grounddb-caching-2016 version "2.0.0-rc.4",
I was thinking about something like:
(mind it's not tested, written directly in SO)
// Create the grounded collection
foo = new Ground.Collection('test');
// Make it observe a source (it's aware of createdAt/updatedAt and
// removedAt entities)
foo.observeSource(bar.find());
bar.find() returns a cursor with a function observe our middleware should do the same. Let's create a createMiddleWare helper for it:
function createMiddleWare(source, middleware) {
const cursor = (typeof (source||{}).observe === 'function') ? source : source.find();
return {
observe: function(observerHandle) {
const sourceObserverHandle = cursor.observe({
added: doc => {
middleware.added.call(observerHandle, doc);
},
updated: (doc, oldDoc) => {
middleware.updated.call(observerHandle, doc, oldDoc);
},
removed: doc => {
middleware.removed.call(observerHandle, doc);
},
});
// Return stop handle
return sourceObserverHandle;
}
};
}
Usage:
foo = new Ground.Collection('test');
foo.observeSource(createMiddleware(bar.find(), {
added: function(doc) {
// just pass it through
this.added(doc);
},
updated: function(doc, oldDoc) {
const fooDoc = foo.findOne(doc._id);
// Example of a simple conflict handler:
if (fooDoc && doc.updatedAt < fooDoc.updatedAt) {
// Seems like the foo doc is newer? lets update the server...
// (we'll just use the regular bar, since thats the meteor
// collection and foo is the grounded data
bar.update(doc._id, fooDoc);
} else {
// pass through
this.updated(doc, oldDoc);
}
},
removed: function(doc) {
// again just pass through for now
this.removed(doc);
}
}));
Let's pretend I have a long-running function working on computing my new state.
Meanwhile another action comes in and changes the state while the first one did not finish and is working on stuff.
If I am imagining things correctly there is no actions queue and the state might be resolved in some unpredictable manner.
Should I be worried about this at all?
I don't mean real threads, just a concept for the lack of better wording. Actions are asynchronous and state keys are being accessed by reference.
I was concerned about the same thing so I just did some digging. It looks like two threads concurrently calling dispatch() (if it were possible) could raise an exception. But it shouldn't be possible and that error message points to a particular, different cause. The "actions queue" is in the browser's own event loop. That event loop runs async/interaction callbacks (from which we call dispatch()) one-at-a-time.
That's the responsibility of your own action creators and your own reducers, and heavily related to how you structure your actions and reducers conceptually. The Redux FAQ question on structuring "business logic" is very relevant here:Redux FAQ
Thunk action creators have access to getState, so it's very common to have a thunk check the current state and only dispatch under certain conditions, such as this example:
// An example of conditional dispatching based on state
const MAX_TODOS = 5;
function addTodosIfAllowed(todoText) {
return (dispatch, getState) => {
const state = getState();
if(state.todos.length < MAX_TODOS) {
dispatch({type : "ADD_TODO", text : todoText});
}
}
}
Your reducer can also have sanity checks as well:
function todosReducer(state, action) {
switch(action.type) {
case "ADD_TODO": {
if(state.todos.length >= state.maxTodos) {
return state;
}
return {
...state,
todos : state.todos.concat(action.newTodo)
}
}
default : return state;
}
}
Personally, I don't like to have my reducers just blindly merge in whatever data's in the action, unless it's very small (like, say, the name of the currently selected tab or something). I prefer to have a reasonable amount of logic in my action creator to set up the action, a minimal-ish amount of data included in the action itself, and a sufficiently smart reducer to do the work based on that action.
I'm building an app where actions are performed as the user scrolls down. It would be nice if I could undo those actions as the user scrolls up again, basically turning scrolling into a way to browse through the time line of actions.
Is there a built-in way in Redux to do this? Or would I have to write middleware for this?
Is there a built-in way in Redux to do this? Or would I have to write middleware for this?
Middleware sounds like the wrong idea in this case because this is purely state management concern. Instead you can write a function that takes a reducer and returns a reducer, “enhancing” it with action history tracking along the way.
I outlined this approach in this answer, and it's similar to how redux-undo works, except that instead of storing the state, you can store actions. (Depends on the tradeoffs you want to make, and whether it's important to be able to “cancel” actions in a different order than they happened.)
I believe the idea is not so much "undo" as much as save a reference to the entire state tree each time an action passes through redux.
You would have a history stack made up of the application state at various times.
let history = [state1, state2, state3]
// some action happens
let history = [state1, state2, state3, state4]
// some action happens
let history = [state1, state2, state3, state4, state5]
// undo an action
let history = [state1, state2, state3, state4]
state = state4
To "undo" an action, you just replace the application state with one of the saved states.
This can be made efficient with data structures that support structural sharing, but in development we don't really need to consider resource constraints too much anyway.
I also wanted to create a simple undo functionality, but had already shipped an app with redux-storage that serializes and loads the state for every user. So to keep it backwards-compatible, I couldn't use any solution that wraps my state keys, like redux-undo does with past: [] and present:.
Looking for an alternative, Dan's tutorial inspired me to override combineReducers. Now I have one part of the state: history that saves up to 10 copies of the rest of the state and pops them on the UNDO action. Here's the code, this might work for your case too:
function shouldSaveUndo(action){
const blacklist = ['##INIT', 'REDUX_STORAGE_SAVE', 'REDUX_STORAGE_LOAD', 'UNDO'];
return !blacklist.includes(action.type);
}
function combineReducers(reducers){
return (state = {}, action) => {
if (action.type == "UNDO" && state.history.length > 0){
// Load previous state and pop the history
return {
...Object.keys(reducers).reduce((stateKeys, key) => {
stateKeys[key] = state.history[0][key];
return stateKeys;
}, {}),
history: state.history.slice(1)
}
} else {
// Save a new undo unless the action is blacklisted
const newHistory = shouldSaveUndo(action) ?
[{
...Object.keys(reducers).reduce((stateKeys, key) => {
stateKeys[key] = state[key];
return stateKeys;
}, {})
}] : undefined;
return {
// Calculate the next state
...Object.keys(reducers).reduce((stateKeys, key) => {
stateKeys[key] = reducers[key](state[key], action);
return stateKeys;
}, {}),
history: [
...(newHistory || []),
...(state.history || [])
].slice(0, 10)
};
}
};
}
export default combineReducers({
reducerOne,
reducerTwo,
reducerThree
});
For me, this works like a charm, it just doesn't look very pretty. I'd be happy for any feedback if this is a good / bad idea and why ;-)
There's no built-in way to do this.
but you can get inspired by how redux-dev-tools works (https://github.com/gaearon/redux-devtools). It basically have "time travel" functionality and it work by keep a track of all actions and reevaluating them each time. So you can navigate easily thorough all your changes.