I wonder if there is a way to observe changes in the whole graph, instead of subscribing to changes on one particular node. I was not able to find an answer reading the Docs/Howtos at gun.eco/docs
Lets say you build a real-time mind-mapping application, so basically a graph/tree structure.
If I add a new node - at some place - to the graph I want to update my UI
If I remove a node or a whole subtree ...
The 2. scenario is a a general concern:
How can I delete multiple nodes together with all related edges ?
Copied from conversation with Gun Community:
First Answer:
To answer the 1st, You could have an index node that you subscribe to unsing gun.get(node).on(callback, changesOnlyFlag). New nodes would trigger the update function, where you will check what that node might be related to in your application.
To answer the 2nd, Delete in a decentralized system is hard. (Google tombstone problem) In Gun deletes are handled by putting null to an object, which cuts all edges from that item and it becomes unreachable from a traversal standpoint. (Although you can still get the children nodes by their soul (UUID of node), or via the index node that you might add all children too, by default)
Second Answer:
https://gun.eco/docs/API#open describes an additional module you can require to open the whole graph. Which can be used to track changes, with slight modifications to the code.
Related
Let assume we got two long lived process managers. Both sagas operates over 10 milion items for example. First saga adds something to each item. Second saga removes it from each item. Given both process managers need few minutes to complete its job if I run them simultaneously I get into troubles.
Part of those items would hold the value while rest of them not. The result is close to random actually and depends on command order that affect particular item. I wondered if redispatching "Remove" command in case of failure would solve the problem. I mean if you try remove non existing value you should wait for the first saga to add the value. But while process managers are working someone else may dispatch "Remove" or "Add" command. In such case my approach would fail.
How may I solve such problem? :)
It seems that you would want the second saga to not run if the first saga is running (and presumably not run until some process which depends on whatever the first saga added being there). So the apparent solution would be to have a component (could be a microservice, could also be a record in a strongly consistent datastore like zookeeper/etcd/consul) that gives permission for the sagas to start executing. An example protocol might look like:
Saga sends a message to the component identifying the saga and conveying the intention to start
Component validates that no sagas might be running which would prevent this saga from running
Component responds with permission to start running
Subsequent saga attempts result in rejection until the running saga tells the component it's OK to run the other saga
Assuming that this component is reliably durable, the failure mode to worry about is that permission is granted but this component never processes the message that the saga finished (causes of this could include the permission message not getting delivered/processed or the saga crashing). No amount of acknowledgements or extra messages can solve this (it's basically the Two Generals' Problem).
A mitigation is to have this component (or something watching this component) alert if it seems that too much time has passed without saga completion. Whatever/whoever is responsible for ensuring liveness would then investigate to see if the saga is still running and if none is running, inform the component that it's OK to run the other saga. Note that this is not foolproof: it's quite possible for the decider in question to make what turns out to be the wrong decision.
I feel like I need more context. Whilst you don't say it explicitly, is the problem that the second saga tries to remove values that haven't been added by the first?
If that was true, a simple solution would be to just use a third state.
What I mean by that is to just more explicitly define and declare item state. You currently seem to have two states with value, and without value, but nothing to indicate if an item is ready to be processed by the second saga because the first saga has already done it's work on the item in question.
So all that needs to happen is that the second saga keeps looking for items where:
(with_value == true & ready_for_saga2 == true)
Ready_for_saga2 or "Saga 1 processing complete", whatever seems more appropriate in your context.
I'd say that the solution would vary based on which actual problem, we're trying to solve.
Say it's an inventory and add are items added to the inventory and remove are items requested for delivery. Then the order of commands does not matter that much because you could just process the request for delivery, when new items are added to the inventory.
This would lead to an aggregate root with two collections: Items and PendingOrders.
One process manager adds new inventory to Items - if any orders are pending, it will complete these orders in the same transaction and remove both the item and the order from the collections.
If the other process manager adds an order (tries to remove an item), it will either do it right away, if there's any items left - or it will add the order to the pending orders to be processed when new items arrive (and maybe notify someone about the delay, while we're at it).
This way we end up with the same state regardless of the order of commands, but the actual real-world-problem has great influence on the model chosen.
If we have other real world problems, we can make a model those too.
Let's say you have two users that each starts a process that bulk updates titles on inventory items. In this case you - and the users - have to decide how best to resolve this conflict - what will lead to the best real world outcome.
If you want consistency across all the items - all or no items should be updated by a single bulk update - I would embed this knowledge in a new model. Let's call it UpdateTitlesProcesses. We have only one instance of this model in the system. The state is shared between processes. This model is effectually a command queue, and when a user initiates the bulk operation, it adds all the commands to the queue and starts processing each item one at a time.
When the second user initiates another title update, the business logic in our models will reject this, as there's already another update started. Or if the experts say that the last write should win, then we ditch the remaining commands from the first process and add the new ones (and similarly we should decide what should happen if a user issues a single title update, not bulk - should it be rejected, prioritized or put on hold?).
So in short I'd say:
Make it clear which real world problem we are solving - and thus which conflict resolution outcome is best (probably a trade off, often also something that requires user interaction or notification).
Model this explicitly (where processes, actions and conflict handling are also part of the model).
How to clone a graph if:
nodes are not labeled
it's only possible to move between nodes through edges
you can't mark or any other way use nodes of original graph
More precise definition of problem:
You are navigating through graph.
At a time you see only one node and it's links. For example as simple as single number (number of links).
Graph is non-directed with unique links between each connected pair of nodes.
To move to another node you just pass a link index into step API call and get next node (number of links in next node). Function step handles moves in a way like list of links is sorted same way every time you enter the node. E.g. if you are in node A connected with node B then passing some constant number i_A_B into step always get you into node B.
At start you are in special start node that has only one connection, then you can only pass "0" into and get into start node. This is an equivalent to call special start API call without parameters and get into start node.
Is there any and what is the well-known name of this problem?
What is the algorithm that can make a clone of graph without modifying (marking, labeling, coloring) the original one?
Creating nodes during navigation and connect them to previous node is trivial.
The main difficulty then obviously would be to match already visited nodes with already created.
What minimal additional information is required?
Is marked initial node is enough?
For example without mark any cycle graph will look equivalent through this API. Also any graph with same regular structure but different size.
What types of graphs can be recognized (cloned) without marked start node?
Upd1: API provides no way to compare nodes or edges of original graph.
Upd2: This problem looks like creation of a map of a maze while walking inside it.
Or like replication of a state machine that is encapsulated in black box and provides only oracle that returns a set of acceptable inputs on call with current acceptable input.
I struggle finding the right way to mutate my state in an ngrx application as the state is rather complex and depending on many factors. This Question is not about doing one piece of code correct but more about how to design such a software in general, what are doe's and don'ts when finding some hacky solutions and workarounds.
The app 'evolved' by time and i wan't to share this process in an abstracted way to make my point clear:
Stage 1
State contains Entities. Those represent nodes in a tree and are linked by ids. Modifying or adding an entity requires a check about the type of nodes the new/modified ones should be connected with. Also it might be that upon modifying a node, other nodes had to be updated.
The solution was: create functions that do the job. Call them right in the reducer so everything is always up to date and synchronus when used (there are services that might modify state).
Stage 2
A configuration is added to the state having an impact on the way the automatically modifyed nodes are modifyed/created. This configuration is saved in it's own state right under the root state.
The solution:
1) Modify the actions to also take the required data from the configuration.
2) Modify the places where the actions are created/dispatched (add some ugly
this.state.select(fromRoot.getX)
.first()
.(subscribe(element => {this.state.dispatch(new Action({...old_payload, newPayload: element}))})
wrapper around the dispatch-calls)
3) modify the functions doing the node-modification and
4) adding the argument-passing to the function calls inside the reducer
Stage 3
Now i'am asked to again add another configuration to the process, also retrived by the backend and also saved in another state right under the root state
State now looks like:
root
|__nodes
|__config_1
|__config_2
i was just about to repeat the steps as in stage 2 but the actions get really ig with all the data passed in and functions have to carry around a lot of data. This seems to be wrong, when i actually dispatch the action on the state containing all the needed info.
How can i handle this correct?
Some ideasi already had:
use Effects: they are able to get everything from state they need and can create everything - so i only need to dispatch an action with only the actions payload, the effect then can grab everything from the state it needs. I don't like this idea because it triggers asynchronus tasks to modify the state and add not-state-changing actions.
use a service: with a service holding state it would be much like with effects but without using actions to just create asynchronus calls which then dispatch the actions that relly change state.
do all the stuffi n the component: at the moment the components are kept pretty simple when it comes to changing state as i prefer the idea that actions carry as little data as possible, since reducers can access the state to get theyr data - but this is where the problem occus, this time i can't get hands on the data i need.
I have several graphs. The breadth and depth of each graph can vary and will undergo changes and alterations during runtime. See example graph.
There is a root node to get a hold on the whole graph (i.e. tree). A node can have several children and each child serves a special purpose. Furthermore a node can access all its direct children in order to retrieve certain informations. On the other hand a child node may not be aware of its own parent node, nor other siblings. Nothing spectacular so far.
Storing each graph and updating it with an object database (in this case DB4O) looks pretty straightforward. I could have used a relational database to accomplish data persistence (including database triggers, etc.) but I wanted to realize it with an object database instead.
There is one peculiar thing with my graphs. See another example graph.
To properly perform calculations some nodes require informations from other nodes. These other nodes may be siblings, children/grandchildren or related in some other kind. In this case a specific node knows the other relevant nodes as well (and thus can get the required informations directly from them). For the sake of simplicity the first image didn't show all potential connections.
If one node has a change of state (e.g. triggered by an internal timer or triggered by some other node) it will inform other nodes (interested obsevers, see also observer pattern) about the change. Each informed node will then take appropriate actions to update its own state (and in turn inform other observers as needed). A root node will not know about every change that occurs, since only the involved nodes will know that something has changed. If such a chain of events is triggered by the root node then of course it's not much of an issue.
The aim is to assure data persistence with an object database. Data in memory should be in sync with data stored within the database. What adds to the complexity is the fact that the graphs don't consist of simple (and stupid) data nodes, but that lots of functionality is integrated in each node (i.e. events that trigger state changes throughout a graph).
I have several rough ideas on how to cope with the presented issue (e.g. (1) stronger separation of data and functionality or (2) stronger integration of the database or (3) set an arbitrary time interval to update data and accept that data may be out of synch for a period of time). I'm looking for some more input and options concerning such a key issue (which will definitely leave significant footprints on a concrete implementation).
(edited)
There is another aspect I forgot to mention. A graph should not reside all the time in memory. Graphs that are not needed will be only present in the database and thus put in a state of suspension. This is another issue which needs consideration. While in suspension the update mechanisms will probably be put to sleep as well and this is not intended.
In the case of db4o check out "transparent activation" to automatically load objects on demand as you traverse the graph (this way the graph doesn't have to be all in memory) and check out "transparent persistence" to allow each node to persist itself after a state change.
http://www.gamlor.info/wordpress/2009/12/db4o-transparent-persistence/
Moreover you can use db4o "callbacks" to trigger custom behavior during db4o operations.
HTH
German
What's the exact question? Here a few comments:
As #German already mentioned: For complex object graphs you probably want to use transparent persistence.
Also as #German mentione: Callback can help you to do additional stuff when objects are read/written etc on the database.
To the Observer-Pattern. Are you on .NET or Java? Usually you don't want to store the observers in the database, since the observers are usually some parts of your business-logic, GUI etc. On .NET events are automatically not stored. On Java make sure that you mark the field holding the observer-references as transient.
In case you actually want to store observers, for example because they are just other elements in your object-graph. On .NET, you cannot store delegates / closures. So you need to introduce a interface for calling the observer. On Java: Often we use anonymous inner classes as listener: While db4o can store those, I would NOT recommend that. Because a anonymous inner class gets generated name which can change. Then db4o will not find that class later if you've changed your code.
Thats it. Ask more detailed questions if you want to know more.
What I'm trying to do is have a 3 state tree expansion.
I have three different icons for "expand" "collapse" "semi-expanded" which I want to use to show a partially populated tree control with all nodes initialized to semi-expanded state and then on clicking the "semi-expanded" icon it gets data from server and populates the tree and open that branch with "expanded" icon.
I tried looking for it but couldn't find anything close to it except the 3-state checkbox but don't know how to use it on 3 state icon when tree would only maintain 2 states.
Thanks in advance.
I think what you are looking for is called a lazy-loading tree. There are lots of examples your can google for, but here is a great example.
As far as the visual part of your request goes (3 different icons to show that state of the branch or node) - you could easily handle that with a custom renderer, by looking at a flag on the node for it's load status.
Does that help?
A 3-State tree control is a bit uncommon and might therefore be a bit confusing; consider that even simple 3-state checkboxes are relatively rare and users may not be accustomed to them. Maybe that's why you didn't find such a tree control.
Thus, maybe you should consider using an alternate design that doesn't require 3-state controls.
For example, the node could start in collapsed node. If the user expands it, and there is no data yet, show a single sub node with the text "retrieving data..." (and a progress wheel or other progress indicator, if you can) and start data retrieval. When the data arrives, replace this sub node with the actual data.