I've messed up my views a bit (big surprise for CC) and I have a child stream that has most of what I need, but not all. My parent stream needs to be updated, but I can't because of some issues (maybe evil twins I dunno).
Is it possible/wise to do the following
1) clear all elements in the parent stream
2) use clearfsimport to perform mass update on child stream
3) deliver child stream to parent
This is of course dependent on the fact that child stream elements are not deleted when deleted from parent.
Should I just clear out all elements of both views and start over? Any suggestions would be appreciated.
Yes, you can do a clearfsimport from whatever source you want to the child stream.
But I wouldn't recommend "clearing" (as in "rmnam'ing") all elements from the parent stream, even though it doesn't rmname them in the child stream, as my answer to your previous question details.
If you have a valid source (ie some directory with every file you need), you can clearfsimport it to your child stream view, in order to be complete.
Then try the deliver and identify the potential evil twins: your deliver will stop quickly at the "directory merge" stage, asking you to choose between two (identically named) files: you will chose the one coming from the child stream.
All the other files present in both stream will see their history updated as expected by that deliver.
Related
I'm using a QTreeView/QFileSystemModel pairing to display directory contents. I'm loading a large directory, with tens of thousands of files and subdirectories in it. When I issue an expandAll() call, the tree takes a long time to fully open - on the order of 30 seconds or so. I'd like to put up a "Please Wait..." dialog until the tree is completely expanded.
The expandAll() function, though, returns almost immediately. I am looking for a signal that's emitted when the tree is fully expanded, but there doesn't seem to be one. The 'expanded' signal is emitted only for a specific index, and I can't tell which index I should use to indicate complete expansion.
I'm starting to use RWMutex in my Go project with map since now I have more than one routine running at the same time and while making all of the changes for that a doubt came to my mind.
The thing is that I know that we must use RLock when only reading to allow other routines to do the same task and Lock when writing to full-block the map. But what are we supposed to do when editing a previously created element in the map?
For example... Let's say I have a map[int]string where I do Lock, put inside "hello " and then Unlock. What if I want to add "world" to it? Should I do Lock or can I do RLock?
You should approach the problem from another angle.
A simple rule of thumb you seem to understand just fine is
You need to protect the map from concurrent accesses when at least one of them is a modification.
Now the real question is what constitutes a modification of a map.
To answer it properly, it helps to notice that values stored in maps are not addressable — by design.
This was engineered that way simply due to the fact maps internally have intricate implementation which
might move values they contain in memory
to provide (amortized) fast access time
when the map's structure changes due to insertions and/or deletions of its elements.
The fact map values are not addressable means you can not do
something like
m := make(map[int]string)
m[42] = "hello"
go mutate(&m[42]) // take a single element and go modifying it...
// ...while other parts of the program change _other_ values
m[123] = "blah blah"
The reason you are not allowed to do this is the
insertion operation m[123] = ... might trigger moving
the storage of the map's element around, and that might
involve moving the storage of the element keyed by 42
to some other place in memory — pulling the rug
from under the feet of the goroutine
running the mutate function.
So, in Go, maps really only support three operations:
Insert — or replace — an element;
Read an element;
Delete an element.
You cannot modify an element "in place" — you can only
go in three steps:
Read the element;
Modify the variable containing the (read) copy;
Replace the element by the modified copy.
As you can now see, the steps (1) and (3) are mere map accesses,
and so the answer to your question is (hopefully) apparent:
the step (1) shall be done under at least an read lock,
and the step (3) shall be done under a write (exclusive) lock.
In contrast, elements of other compound types —
arrays (and slices) and fields of struct types —
do not have the restriction maps have: provided the storage
of the "enclosing" variable is not relocated, it is fine to
change its different elements concurrently by different goroutines.
Since the only way to change the value associated with the key in the map is to reassign the changed value to the same key, that is a write / modification, so you have to obtain the write lock–simply using the read lock will not be sufficient.
Simplified structure of what I have is:
some single roots
each root has many (e.g. 100s) of children
User may update the root information, and no other operation on children should be allowed (because root change may affect all of them).
Also, user may operate on children (if root is not in use, of course). For example, user may change 2 children in the same time, and this is allowed, since each children is independent.
I need locks in this structure in order to be sure there are no corruptions:
when children is in use, lock the children. This will not allow two operations on the same children in same time.
when root is in use, lock root AND all the children. This will forbid the operations on any children while root is updated.
What bothers me here is the need to lock all the children - in a distributed system that means sending that many requests to distributed lock.
Is there any better solution I don't see?
You're missing two things. First, it's safe for multiple threads to read from a node at the same time, as long as nobody is writing to it. Second, the child nodes can be viewed as their own roots of smaller trees, so the same algorithm/solution may be applied to all nodes except leaf nodes. The first one is most important. Here's how you could do this:
Use a read/write mutex on all nodes in the tree. This allows any number of processes to concurrently read, or a single process to write to a node at any time.
To read:
read-lock node and all parents all the way up to root.
read.
release all read-locks.
To write:
write-lock the least-upper-bound of the nodes you want to modify. If you're modifying a node (and possibly any of its children), write-lock that node.
do your modifications
release the write-lock
This means two siblings may be modified concurrently, and that any number of reads may execute concurrently. However, the cost of reading is that you need to grab O(log100(tree_height)) read-locks, for a tree with roughly 100 children at each level. It's unlikely to be a real problem, unless your tree is huge, with extremely many reads and writes to the same leaf node.
This assumes that no child may modify its parent.
I use a QAbstractItemModel to represent a tree model (of up to a few hounded items). The data itself is dynamic, at any time nodes may appear or disappear, values (or other roles) may change.
Making changes to the model is easy; I am wondering how to efficiently emit the signals in order to notify a QTreeView of the changes (most of it's nodes are collapsed).
At any given time multiple changes may occur simultaneously (row insertions and/or deletions).
Using beginInsertRows / endInsertRows / beginRemoveRows / endRemoveRows - shouldn't there be a method to notify the view of multiple changes?
In terms of performance, what would be the best strategy? For example, starting at the leaves and going up to the root / for each node - bottom to top (vs top to botton) / deletions before insertions / etc.
Would beginResetModel / endResetModel necessarily be less efficient?
Is there any advantage for using QStandardItemModel? (for this specific case).
Yes. The method of notifying everyone of disjoint removals/additions is to emit multiple signals. It would, in most cases, cause more overhead to pass some complex data structure instead of just the parent index and delimiting row/column indices.
You should only notify about the removal/addition of an item closes to the root. It makes no sense to notify about removal of children if their parent is subsequently to vanish. The notification about the parent means that the children, obviously, aren't there anymore.
It's not only about efficiency, but also about state. A model reset resets the view state. The view, upon receiving a reset, can't but assume that it got an entirely new, unrelated model - so you lose selections, expanded/collapsed state, etc. There's no way for a view to act any other way in face of a reset. Doing otherwise would require a view to keep its own copy of the model's contents.
Since a model reset implies relayout of all of the items, and it can be a very expensive thing to do, you should only do it if, in aggregate, more than 50% of the original items are changed (removed/replaced/added).
No, there is no advantage, and unless you store your data as variants, using QStandardItemModel will always end up with larger memory overhead. It is a convenience class that makes sense if it fits your needs exactly. In fact, if you're not careful about how you use it, it will work worse.
For example, if you remove an item by iterating depth-first and removing the farthest children first, then the QStandardItemModel can't foresee the future - namely, that you really want to remove the common ancestor of all of those children, and will emit a lot of change events unnecessarily. You can deal with it properly in your own model, or if you simply remove the common parent without touching the children - as they will be implicitly removed, too.
It is my understanding that it is the erases that wear out SSDs, not the writes themselves. Therefore, optimizing away the need for erases would be hugely beneficial from the point of view of drive manufacturers. Can I take it as a given that they do this?
I'd like you to assume that I'm writing directly to the disk and that there isn't a filesystem to mess things up.
If there are empty pages in a block and the SSD wants to write to those pages it will not erase the block first. A SSD will only erase a block before a write if it cannot find any empty pages to write to because doing a full read-erase-write is very slow.
Besides, the wear-out from writing and erasing is about the same. Both involve pulling electrons through the oxide layer, just in different direction.
Also, the erased state for NAND is all 1. You then write 1 to 0. You need to erase the 0 to get it back to a 1.
Unless I'm reading your question wrong I think you misunderstand how SSDs work.
SSDs are made up of large blocks (usually 512k), which are much larger than we are used to in a filesystem (usually 4k).
The Erase pass is necessary before anything can be written to the block unless the block is already empty.
So the problem with erases wearing out the disk is that if 4k of your 512k block is used, you must erase the whole 512k block and write the original 4k + anything else you are adding. This creates excessive wear and slows things down as instead of one "write" you need a "read-wipe-write" (known as "write amplification").
This is simplifying it a bit as the drive firmware does a lot of clever things to try and make sure the blocks are optimally filled e.g. it tries to keep lots of empty blocks to avoid slow writes.
Hope that helps/didn't confuse things further!
In the case that the SSD are using read-erase-write, it first read the content of the block, then erase it and write the new values. So it erase the entire block before writing, but it has saved the content for the next write operation. in some SSDs which are not read-erase-write, when you are writing, it will write on the new page (may be on the same block, then invalid the previous page). In this case, it erase the block only after making sure that every pages in the block are invalid or has been copied to another place.