What is a vector? Why would I use it rather anything else?
A vector is a dynamically sized array. Use it whenever you need an array of elements that are stored contiguously in memory but you do not know the array's size at compile-time, only at runtime.
Related
I am going to create a list of string, for example, ["a", "10", "p20", ...]. Which way is more efficient, using string, vector, QVector, QLinkedList or others?
The efficiency depends on how you're going to use them. For instance, if you're inserting into the middle, QStringList (which is QList ) handles insertions very well. If you know the exact size up front and it isn't going to change, then QVector might be a better choice. Read up on the various possibilities and then use the one that's appropriate for your needs.
The QVector documentation provides a nice guideline for comparing the different Qt-based possibilities.
I am looking for an Array-like type with the following properties:
stores elements on disk
elements can have composite type
elements are read into memory, not the whole array
it is possible to write individual elements without writing the whole array
supports setindex!, getindex, push!, pop!, shift!, unshift! and maybe vcat
is reasonably efficient
So far I have found the following leads:
https://docs.julialang.org/en/latest/stdlib/SharedArrays/
http://juliadb.org
https://github.com/JuliaIO/JLD.jl
The first one seems promising, but it seems the type of the elements has to be isbits (meaning a simple number, some structs but not, e.g., an Array{Float64,1}). And it's not clear if the whole array contents are loaded into memory.
If it does not exist yet, I will of course try to construct it myself.
NCDatasets.jl addresses part of the requirements:
stores elements on disk: yes
elements can have composite type: no (although some support for composite type is in NetCDF4, but not yet in NCDatasets.jl). Currently you can have only Arrays of basic types and Arrays of Vectors (of basic types).
elements are read into memory, not the whole array: yes
it is possible to write individual elements without writing the whole array supports setindex!, getindex, push!, pop!, shift!, unshift! and maybe vcat: just setindex!, getindex
is reasonably efficient: the efficency is reasonable for me :-)
The project making it yourself sounds very interesting. I think it would server certainly a gap in the current ecosystem.
Some storage technologies that might be good to have a look at are:
HDF5 (for storage, cross-platform and cross-language)
JLD2 (successor of JLD) https://github.com/simonster/JLD2.jl
rasdaman (a "database" for arrays) http://www.rasdaman.org/
possibly also BSON http://bsonspec.org/
Maybe you can also reach out to the JuliaIO group.
I am on several lines codes from a Qt project to appending into a Qvector every 1s. I noticed that in STL deque could have a better performance in adding a new element into the end that vector. What's the equivalent or similar Qt way? Cause I don't find any in Qt libraries.
Julio
There is no direct equivalent to the std::deque class in QT.
However, your best bet is to use QList.
Here is what the documentation says about QT container classes:
For most purposes, QList is the right class to use. Its index-based API is more convenient than QLinkedList's iterator-based API, and it is usually faster than QVector because of the way it stores its items in memory. It also expands to less code in your executable.
Anyways, if you are only appending items once every second, there will not be much impact to choose one over the other.
There is no need to have a Qt equivalent for every std container, you can use std::deque if that is what you are after.
Anyway, note that for the case when you do a lot of insertions at the end of the vector both std::vector and QVector have a member function named reserve (see the links) that can be used to pre-allocates a bigger buffer and make insertions at the end faster.
I'm trying to get a reference to the memory of a concurrent_vector in TBB
(Threaded Building Blocks) in a way similar to std::vector.
So an std::vector would be accessed like: &stdVector[0].
But the equivalent for a concurrent_vector doesn't work: &tbbVector[0].
I guess this may have something to do with how the memory is stored internally
in order to be concurrent, but is there a way to do this?
unlike std::vector, concurrent_vector does not provide a guarantee of contiguous storage. So taking the address of the first element and doing anything other than accessing the first element is not a good idea.
There are lots of real-world reasons you'd want to do this. Ours is because we have a list of variable length data structures, and we want to be able to change the size of one of the elements without recopying them all.
Here's a few things I've tried:
Just have a lot of kernel arguments. Sure, sounds hacky, but works for small N. This is actually what we've been doing.
Do 1) with some sort of macro loop which extends the kernel args to the max size (which I think is device dependent). I don't really want to do this... it sounds bad.
Create some sort of list of structs which contain pointers, and fill it before your kernel invocation. I tried this, and I think it violates the spec. According to what I've seen on the nVidia forums, preserving the address of a device pointer beyond one kernel invocation is illegal. If anyone can point to where in the spec it says this, I'd love to know, because I can't find it. However, this definitely breaks on ATI hardware, as it moves the objects around.
Give up, store the variable sized objects in a big array, and write a clever algorithm to use empty space so the whole array must be reflowed less often. This will work, but is an inelegant, complicated design. Also, it requires lots of scary pointer arithmetic...
Does anyone else have other ideas? What about experiences trying to do this; is there a least hacky way? Why?
To 3:
OpenCL 1.1 spec page 193 says "Arguments to kernel functions in a program cannot be declared as a pointer to a pointer(s)."
Struct containing a pointer to pointer (pointer to a buffer object) might not be against strict reading of this sentence but it's within the spirit: No pointers to buffer objects may be passed as arguments from host code to kernel even if they're hidden inside a user defined struct.
I'd opt for option 5: Do not use variable size data structures. If you have any way of making them constant size by all means do it. It will make your life a whole lot easier. To be precise there is no 'variable size structure'. Every struct definition produces constant sized structs, so if the size has changed then the struct itself has changed and therefore requires another mem object. Every pointer passed to kernel function must have a single type.
In addition to sharpnelis answer option 5:
If the objects have similar size you could use unions on the biggest possible object size. But make sure you use explicit alignment. Pass a second buffer identifying the union used in each object in your variable-sized-objects-in-static-size-union buffer.
I reverted to this when using opencl lib code that only allowed one variable array of arbitrary type. I simply used cl_float2 to pass two floats. Since the cl_floatN types are implemented as unions - what works for the build in types will work for you as well.