I have a video stream as describe in Qt Video Overview, using the MyVideoProducer mechanics. The source images are analyzed and I have a list of connected components (x,y,width,height) and I want overlay rectangles on the video.
Can I do this by sending a list of rectangle co-ordinates to QML and have it place the rectangles or do I need to create my own overlay images?
I looked at the QtQuick particle system but it doesn't seem to fit. Other questions have the layout of the rectangle managed by Qt/Qml, but I need the rectangle to be placed according to the co-ordinates that the vision pipeline has determined in C++ and sent to the QML front-end. They will be stale/related to the video frames.
There is an example, but the overlay is unrelated to the video. I think I need an overlay that is synced to the onNewVideoContentReceived(). QML won't be able to determine how to keep any list of rectangle in sync with the video easily.
I just modified the original buffer creation, debayered from a camera, to draw the rectangles myself in the RGBA format. It avoids the synchronization issue of the video frame with the object location data. I did not use alpha but just replacement of pixels. For my content, the amount of boxes versus the video area was not great. With alpha rectangles and a lot of objects, it may be more efficient to involve a GPU. In fact, you could used fixed size squares and not the CCL bounded region and this might be significantly faster with a GPU.
A QML solution would be more elegant, but this solution works.
Alternative options are QVideoFrame::setMetaData, this can tie the CCL QRect list to the frame, so that the association is clear and tied to the frame. The method onNewVideoContentReceived() of the MyVideoProducer could render the rectangles from C++.
Another option is QAbstractVideoFilter, which will modify the original buffer to add additional data to the images presented. This is easy to enable/disable via the QML front end.
All solutions rely on C++ so it is not easy to change coloring, etc in QML. For example if the object has a recognized property such as 'male', 'female', 'cat', 'vehicle', etc the QML could update the highlighting appropriately and maintain an accounting of the object types.
Related
I have a QAbstractListModel with custom objects as items. Each object has a QImage that is loaded from a database. I use ListView in QML to visualize it but I do not see any mean to represent QImage in the delegate. Image primitive seems to accept only URLs.
Is the only way for me to show QImages is to create a QQuickImageProvider with some custom system of a URL per element (looks like a total overkill)?
I think QQuickImageProvider is the proper way.
Also, I think you can use the word 'overkill' if you know exactly how the Qt internals work. Otherwise it's just guessing.
AFAIK there is a complex caching system of images (and other data) underneath, so once an image pixmap is loaded (and doesn't change) data retrieval is immediate. So no overkill at all, since in any case at some point you need to load those QImage, but just once.
I believe a QQuickImageProvider provides pointers to the cached data, and not the whole rasterized data every time. Moreover blitting operations are nowadays performed with hardware accelerations, so it's a single operation taking a fraction of millisecond.
In other words you end up having:
give me image with url "image://xyz"
Qt looks up in the cache and returns the data pointer or performs a full load of the image if not found
the QML renderer passes the data array to OpenGL
one single blit operation (microseconds) and you have it on screen
A QML ShaderEffect will bind a QImage to a GLSL Sampler2D. See the list of "how properties are mapped to GLSL uniform variables" in the ShaderEffect docs. Needs a few lines of GLSL writing in the ShaderEffect to pass pixels through, (will update this post with an example sometime, when I have access to my code), but the result is a fast, QSG-friendly all-QML element which can render QImages.
I am trying to render geographical data obtained at different time with different sensors. Currently, I manage (through OpenGL and QOpenGL widget) to render a single image (i.e. all vertices have a z=0 coordinates). However, I am wondering how to add new "images" (still with different vertices and texture) which can overlap (in the same plane z=0) the others.
Sample from each texture in your fragment shader doing whatever composing you need, such as additive, though for geospatial data its probably more complex than that.
If using a library that does all that, then simply disable depth testing, and render each layer, adjusting transparency function between passes.
I am trying to implemente a CoverFlow like effect using a QGLWidget, the problem is the texture loading process.
I have a worker (QThread) for loading images from disk, and the main thread checks for new loaded images, if it finds any then uses bindTexture for loading them into QGLContext. While the texture is being bound, the main thread is blocked, so I have a fps drop.
What is the right way to do this?
I have found that the default behaviour of bindTexture in Qt4 is extremelly slow:
bindTexture(image,target,format,LinearFilteringBindOption | InvertedYBindOption | MipmapBindOption)
using only the LinearFilteringBindOption in the binding options speeds up the things a lot, this is my current call:
bindTexture(image, GL_TEXTURE_2D,GL_RGBA,QGLContext::LinearFilteringBindOption);
more info here : load time for a 3800x2850 bmp file reduced from 2 seconds to 34 milliseconds
Of course, if you need mipmapping, this is not the solution. In this case, I think that the way to go is Pixel Buffer Objects.
Binding in the main thread (single QGLWidget solution):
decide on maximum texture size. You could decide it based on maximum possible widget size for example. Say you know that the widget can be at most (approximately) 800x600 pixels and the largest cover visible has 30 pixels margins up and down and 1:2 aspect ratio -> 600-2*30 = 540 -> maximum size of the cover is 270x540, e.g. stored in m_maxCoverSize.
scale the incoming images to that size in the loader thread. It doesn't make sense to bind larger textures and the larger it is, the longer it'll take to upload to the graphics card. Use QImage::scaled(m_maxCoverSize, Qt::KeepAspectRatio) to scale loaded image and pass it to the main thread.
limit the number of textures or better time spent binding them per frame. I.e. remember the time at which you started binding textures (e.g. QTime bindStartTime;) and after binding each texture do:
if (bindStartTime.elapsed() > BIND_TIME_LIMIT)
break;
BIND_TIME_LIMIT would depend on frame rate you want to keep. But of course if binding each one texture takes much longer than BIND_TIME_LIMIT you haven't solved anything.
You might still experience framerate drop while loading images though on slower machines / graphics cards. The rest of the code should be prepared to live with it (e.g. use actual time to drive animation).
Alternative solution is to bind in a separate thread (using a second invisible QGLWidget, see documentation):
2. Texture uploading in a thread.
Doing texture uploads in a thread may be very useful for applications handling large amounts of images that needs to be displayed, like for instance a photo gallery application. This is supported in Qt through the existing bindTexture() API. A simple way of doing this is to create two sharing QGLWidgets. One is made current in the main GUI thread, while the other is made current in the texture upload thread. The widget in the uploading thread is never shown, it is only used for sharing textures with the main thread. For each texture that is bound via bindTexture(), notify the main thread so that it can start using the texture.
Is there a fast way to get the viewport coordinates of QGraphicsItems in a QGraphicsView? The only way I can think of is to call QGraphicsView::items(), and then QGraphicsItem::pos() followed by QGraphicsView::mapFromScene.
I must be missing something, though, because items are already converted to viewport coordinates to position them correctly on the QGraphicsView, so converting it to viewport coordinates again with mapFromScene seems inefficient--especially because in my case this is occurring often and for many items. Is there a more direct approach?
Probably not. A QGraphicScene can be rendered by more than one QGraphicsView simultaneously. It makes no sense to keep only one set of view port coordinates.
Also. All operation between QGraphicsItems are calculated in scene coordinate directly. Events from view port are convert to scene before processing. Working off view port which is only integer-based can also loose precision. A QGraphicsView is only a representation of the mathematical model of a scene. It's not the actual model.
Maybe you can ask a more specific question on what exactly you are trying to accomplish. There may be a better way to do it in scene coordinate.
I am developing a program that must calculate a color of each point on a 1024x1024 picture using a special algorithm. The color of a point represents some value. So each point is independent of other points and must be drawn separately. I do not have to refresh the picture too frequently. Actually, I need to display it only once.
What is the fastest approach to drawing separate pixels in Qt?
Can I get some sort of "screen memory" and write all the picture as an array of 4-byte sets, representing each pixel as 4 bytes in that memory?
The QImage class is optimized for pixel manipulation. You can instantiate one with the requred size and then either set the pixels individually setPixel, or access the raw data and manipulate them in place via bits(). Just be sure to use the correct format (e.g. RGBA values or color indices for 8-bit images)
The fastest solution may be to create a QImage, manipulate it (set the pixels) and then get Qt to draw it.
The QImage class is for fast IO, from the manual:
The QImage class provides a hardware-independent image representation that allows direct access to the pixel data, and can be used as a paint device.
The QImage class supports several image formats described by the Format enum. These include monochrome, 8-bit, 32-bit and alpha-blended images which are available in all versions of Qt 4.x.
There is information on pixel manipulation in the Detailed Description section.
To display it the simplest way would be to convert it to a pixmap with QPixmap::fromImage and then put it in a label with QLabel::setPixmap.
For more control, you could sub-class QWidget, overload the paintEvent, and draw the QImage with a QPainter with QPainter::drawImage.
You might try to use an OpenGL widget and the glDrawPixels function.