I get raw video data from the V4L2 driver using VIDIOC_DQBUF and I want to render this frames in qt using QVideoFrame as described here: https://blog.katastros.com/a?ID=9f708708-c5b3-4cb3-bbce-400cc8b8000c
This code works well but has huge performance issues.
Here is the problematik code part when doing this:
QVideoFrame f(size, QSize(width, height), width, QVideoFrame::Format_YUV420P);
if (f.map(QAbstractVideoBuffer::WriteOnly)) {
memcpy(f.bits(), data, size);
f.setStartTime(0);
f.unmap();
emit newFrameAvailable(f);
}
The memcpy operation for my 4K video reduces the framerate from 35fps to 5fps on my arm based embedded system.
This constructor is supposed to constructs a video frame from a buffer with the given pixel format and size in pixels. However I cannot find any example of this:
QVideoFrame::QVideoFrame(QAbstractVideoBuffer *buffer, const QSize &size, QVideoFrame::PixelFormat format)
I just need to pass valid buffer to QVideoFrame. I don't need to map or unmap the QVideoFrame. Like this:
unsigned char * pBuffer = get_pointer_to_a_frame();
QVideoFrame frame((QAbstractVideoBuffer *) pBuffer, QSize(width, height), QVideoFrame::Format_YUV420P);
frame.setStartTime(0);
emit newFrameAvailable(frame);
Any zero-copy QVideoFrame usage will wellcome.
Related
I'm trying to use a QOpenGLWidget to show some images instead of using QLabel. But I'm a bit confused about how to do this.
To make the widget get the job done, I know I need to reimplement the initializeGL() method and paintGL() method.
To get the texture of an image, what I used is SOIL_load_image(). Why is unsigned char* img_data over unsigned char* img_data[3]? I think each pixel of an image has 3 values(RGB).
After getting the texture, I have no idea what I should do and where should I do them in initializeGL() or paintGL(). Can anyone tell the steps?
void MyOpenGLWidget::loadTexture(const char* file_path)
{
*image = cv::imread(file_path, cv::IMREAD_COLOR);
width = image->rows;
height = image->cols;
int channels = image->channels();
img_data = SOIL_load_image(file_path, &width, &height, &channels, SOIL_LOAD_RGB);
}
Why is unsigned char* img_data over unsigned char* img_data[3]
unsigned char* is a pointer to a buffer (of arbitrary length) of data. unsigned char* …[3] is an array of 3 pointers to buffers of data. You have only one buffer, not 3.
For some reason you're using both OpenCV and then SOIL to read the same image two times. Why?
Once you've loaded the image, to display it with OpenGL you have to
Create a texture object (glGenTextures, glBindTexture, glTexImage)
Create some geometry to draw it (usually a quad, or a viewport filling triangle), by filling a vertex buffer object (glGenBuffers, glBindBuffer, glBufferData) and associating the data in the buffer with vertex attributes of a vertex array object (glGenVertexArrays, glBindVertexArray, glEnableVertexArrayAttrib, glVertexAttribPointer)
Create a shader program, consisting of a vertex shader that places the geometry and paramtizes the fragment shader, which actually samples from the texture. (glCreateShader, glShaderSource, glCreateProgram, glLinkProgram)
Then to draw
select the shader program (glUseProgram)
set parameters (glUniform)
draw (glDrawArrays)
Some background info about my issue. My goal is to optimize drawing of images coming from webcam, the images come as QVideoFrame and are currently loaded in to QImage and drawn from there. This solution works fine, but drawing QImage is very slow on X11. Drawing one image takes about 20ms which doesn't sound like much but when you do this for every frame this cut's the framerate of the camerafeed to half.
I did some research and testing, drawing QPixMaps in X11 can be done about 10 times faster than drawing QImages.
This is how the drawing process is done currently
if(mVFcurrentFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage image(mVFcurrentFrame.bits(), mVFcurrentFrame.width(), mVFcurrentFrame.height(), mVFcurrentFrame.bytesPerLine(), imageFormat);
painter->drawImage(0,0,image); //Takes about 20ms
mVFcurrentFrame.unmap();
}
What i have tried so far:
Converting the QImage to QPixMap, this works but the conversion is as slow as painting the Qimage
Loading the QVideoFrame straight to QPixMap with QPixMap::loadFromData(), can't make it work.
So my question is, can i convert QVideoFrame straight to QPixMap and draw it instead of using QImage and how would you do the QVideoFrame to QPixmap conversion without using QImage in between?
I have tried using QPixMap::loadFromData() method to load the video frame but so far i have been unable to make it work.
If this isn't possible could i thread the QImage to QPixMap conversion or optimize the drawing in some other way?
This is my problem too.
camera frames are shown very slowly in QLabel.
my code is here:
QCamera *camera = new QCamera(this);
camera->setCaptureMode(QCamera::CaptureViewfinder);
QVideoProbe *videoProbe = new QVideoProbe(this);
bool ret = videoProbe->setSource(camera);
if (ret) {
connect(videoProbe, SIGNAL(videoFrameProbed(const QVideoFrame &)),
this, SLOT(present(const QVideoFrame &)));
}
camera->start();
...
...
bool MainWindow::present(const QVideoFrame &frame)
{
QVideoFrame cloneFrame(frame);
if(cloneFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage img(
cloneFrame.size(), QImage::Format_ARGB32);
qt_convert_NV21_to_ARGB32(cloneFrame.bits(),
(quint32 *)img.bits(),
cloneFrame.width(),
cloneFrame.height());
label->setPixmap(QPixmap::fromImage(img));
cloneFrame.unmap();
}
return true;
}
I need to construct an image with unsigned char data I receive from a compressed/decompressed image. For this, I just wrote a simple program to test buffer loading from image and vice versa. As I run the code, I cannot setpixmap the image to the background.
void MainWindow::LoadImage()
{
//======== Load buffer from image
unsigned char buffer[_width*_height*COLOR_COMPONENTS]; //1024 * 768 * 3
QImage image;
image.load("://image.jpg", "JPEG");
memcpy(buffer, image.bits(), _width*_height*COLOR_COMPONENTS);
//========= Load image from buffer
QImage img;
img.loadFromData((const char*)buffer);
QPixmap px = QPixmap::fromImage(img);
ui->label->setPixmap(px);
}
UPDATED:
I changed the code to this, however, I get segmentation fault with memcpy.
unsigned char buffer[400*300*3];
QImage image(_width, _height, QImage::Format_RGB32);
image.load("://image.jpg", "JPEG");
memcpy(buffer, image.bits(), 400*300*3);
QImage img(400, 300, QImage::Format_RGB32);
img.loadFromData((const uchar*)buffer, sizeof(buffer)/sizeof(char), "JPG");
QPixmap px = QPixmap::fromImage(img);
ui->label->setPixmap(px);
loadFromData requires the data to be in a certain format (PNG, JPG...), the plain array is not a valid one
however doing
QImage img(buffer,_width, _height, QImage::Format_RGB888);
will return a image of the correct size and format but will only be valid for as long as buffer is alive
I'm trying to create QImage that wrap a existing image buffer that is created by OpenCv
I was considering use following constructor to do this.
QImage::QImage ( const uchar * data, int width, int height,
int bytesPerLine, Format format )
so, my code is like
QImage qimage((const uchar*)iplImage->imageData,
iplImage->width, iplImage->height,
iplImage->widthStep,
QImage::Format_Indexed); // image buffer not copied!
qimage.setColorTable(grayScaleColorTable); // color table's item count 256 for grayscale.
// now new image buffer is allocated here.
Ok, no memory copy actually was done at the time of calling this ctor.
But, here comes my problem. QImage::setColorTable() is non const member function where QImage allocates new image buffer for copying by its internal detach() function.
I found there was Qt3 support for this kind of problem where ctor could accept color table as argument in its ctor, but I've not found any such support in > Qt4.
How can I create gray scale QImage for existing image buffer?
Thanks for in advance
[EDITED]
Thanks to Stephen Chu, I realized that following contstructors create read/write-able QImage object
QImage ( uchar * data, int width, int height, Format format )
QImage ( uchar * data, int width, int height, int bytesPerLine, Format format )
which even if QImage::setColorTable() is called later right after instantiation, no new buffer is allocated. On the other hand, following constructors receiving 'const'ed data buffer create read-only QImage objects which new buffer is allocated and deep copied from original buffer when any non-const member function like QImage::setColorTable() is called(that I do not want).
QImage ( const uchar * data, int width, int height, Format format )
QImage ( const uchar * data, int width, int height, int bytesPerLine, Format format )
I want bind QImage to the MMF file to manipulate the image without the cost of memory directly on the disc. Unfortunately, my code creates a copy in memory.
QFile file("Boston City Flow.jpg");
if(!file.open(QIODevice::ReadOnly))
QMessageBox::information(this, "Error", "Error");
qint64 size = file.size();
unsigned char *mmf = file.map(0, size);
QImage image;
image.loadFromData(mmf, size, NULL);
My program needs to handle very large images.
Try with declaring mmf const:
const unsigned char* mmf = file.map(0, size);
and then have a look at the QImage ctors, especially
QImage( const uchar*, int width, int height, Format )
QImage::QImage ( const uchar * data, int width, int height, Format format )
The docs say:
"The buffer must remain valid throughout the life of the QImage and all copies that have not been modified or otherwise detached from the original buffer. The image does not delete the buffer at destruction.
[...]
Unlike the similar QImage constructor that takes a non-const data buffer, this version will never alter the contents of the buffer. For example, calling QImage::bits() will return a deep copy of the image, rather than the buffer passed to the constructor. This allows for the efficiency of constructing a QImage from raw data, without the possibility of the raw data being changed."
Note that the non-const uchar* version copies the right away, so make sure to pass a const uchar*. Also note that calling non-const methods of QImage will copy the data.