OpenGL texture not rendering if it's currently active - qt

I was using QOpenGLWidget to render textured triangle, the code was looking good but the triangle was always rendering black i had problem with it for two days until i accidentally found out what the title says.
This is the code, the texture gets loaded to default location of GL_TEXTURE0 and the code will not work unless i call glActiveTexture(GL_TEXTURE1) at the end, GL_TEXTURE1 is just an example it can be any other texture slot except the one where texture actually is. Without the call the object will be black.
QImage ready;
QImage image("C:/Users/Gamer/Desktop/New folder/ring.jpg");
ready = image.convertToFormat(QImage::Format_RGBA8888);
glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glUniform1i(glGetUniformLocation(program.programId(), "samp"), 0);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, ready.width(), ready.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, ready.constBits());
glGenerateMipmap(GL_TEXTURE_2D);
glActiveTexture(GL_TEXTURE1)
I've tried some tests, creating multiple textures and displaying them all at once, the last active texture was always black unless i activate some other unoccupied slot.
I don't know what to make of this, i'm begginer in OpenGL and Qt but this doesn't sound right.
EDIT:
Main function
#include "mainwindow.h"
#include <QApplication>
#include <QSurfaceFormat>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
QSurfaceFormat format;
format.setVersion(3, 3);
format.setProfile(QSurfaceFormat::CoreProfile);
format.setDepthBufferSize(24);
format.setStencilBufferSize(8);
format.setSamples(4);
format.setSwapInterval(0);
QSurfaceFormat::setDefaultFormat(format);
MainWindow w;
w.show();
return a.exec();
}
Widget code
#include "openglwidget.h"
#include <QOpenGLShaderProgram>
#include <QImage>
#include <QDebug>
OpenGLWidget::OpenGLWidget(QWidget *parent) :
QOpenGLWidget(parent)
{
}
OpenGLWidget::~OpenGLWidget()
{
glDeleteBuffers(1, &vbo);
glDeleteVertexArrays(1, &vao);
glDeleteTextures(1, &texture);
}
void OpenGLWidget::initializeGL()
{
QOpenGLFunctions_3_3_Core::initializeOpenGLFunctions();
GLfloat vertices[] = {
0.0f, 0.75f, 0.0f,
-0.75f, -0.75f, 0.0f,
0.75f, -0.75f, 0.0f,
0.5f, 0.0f,
0.0f, 1.0f,
1.0f, 1.0f
};
glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
program.addShaderFromSourceFile(QOpenGLShader::Vertex, "C:/Users/Gamer/Desktop/New folder/vertex.vert");
program.addShaderFromSourceFile(QOpenGLShader::Fragment, "C:/Users/Gamer/Desktop/New folder/fragment.frag");
program.link();
program.bind();
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);
glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 0, (void*)0);
glEnableVertexAttribArray(0);
glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 0, (void*)36);
glEnableVertexAttribArray(1);
QImage ready;
QImage image("C:/Users/Gamer/Desktop/New folder/ring.jpg");
ready = image.convertToFormat(QImage::Format_RGBA8888);
glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glUniform1i(glGetUniformLocation(program.programId(), "samp"), 0);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, ready.width(), ready.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, ready.constBits());
glGenerateMipmap(GL_TEXTURE_2D);
// glActiveTexture(GL_TEXTURE1);
}
void OpenGLWidget::paintGL()
{
GLfloat yellow[] = {1.0, 1.0, 0.0, 0.0};
glClearBufferfv(GL_COLOR, 0, yellow);
glDrawArrays(GL_TRIANGLES, 0, 3);
}
void OpenGLWidget::resizeGL(int w, int h)
{
glViewport(0, 0, w, h);
}
And shaders
#version 330 core
layout(location = 0) in vec3 pos;
layout(location = 1) in vec2 coord;
out vec2 tc;
void main(void)
{
tc = coord;
gl_Position = vec4(pos, 1.0);
}
#version 330 core
uniform sampler2D samp;
in vec2 tc;
out vec4 color;
void main(void)
{
color = texture(samp, tc);
}

QOpenGLWidget is a rather complex abstraction which has some side effects which you might not expect. Quoting from the Qt5 docs:
All rendering happens into an OpenGL framebuffer object. makeCurrent() ensure that it is bound in the context. Keep this in mind when creating and binding additional framebuffer objects in the rendering code in paintGL(). Never re-bind the framebuffer with ID 0. Instead, call defaultFramebufferObject() to get the ID that should be bound.
Now, this in itself isn't an issue. However, looking at the description for the initializeGL() method (my emphasis):
There is no need to call makeCurrent() because this has already been done when this function is called. Note however that the framebuffer is not yet available at this stage, so avoid issuing draw calls from here. Defer such calls to paintGL() instead.
Now, this in itself still is not the issue. But: it means that Qt will create the FBO in-between initializeGL and the first paintGL. Since Qt creates a texture as the color buffer for the FBO, this means it will re-use the currently active texture unit, and change the texture binding you did establish in initializeGL.
If you, on the other hand set glActiveTexture to something other than unit 0, Qt will screw up the binding of that unit, but since you only use unit 0, it will not have any negative effects in your example.

You need to bind the texture to the texture unit before drawing. Texture unit state is not part of program state, unlike uniforms. It is unusual to try and set texture unit state during program startup, that would require allocating different texture units to each program (not out of the question, it's just not the way things are normally done).
Add the following line to paintGL, before the draw call:
glBindTexture(GL_TEXTURE_2D, texture);

Related

Qt signal slot cv::Mat unable to read memory access violation

I have a Microsoft visual studio application that is grabbing frames from cameras and I am trying to display those frames in a Qt application. I am doing some processing with the frames using OpenCV, so the frames are Mat objects. I use QThreads to parallelize the application. I am getting a Access Violation reading location when I try to emit a Mat signal from my CameraThread class.
main.cpp
int main(int argc, char *argv[])
{
QApplication app(argc, argv);
MainWindow window;
window.show();
return app.exec();
}
mainwindow.cpp
#include "main_window.h"
MainWindow::MainWindow()
{
// create a horizontal widget
main_layout = new QVBoxLayout;
QHBoxLayout* row1 = new QHBoxLayout;
QHBoxLayout* row2 = new QHBoxLayout;
for (int i = 0; i < 1; i++) {
camera_array[i] = new CameraWidget(i);
if (i < 4)
row1->addWidget(camera_array[i]);
else
row2->addWidget(camera_array[i]);
}
main_layout->addLayout(row1);
main_layout->addLayout(row2);
// make the central widget the main layout window
central = new QWidget();
central->setLayout(main_layout);
setCentralWidget(central);
}
camerawidget.cpp
#include "stdafx.h"
#include "camera_widget.h"
CameraWidget::CameraWidget(int id)
{
camera_id = id;
qRegisterMetaType<cv::Mat>("cv::Mat");
current_frame = cv::imread("camera_1.png");
thread = new CameraThread(camera_id);
QObject::connect(thread, SIGNAL(renderFrame(cv::Mat)), this, SLOT(updateFrame(cv::Mat)));
thread->start();
}
CameraWidget::~CameraWidget()
{
qDebug("camera widget destructor");
thread->wait(5000);
}
// initializeGL() function is called just once, before paintGL() is called.
void CameraWidget::initializeGL()
{
qglClearColor(Qt::black);
glDisable(GL_DEPTH_TEST);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, 480.0f, 640.0f, 0.0f, 0.0f, 1.0f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glEnable(GL_TEXTURE_2D);
glGenTextures(3, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glBindTexture(GL_TEXTURE_2D, texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 0, 480.0f, 640.0f, GL_BGR, GL_UNSIGNED_BYTE, NULL);
glDisable(GL_TEXTURE_2D);
}
void CameraWidget::paintGL()
{
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glDisable(GL_DEPTH_TEST);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, 480.0f, 640.0f, 0.0f, 0.0f, 1.0f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glEnable(GL_TEXTURE_2D);
current_frame_i = QImage(current_frame.data, current_frame.cols, current_frame.rows, current_frame.cols * 3, QImage::Format_RGB888);
glBindTexture(GL_TEXTURE_2D, texture);
// ******************************
// getting access violation here
// ******************************
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 480.0f, 640.0f, 0.0f, GL_BGR, GL_UNSIGNED_BYTE, current_frame.ptr());
glBegin(GL_QUADS);
glTexCoord2i(0, 1); glVertex2i(0, 640.0f);
glTexCoord2i(0, 0); glVertex2i(0, 0);
glTexCoord2i(1, 0); glVertex2i(480.0f, 0);
glTexCoord2i(1, 1); glVertex2i(480.0f, 640.0f);
glEnd();
glFlush();
}
void CameraWidget::resizeGL(int w, int h)
{
// setup viewport, projection etc.
glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, 480.0f, 640.0f, 0.0f, 0.0f, 1.0f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
}
void CameraWidget::updateFrame(cv::Mat image)
{
current_frame = image;
update();
}
camerathread.cpp
CameraThread::CameraThread(int id)
{
camera_q = new bounded_frame_queue(50);
}
void CameraThread::run()
{
cv::Mat image;
while (true) {
if (!camera_q->empty()) {
image = camera_q->pop();
if (!image.empty())
emit renderFrame(image);
}
else {
msleep(1);
}
}
}
When I emit renderFrame from the camerathread.cpp, I get an access violation reading location. I cannot read the current_frame.ptr() value in camerawidget.cpp.
Can someone direct me on how I can fix this issue?
What I see is happenning:
You get an image from queue. As per OpenCV docs:
Mat& cv::Mat::operator= ( const Mat & m )
Assigned, right-hand-side matrix. Matrix assignment is an O(1)
operation. This means that no data is copied but the data is shared
and the reference counter, if any, is incremented. Before assigning
new data, the old data is de-referenced via Mat::release .
Then you pass it as cv::Mat image (by value) when emitting signal. The copy constructor again doesn't copy any data:
Array that (as a whole or partly) is assigned to the constructed
matrix. No data is copied by these constructors. Instead, the header
pointing to m data or its sub-array is constructed and associated with
it. The reference counter, if any, is incremented. So, when you modify
the matrix formed using such a constructor, you also modify the
corresponding elements of m . If you want to have an independent copy
of the sub-array, use Mat::clone() .
Your data pointers are queued on UI thread
You get/try-get new frame triggering release from p.1
Your queued slot is executed and crashes...
Suggestion: I haven't worked much with it, but it seems something like cv::Mat::clone to make a deep copy is what you need, to prevent release of memory before it would be used by UI thread.
Or possibly it would be enough to define image right when you pop it from queue:
cv::Mat image = camera_q->pop();

OpenGL + Qt 4.8 is not drawing anything

I've been trying to use OpenGL in Qt with shaders and a simple vertex array. I basically want a plain to be drawn in the middle of the screen but nothing appears when I run the program. I'm basing my code in the "Texture" example of Qt, everything looks the same to me, but It's not working!
Here's the code of my glwidget.cpp:
#include "glwidget.h"
GLWidget::GLWidget(QWidget *parent):QGLWidget(parent)
{
timer.start(10);
//connect(&timer, SIGNAL(timeout()), this, SLOT(updateGL()));
Object aux;
QVector3D auxV;
auxV.setX(0.4); auxV.setY(0.4); auxV.setZ(1.0);
aux.vertices.append(auxV);
auxV.setX(0.4); auxV.setY(-0.4); auxV.setZ(1.0);
aux.vertices.append(auxV);
auxV.setX(-0.4); auxV.setY(-0.4); auxV.setZ(1.0);
aux.vertices.append(auxV);
auxV.setX(-0.4); auxV.setY(-0.4); auxV.setZ(1.0);
aux.vertices.append(auxV);
Objects.append(aux);
}
GLWidget::~GLWidget()
{
}
void GLWidget::initializeGL()
{
#define PROGRAM_VERTEX_ATTRIBUTE 0
printf("Objects Size: %d\nObj1 Size: %d\n", Objects.size(), Objects.at(0).vertices.size());
glEnable(GL_DEPTH_TEST);
glEnable(GL_CULL_FACE);
//printf("Version: %s\n", glGetString(GL_VERSION));
vShader= new QGLShader (QGLShader::Vertex, this);
vShader->compileSourceFile("../src/shaders/editorVshader.glsl");
fShader= new QGLShader (QGLShader::Fragment, this);
fShader->compileSourceFile("../src/shaders/editorFshader.glsl");
editor= new QGLShaderProgram (this);
editor->addShader(vShader);
editor->addShader(fShader);
editor->bindAttributeLocation("vertices", PROGRAM_VERTEX_ATTRIBUTE);
editor->link();
editor->bind();
}
void GLWidget::paintGL()
{
glClearColor(0.4765625, 0.54296875, 0.6171875, 1.0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glLoadIdentity();
glTranslatef(0.0f, 0.0f, -10.0f);
glVertexPointer(3, GL_FLOAT, 0, Objects.at(0).vertices.constData());
glEnableClientState(GL_VERTEX_ARRAY);
QMatrix4x4 auxM;
auxM.ortho(-0.5, 0.5, 0.5, -0.5, 4.0, -15.0);
auxM.translate(0.0f, 0.0f, -10.0f);
editor->setUniformValue("modelmatrix", auxM);
editor->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
//editor->enableAttributeArray(editor->attributeLocation("vertices"));
//editor->setAttributeArray(editor->attributeLocation("vertices"), Objects.at(0).vertices.constData(), 3);
editor->setAttributeArray (PROGRAM_VERTEX_ATTRIBUTE, Objects.at(0).vertices.constData());
glDrawArrays(GL_QUADS, 0, 4);
}
void GLWidget::resizeGL(int w, int h)
{
int side = qMin(w, h);
glViewport((w - side) / 2, (h - side) / 2, side, side);
//glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(-0.5, 0.5, -0.5, 0.5, 4.0, -15.0);
glMatrixMode(GL_MODELVIEW);
updateGL();
}
And here are my vShader:
attribute highp vec4 vertices;
attribute highp mat4x4 modelmatrix;
void main (void)
{
gl_Position= modelmatrix*vertices;
}
And my fShader:
void main(void)
{
gl_FragColor= vec4(0.0, 0.1, 1.0, 1.0);
}
Do you see the error in there?
You are mixing OpenGLES1.1 (ex calls to glOrtho, glTranslate) and 2.0 (using shaders). Are you mixing the textures+overpainting examples ? You should instead take just one example that uses OpenGL / ES/ 1.1 or 2.0 like - http://qt-project.org/doc/qt-5.0/qtopengl/textures.html, then make changes and see how the code works.
I found what the problem was.
You're right, prabindh, I was mixing OpenGL versions, and the problem was related to that. After cleaning the code from OpenGL ES instructions, I also added the model, view and projection matrices in order to pass them to the shaders. It worked! Actually I think the plane was there all the time but I just couldn't see it because "the camera was aiming somewhere else".
But I wasn't mixing the examples, I based all my code in the textures example. I still can't understand why its code works and mine didn't. But anyway, everything went fine after that.
Thanks for your answers and your comments!

What is the magic inside Qt's bindTexture?

I am using Qt5 and QGLWidget class to render a live stream of pixels. I am having some performance problems and want to setup a pixel buffer to perform asynchronous data transfer.
I am trying to bind a texture the old fashioned way but it render as a blank. When I go back to my Qt5 code everything works.
Does anybody know how to get the standard OpenGL version to work?
Working
void glStream::reserveTextures()
{
displayBuff = (GLubyte*) calloc(numGLFrames*widthGL*heightGL,
sizeof(GLubyte));
QImage mySurface(&displayBuff[displayStart],widthGL,heightGL,
QImage::Format_Indexed8);
textures[0]=bindTexture(mySurface,GL_TEXTURE_2D,GL_LUMINANCE);
glBindTexture(GL_TEXTURE_2D,0);
glECheck();//no errors
}
Not Working
void glStream::reserveTextures()
{
displayBuff = (GLubyte*) calloc(numGLFrames*widthGL*heightGL,sizeof(GLubyte));
glGenTextures(1,&textures[0]);
glBindTexture(GL_TEXTURE_2D,textures[0]);
glTexImage2D(GL_TEXTURE_2D,0,GL_LUMINANCE,widthGL,heightGL,0,GL_LUMINANCE,
GL_UNSIGNED_BYTE,&displayBuff[displayStart]);
glBindTexture(GL_TEXTURE_2D,0);
glECheck();//no errors
}
Update Function
glECheck();
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glLoadIdentity();
program1.bind();
program1.setUniformValue("texture", 0);
program1.enableAttributeArray(vertexAttr1);
program1.enableAttributeArray(vertexTexr1);
program1.setAttributeArray(vertexAttr1, vertices.constData());
program1.setAttributeArray(vertexTexr1, texCoords.constData());
glBindTexture(GL_TEXTURE_2D, textures[0]);
glECheck();
glDrawArrays(GL_TRIANGLES, 0, vertices.size());
glBindTexture(GL_TEXTURE_2D,0);
program1.disableAttributeArray(vertexTexr1);
program1.disableAttributeArray(vertexAttr1);
program1.release();
glECheck();
Shader
QGLShader *fshader1 = new QGLShader(QGLShader::Fragment, this);
const char *fsrc1 =
"uniform sampler2D texture;\n"
"varying mediump vec4 texc;\n"
"void main(void)\n"
"{\n"
" gl_FragColor = texture2D(texture, texc.st);\n"
"}\n";
fshader1->compileSourceCode(fsrc1);
program1.addShader(fshader1);
It would appear that you need a few other settings enabled to display a texture.
void glStream::reserveTextures()
{
displayBuff = (GLubyte*) calloc(numGLFrames*widthGL*heightGL,sizeof(GLubyte));
glGenTextures(1,&textures[0]);
glBindTexture(GL_TEXTURE_2D,textures[0]);
glTexImage2D(GL_TEXTURE_2D,0,GL_LUMINANCE,widthGL,heightGL,0,GL_LUMINANCE,
GL_UNSIGNED_BYTE,&displayBuff[displayStart]);
//This is the magic inside Qt5's implementation
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glBindTexture(GL_TEXTURE_2D,0);
glECheck();//no errors
}

How can I draw my opengl background keeping good performance?

I draw my background every frames :
void Window::paintGL()
{
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glLoadIdentity();
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
backgrd->Draw();
texture = text->loadTexture(text->amis_path.toStdString(),text->w,text->h);
amis->Draw();
texture = text->loadTexture(text->enemis_path.toStdString(),text->w,text->h);
for (int i = 0; i<liste_enemis.length(); i++){
liste_enemis[i]->Draw();
}
for (int i = 0; i<liste_missiles.length(); i++){
liste_missiles[i]->Draw();
}
swapBuffers();
}
But when I run the game, the fps a pretty bad (1fps).
Edit :
Well I'm trying to load my background in a texture one time but it doesn't work :
Background::Background(int w, int h)
{
width = w;
height = h;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexImage2D(GL_TEXTURE_2D,0, GL_RGBA, back.width(), back.height(), 0,
GL_RGBA, GL_UNSIGNED_BYTE, back.bits());
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
}
void Background::Draw(){
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D,texture);
glPushMatrix();
glTranslatef(width/2, height/2, 0.0f);
glBegin(GL_QUADS);
glTexCoord2d(0,0); glVertex3f(-width/2, -height/2, 0.0f);
glTexCoord2d(0,1); glVertex3f(-width/2, height/2, 0.0f);
glTexCoord2d(1,1); glVertex3f(width/2, height/2, 0.0f);
glTexCoord2d(1,0); glVertex3d(width/2, -height/2, 0.0f);
glEnd();
glPopMatrix();
}
The constructor load the background with glTexImage2D(...) and links it to the "texture" variable.
Then in my paintGL() function, I call the Background::Draw() function.
But when I run the game, there is no background.
If I move glTexParameteri(...) and glTexImage2D(...) functions from constructor to Draw function, it works.
You're reloading the textures each and every frame, that's what's causing your performance it (it will also consume your memory in no time).
Instead of reloading the texture every frame, load them only once, and then switch between textures using glBindTexture

Integrating OpenCV with larger programs

Can anyone recommend a how-to guide or provide a brief overview of what's involved with integrating OpenCV with larger GUI-based programs? What are the popular ways to do it?
Particularly, processing video with OpenCV while doing video capture/preview without using HighGUI seems especially arcane. I hope someone can demystify this.
My particular configuration is with either Juce or Qt depending on what can be done. The cross platform thing is not critical -- if there is an awesome way of doing this in Windows, I might be convinced. The availability of community support is important.
I have heard that HighGUI is entirely for testing and unsuitable for real applications. Someone recommended the VideoInput library, but it is experimental.
Key points from answers:
Use Qt (because Qt is great and has a big community).
Open a new thread to run cv::VideoCapture in a loop and emit signal after frame capture. Use Qt's msleep mechanism, not OpenCV. So, we are still using OpenCV highgui for capture.
Convert cv::Mat to QtImage:
QImage qtFrame(cvFrame.data, cvFrame.size().width, cvFrame.size().height, cvFrame.step, QImage::Format_RGB888);
qtFrame = qtFrame.rgbSwapped();
Optional: Render with GLWidget. Convert QtImage to GLFormat with Qt built-in method:
m_GLFrame = QGLWidget::convertToGLFormat(frame);
this->updateGL();
Here is how I am doing it with Qt. You are welcome to use whatever may be useful to you :)
/// OpenCV_GLWidget.h
#ifndef OPENCV_GLWIDGET_H_
#define OPENCV_GLWIDGET_H_
#include <qgl.h>
#include <QImage>
class OpenCV_GLWidget: public QGLWidget {
public:
OpenCV_GLWidget(QWidget * parent = 0, const QGLWidget * shareWidget = 0, Qt::WindowFlags f = 0);
virtual ~OpenCV_GLWidget();
void renderImage(const QImage& frame);
protected:
virtual void paintGL();
virtual void resizeGL(int width, int height);
private:
QImage m_GLFrame;
};
#endif /* OPENCV_GLWIDGET_H_ */
/// OpenCV_GLWidget.cpp
#include "OpenCV_GLWidget.h"
OpenCV_GLWidget::OpenCV_GLWidget(QWidget* parent, const QGLWidget* shareWidget, Qt::WindowFlags f) :
QGLWidget(parent, shareWidget, f)
{
// TODO Auto-generated constructor stub
}
OpenCV_GLWidget::~OpenCV_GLWidget() {
// TODO Auto-generated destructor stub
}
void OpenCV_GLWidget::renderImage(const QImage& frame)
{
m_GLFrame = QGLWidget::convertToGLFormat(frame);
this->updateGL();
}
void OpenCV_GLWidget::resizeGL(int width, int height)
{
// Setup our viewport to be the entire size of the window
glViewport(0, 0, width, height);
// Change to the projection matrix and set orthogonal projection
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, width, height, 0, 0, 1);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
}
void OpenCV_GLWidget::paintGL() {
glClear (GL_COLOR_BUFFER_BIT);
glClearColor (0.0, 0.0, 0.0, 1.0);
if (!m_GLFrame.isNull()) {
m_GLFrame = m_GLFrame.scaled(this->size(), Qt::IgnoreAspectRatio, Qt::SmoothTransformation);
glEnable(GL_TEXTURE_2D);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA, m_GLFrame.width(), m_GLFrame.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, m_GLFrame.bits() );
glBegin(GL_QUADS);
glTexCoord2f(0, 0); glVertex2f(0, m_GLFrame.height());
glTexCoord2f(0, 1); glVertex2f(0, 0);
glTexCoord2f(1, 1); glVertex2f(m_GLFrame.width(), 0);
glTexCoord2f(1, 0); glVertex2f(m_GLFrame.width(), m_GLFrame.height());
glEnd();
glDisable(GL_TEXTURE_2D);
glFlush();
}
}
This class handles the rendering of the image onto a promoted QWidget. Next, I created a thread to feed the widget. (I cheated using the Qt signal-slot architecture here because it was easy...may not be the best performer in the book, but it should get you started).
void VideoThread::run()
{
cv::VideoCapture video(0);
while(!m_AbortCapture)
{
cv::Mat cvFrame;
video >> cvFrame;
cv::Mat gray(cvFrame.size(), CV_8UC1);
cv::GaussianBlur(cvFrame, cvFrame, cv::Size(5, 5), 9.0, 3.0, cv::BORDER_REPLICATE);
cv::cvtColor(cvFrame, gray, CV_RGB2GRAY);
m_ThresholdLock.lock();
double localThreshold = m_Threshold;
m_ThresholdLock.unlock();
if(localThreshold > 0.0)
{
qDebug() << "Threshold = " << localThreshold;
cv::threshold(gray, gray, localThreshold, 255.0, cv::THRESH_BINARY);
}
cv::cvtColor(gray, cvFrame, CV_GRAY2BGR);
// convert the Mat to a QImage
QImage qtFrame(cvFrame.data, cvFrame.size().width, cvFrame.size().height, cvFrame.step, QImage::Format_RGB888);
qtFrame = qtFrame.rgbSwapped();
// queue the image to the gui
emit sendImage(qtFrame);
msleep(20);
}
}
Took me a bit to figure that out, so hopefully it will help you and others save some time :D
Create an openCV image to hold the image you have captured.
Do processing on it and then copy the data into the image you want to display (eg QImage)
You can optomise things by creating the opencv cv::Mat image to share the memory with the
QImage but since QImage generally uses ARGB and most image processing tasks are better done as greyscale or RGB it's probably better to copy images and convert between them using the opencv cvtColor() function
Then simply include include the opencv headers, and link with the opencv libs - there are guides on the opencv wiki for your particular environment

Resources