How to draw QGLFrameBufferObject onto the painter from within QGraphicsItem::paint() - qt

Small version of my question
In a QGraphicsItem::paint() function I have a QGLFrameBufferObject. How do I get it on the paintdevice of the painter that is passed as an argument? (provided that the QGraphicsItem is in a scene that is being rendered by a QGraphicsView that has a QGLWidget as viewport => painter is using opengl engine)
QGraphicsItem::paint(QPainter* painter, ...)
{
QGLFramebufferObject fbo;
QPainter p(fbo);
... // Some painting code on the fbo
p.end();
// What now? How to get the fbo content drawn on the painter?
}
I have looked at the framebufferobject and pbuffer examples provided with Qt. There the fbo/pbuffer is drawn in a QGLWidget using custom opengl code. Is it possible to do the same thing within a paint() method of a QGraphicsItem and take the position of the QGraphisItem in the scene/view into account?
Big version of my question
Situation sketch
I have a QGraphicsScene. In it is an item that has a QGraphicsEffect (own implementation by overriding draw() from QGraphicsEffect). The scene is rendered by a QGraphicsView that has a QGLWidget as viewport.
In the QGraphicsEffect::draw(QPainter*) I have to generate some pixmap which I then want to draw using the painter provided (the painter has the QGLWidget as paintdevice). Constructing the pixmap is a combination of some draw calls and I want these to be done in hardware.
Simplified example: (I don't call sourcepixmap in my draw() method as it is not needed to demonstrate my problem)
class OwnGraphicsEffect: public QGraphicsEffect
{
virtual void draw(QPainter* painter);
}
void OwnGraphicsEffect::draw(QPainter* painter)
{
QRect rect(0,0,100,100);
QGLPixelBuffer pbuffer(rect.size(), QGLFormat(QGL::Rgba));
QPainter p(pbuffer);
p.fillRect(rect, Qt::transparent);
p.end();
painter->drawImage(QPoint(0,0), pbuffer->toImage(),rect);
}
Actual problem
My concerns are with the last line of my code: pbuffer->toImage(). I don't want to use this. I don't want to have a QImage conversion because of performance reasons. Is there a way to get a pixmap from my glpixelbuffer and then use painter->drawpixmap()?
I know I also can copy the pbuffer to a texture by using :
GLuint dynamicTexture = pbuffer.generateDynamicTexture();
pbuffer.updateDynamicTexture(dynamicTexture);
but I have no idea on how to get this texture onto the "painter".

Extending leemes' answer, here is a solution which can also handle multisample framebuffer objects.
First, if you want to draw on a QGLWidget, you can simply use the OpenGL commands
leemes suggested in his answer. Note that there is a ready-to-use drawTexture()
command available, which simplifies this code to the following:
void Widget::drawFBO(QPainter &painter, QGLFramebufferObject &fbo, QRect target)
{
painter.beginNativePainting();
drawTexture(target, fbo.texture());
painter.endNativePainting();
}
To draw multisample FBOs, you can convert them into non-multisample ones
using QGLFramebufferObject::blitFramebuffer (Note that not every hardware
/ driver combination supports this feature!):
if(fbo.format().samples() > 1)
{
QGLFramebufferObject texture(fbo.size()); // the non-multisampled fbo
QGLFramebufferObject::blitFramebuffer(
&texture, QRect(0, 0, fbo.width(), fbo.height()),
&fbo, QRect(0, 0, fbo.width(), fbo.height()));
drawTexture(targetRect, texture.texture());
}
else
drawTexture(targetRect, fbo.texture());
However, as far as I know, you can't draw using OpenGL commands on a non-OpenGL context.
For this, you first need to convert the framebuffer to a (software) image, like
a QImage using fbo.toImage() and draw this using your QPainter instead of the
fbo directly.

I think I figured it out. I use the QPainter::beginNativePainting() to mix OpenGL commands in a paintEvent:
void Widget::drawFBO(QPainter &painter, QGLFramebufferObject &fbo, QRect target)
{
painter.beginNativePainting();
glEnable(GL_TEXTURE_2D);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
glBindTexture(GL_TEXTURE_2D, fbo.texture());
glBegin(GL_QUADS);
glTexCoord2d(0.0,1.0); glVertex2d(target.left(), target.top());
glTexCoord2d(1.0,1.0); glVertex2d(target.right() + 1, target.top());
glTexCoord2d(1.0,0.0); glVertex2d(target.right() + 1, target.bottom() + 1);
glTexCoord2d(0.0,0.0); glVertex2d(target.left(), target.bottom() + 1);
glEnd();
painter.endNativePainting();
}
I hope this will also work in the paintEvent of a QGraphicsItem (where the QGraphicsView uses a QGLWidget as the viewport), since I only tested it in QGLWidget::paintEvent directly.
There is, however, still the following problem: I don't know how to paint a multisample framebuffer. The documentation of QGLFramebufferObject::texture() says:
If a multisample framebuffer object is used then the value returned from this function will be invalid.

Related

Using QGLFramebufferObject and shaders without QGLWidget

I want to create an OGL data processor using QGLFunctions shaders and framebuffers. I don't need any widgets. But to create valid Shader and framebuffer instances, I need a valid QGLContext with support for the appropriate glExtensions.
With zero context, of course, nothing works. With context of zero QPaintDevice too. With Qpixmap as device it creates a valid context, but it lacks glExtensions for Shader and framebuffer.
#include <QGLFramebufferObject>
#include <QGLShaderProgram>
#include <QtOpenGL/QGLFunctions>
// ...
void GLProcessor::init()
{
auto format = QGLFormat::defaultFormat();
if (!context()){
m_context = new QGLContext(format, new QPixmap(1, 1));
bool ok = m_context->create();
qDebug() << "CREATING CONTEXT "<< ok;
Q_ASSERT(context()->isValid());
}
context()->makeCurrent();
initializeGLFunctions(context());
m_binFBO = new QGLFramebufferObject(lowsize ,lowsize ,QGLFramebufferObject::NoAttachment, GL_TEXTURE_2D, GL_RED);
m_outFBO = new QGLFramebufferObject(lowsize ,1 ,QGLFramebufferObject::NoAttachment, GL_TEXTURE_2D, GL_RED);
setupShaders();
// ...
}
There is an option, of course, to do as always is to get the context from the QGLWidget and hide it. But somehow inelegant. PS CUDA, OpenCL, AMP and so on I don't need. For my tasks need OpenGL.
How do I use shaders and framebuffers in qt4 without creating a QGLWidget?

Performance Issue while using QGLWidget with Qt5

I'm trying to develop an application which will be used for the visualization of 3D objects and its simulations. In this I have to draw 'n' number of objects (may be a triangle, rectangle or some other non-convex polygons) with individual color shades.For this I'm using QGLWidget in Qt5 (OS - Windows 7/8/10).
structure used for populating objects information:
typedef struct {
QList<float> r,g,b;
QList<double> x,y,z;
}objectData;
The number of objects and their corresponding coordinate values will be read from a file.
paintGL function:
void paintGL() {
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(25, GLWidget::width()/(float)GLWidget::height(), 0.1, 100);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
gluLookAt(0,0,5, 0,0,0, 0,1,0);
glRotatef(140, 0.0, 0.0, 1.0);
glRotatef(95, 0.0, 1.0, 0.0);
glRotatef(50, 1.0, 0.0, 0.0);
glTranslated(-1.0, 0.0, -0.6);
drawObjects(objData, 1000)
}
Drawing of Objects Function:
void drawObjects(objectData objData,int objCnt) {
glPushMatrix();
glBegin(GL_POLYGON);
for(int i = 0; i < objCnt; i++) {
glColor3f(objData.r[i],objData.g[i],objData.b[i] );
glVertex3d(objData.x[i],objData.y[i],objData.z[i]);
}
glEnd();
glFlush();
glPopMatrix();
}
Issue:
Now, when the number of objects to be drawn exceeds a certain maximum value (for example say n = 5000), the application speed gradually decreases. I'm unable to use QThread since it already inherits QGLWidget.
Please suggest how to improve the performance of the application when number of objects count is higher. I don't know where I'm doing mistake.
Screenshot of that sample:
Sample image which contains number of objects in mesh view
You are using the fixed pipeline instead of the programmable one, where you tell to each stage of the rendering process, what should be done, and nothing more. Among other noticeable differences that I encourage you to research (research "modern opengl", which will lead you to doing OpenGL 3.3 and above type of work).
The old fixed pipeline is terribly inefficient, when the computer has to talk to the graphics card for every geometries while rendering. By contrast, the modern programmable pipeline allows you to push the data of the models to render into the VRAM, from where it will be directly accessed during rendering (very fast memory accesses).
You also get rid of the generic ways of "doing stuff", that are mechanically slower than customized ones.
Also, I encourage you to use QOpenGLWidget instead of the former QGLWidget class. As mentioned in http://doc.qt.io/qt-5/qglwidget.html, this class is obsolete.
Modern OpenGL quick start:
http://www.opengl-tutorial.org/
So, you are not doing anything "wrong". You are just not using the current technology. Have fun!
You are using OpenGLs immediate mode which is very slow for large numbers of vertices und should almost never be used. Use the retained mode instead. See this answer for more detail: https://stackoverflow.com/a/6734071
Thank you #dave and #Zedka9. It works fine for me when I started to use the intermediate mode in openGL. I have modified the drawObject function like this
Drawing of Objects Function:
After organizing and copying the vertices and colors to these buffers
GLfloat vertices[1024*1024],colors[1024*1024];
int vertArrayCnt; // number of verticies
void drawObjects(void) {
glEnableClientState(GL_COLOR_ARRAY);
glEnableClientState(GL_VERTEX_ARRAY);
glColorPointer(3, GL_FLOAT, 0, colors);
glVertexPointer(3, GL_FLOAT, 0, vertices);
glPushMatrix();
glDrawArrays(GL_TRIANGLES, 0, vertArrayCnt);
glPopMatrix();
glDisableClientState(GL_VERTEX_ARRAY); // disable vertex arrays
glDisableClientState(GL_COLOR_ARRAY);
}

Optimizing QPainter drawing & Converting QVideoFrame straight to QPixMap

Some background info about my issue. My goal is to optimize drawing of images coming from webcam, the images come as QVideoFrame and are currently loaded in to QImage and drawn from there. This solution works fine, but drawing QImage is very slow on X11. Drawing one image takes about 20ms which doesn't sound like much but when you do this for every frame this cut's the framerate of the camerafeed to half.
I did some research and testing, drawing QPixMaps in X11 can be done about 10 times faster than drawing QImages.
This is how the drawing process is done currently
if(mVFcurrentFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage image(mVFcurrentFrame.bits(), mVFcurrentFrame.width(), mVFcurrentFrame.height(), mVFcurrentFrame.bytesPerLine(), imageFormat);
painter->drawImage(0,0,image); //Takes about 20ms
mVFcurrentFrame.unmap();
}
What i have tried so far:
Converting the QImage to QPixMap, this works but the conversion is as slow as painting the Qimage
Loading the QVideoFrame straight to QPixMap with QPixMap::loadFromData(), can't make it work.
So my question is, can i convert QVideoFrame straight to QPixMap and draw it instead of using QImage and how would you do the QVideoFrame to QPixmap conversion without using QImage in between?
I have tried using QPixMap::loadFromData() method to load the video frame but so far i have been unable to make it work.
If this isn't possible could i thread the QImage to QPixMap conversion or optimize the drawing in some other way?
This is my problem too.
camera frames are shown very slowly in QLabel.
my code is here:
QCamera *camera = new QCamera(this);
camera->setCaptureMode(QCamera::CaptureViewfinder);
QVideoProbe *videoProbe = new QVideoProbe(this);
bool ret = videoProbe->setSource(camera);
if (ret) {
connect(videoProbe, SIGNAL(videoFrameProbed(const QVideoFrame &)),
this, SLOT(present(const QVideoFrame &)));
}
camera->start();
...
...
bool MainWindow::present(const QVideoFrame &frame)
{
QVideoFrame cloneFrame(frame);
if(cloneFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage img(
cloneFrame.size(), QImage::Format_ARGB32);
qt_convert_NV21_to_ARGB32(cloneFrame.bits(),
(quint32 *)img.bits(),
cloneFrame.width(),
cloneFrame.height());
label->setPixmap(QPixmap::fromImage(img));
cloneFrame.unmap();
}
return true;
}

OpenGL + QT: render to texture and display it back

After some trouble I've managed to correctly render to texture inside a Frame Buffer Object in a Qt 4.8 application: I can open an OpenGL context with a QGLWidget, render to a FBO, and use this one as a texture.
Now I need to display the texture rendered in a QPixmap and show it in some other widget in the gui. But.. nothing is shown.
Those are some pieces of code:
// generate texture, FBO, RBO in the initializeGL
glGenTextures(1, &textureId);
glBindTexture(GL_TEXTURE_2D, textureId);
glGenFramebuffers(1, &fboId);
glBindFramebuffer(GL_FRAMEBUFFER, fboId);
glGenRenderbuffers(1, &rboId);
glBindRenderbuffer(GL_RENDERBUFFER, rboId);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT, TEXTURE_WIDTH, TEXTURE_HEIGHT);
glBindRenderbuffer(GL_RENDERBUFFER, 0);
// now in paintGL
glBindFramebuffer(GL_FRAMEBUFFER, fboId);
// .... render into texture code ....
if(showTextureInWidget==false) {
showTextureInWidget = true;
char *pixels;
pixels = new char[TEXTURE_WIDTH * TEXTURE_HEIGHT * 4];
glReadPixels(0, 0, TEXTURE_WIDTH, TEXTURE_HEIGHT, GL_RGB, GL_UNSIGNED_BYTE, pixels);
QPixmap qp = QPixmap(pixels);
QLabel *l = new QLabel();
// /* TEST */ l->setText(QString::fromStdString("dudee"));
l->setPixmap(qp);
QWidget *d = new QWidget;
l->setParent(d);
d->show();
}
glBindFramebuffer(GL_FRAMEBUFFER, 0); // unbind
// now draw the scene with the rendered texture
I see the Widget opened but.. there is nothing inside it. If I decomment the test line.. I see the "dudee" string so I know that there is a qlabel but.. no image from the QPixmap.
I know that the original data are ´unsigned char´ and I'm using ´char´ and I've tried with some different color parameters (´GL_RGBA´, ´GL_RGB´ etc) but I don't think this is the point.. the point is that I don't see anything..
Any advice? If I have to post more code I will do it!
Edit:
I haven't posted all the code, but the fact I'd like to be clear is that the texture is correctly rendered as a texture inside a cube. I'm just not able to put it back in the cpu from gpu
Edit 2:
Thanks to the peppe answer I found out the problem: I needed a Qt object that accept as a constructor some raw pixels data. Here is the complete snippet:
uchar *pixels;
pixels = new uchar[TEXTURE_WIDTH * TEXTURE_HEIGHT * 4];
for(int i=0; i < (TEXTURE_WIDTH * TEXTURE_HEIGHT * 4) ; i++ ) {
pixels[i] = 0;
}
glBindFramebuffer(GL_FRAMEBUFFER, fboId);
glReadPixels( 0,0, TEXTURE_WIDTH, TEXTURE_HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
qi = QImage(pixels, TEXTURE_WIDTH, TEXTURE_HEIGHT, QImage::Format_ARGB32);
qi = qi.rgbSwapped();
QLabel *l = new QLabel();
l->setPixmap(QPixmap::fromImage(qi));
QWidget *d = new QWidget;
l->setParent(d);
d->show();
Given that that's not all of your code and -- as you say -- the texture is correctly filled, then there's a little mistake going on here:
glReadPixels(0, 0, TEXTURE_WIDTH, TEXTURE_HEIGHT, GL_RGB, GL_UNSIGNED_BYTE, pixels);
QPixmap qp = QPixmap(pixels);
The QPixmap(const char *) ctor wants a XPM image, not raw pixels. You need to use one of the QImage ctors to create a valid QImage. (You can also pass ownership to the QImage, solving the fact that you're currently leaking pixels...)
Once you do that, you'll figure out that
the image is flipped vertically, as OpenGL has the origin in the bottom left corner, growing upwards/rightwards, while Qt assumes origin in the top left, growing to downwards/rightwards;
the channels might be swapped -- i.e. OpenGL is returning data with the wrong endianess. I don't remember in this case if using glPixelStorei(GL_PACK_SWAP_BYTES) or GL_UNSIGNED_INT_8_8_8_8 as the type may help, eventually you need to resort to a CPU-side loop to fix your pixel data :)

How to trace the missing pixels when using drawLine

We know that for drawing on an image in qt, qpainter is used. Recently, I used drawLine() function to draw whatever an user is scribbling. This was done by passing the lastPoint and currentPoint from the mouseMoveEvent to a custom function which implements drawLine(). I have passed the arguments for that custom function as given below:
void myPaint::mouseMoveEvent(QMouseEvent *event) {
qDebug() << event->pos();
if ((event->buttons() & Qt::LeftButton) && scribbling) {
pixelList.append(event->pos());
drawLineTo(event->pos());
lastPoint = event->pos();
}
}
Now with the help of qDebug() I noticed that some pixels are missed while drawing but the drawing is precise. I looked up the source of qt-painting where I saw that drawLine() was calling drawLines() which was making use of qpainterPath to have a shape drawn on the image.
My question is that, is there anyway to track these "missed" pixels or any approach to find all the pixels which have been drawn?
Thanks!
void myPaint::drawLineTo(const QPoint &endPoint) {
QPainter painter(image); //image is initialized in the constructor of myPaint
painter.setRenderHint(QPainter::Antialiasing);
painter.setPen(QPen(Qt::blue, myPenWidth, Qt::SolidLine, Qt::RoundCap,Qt::RoundJoin));
painter.drawLine(lastPoint, endPoint);
modified = true;
lastPoint = endPoint; //at the mousePressEvent, the event->pos() will be stored as
// lastPoint
update();
}
For a start, don't draw in a mouseEvent(). Actually handling a mouseevent should be done as quick as possible. Also, it is not a good idea to look at the Qt source, it can be confusing. Rather assume that what Qt gives you work, and first try to answer "What I am doing wrong?". As I said drawing in a mouse event is definitely wrong.
Your description is really subjective, maybe an image of your output is better. Are you trying to emulate a pen (like in windows paint)? In this case do the mouse button has to be down ? is that the purpose of your variable scribbling?
There is more. following the documentation, QMouseEvent::buttons() always return a combination of all buttons for mouse move event. Which make sense : the mouse movements are independent of the buttons. It means
if ((event->buttons() & Qt::LeftButton)
will always be true.
Let's assume you want to draw the path of your mouse when the left button is pressed. Then you use something like :
void myPaint::mousePressEvent(QMouseEvent *event){
scribbling = true;
pixelList.clear();
}
void myPaint::mouseReleaseEvent(QMouseEvent *event){
scribbling = false;
}
void myPaint::mouseMoveEvent(QMouseEvent *event) {
if ( scribbling) {
pixelList.append(event->pos());
}
}
void myPaint::paintEvent(){
QPainter painter(this)
//some painting here
if ( scribbling) {
painter.setRenderHint(QPainter::Antialiasing);
painter.setPen(QPen(Qt::blue, myPenWidth, Qt::SolidLine, Qt::RoundCap,Qt::RoundJoin));
// here draw your path
// for example if your path can be made of lines,
// or you just put the points if they are close from each other
}
//other painting here
}
If after all of this you don't have a good rendering, try using float precision (slower), ie QMouseEvent::posF() instead of QMouseEvent::pos().
EDIT :
"I want to know whether there is any way to calculate all the sub-pixels between any two pixels that we send as arguments to drawLine"
Yes there is. I don't know why you need to do such thing but is really simple. A line can be characterized with the equation
y = ax + b
Both of the endpoints of the line p0 = (x0, y0) and p1 = (x1, y1) satisfy this equation so you can easily find a and b. Now all you need to do is increment from x0 to x1 by the amount of
pixels you want (say 1), and to compute the corresponding y value, each time saving point(x,y).
So will go over all of the points saved in pixelList and repeat this process for any two consecutive points.

Resources