QGraphicsScene & OpenGL Fragment Shader Not Working - qt

Question

I have a very large QGraphicsScene that can contain a very large number of graphics. I'm using a QGLWidget as the viewport so that I can leverage OpenGL to try to improve how some things get rendered. I have created a custom QGraphicsItem that I can use to draw several quads with the same texture in one render call rather than having hundreds or thousands of different QGraphicsItems in the scene that really all get drawn the same way, just in different locations. In my custom QGraphicsItem's paint() method, I called beginNativePainting() and endNativePainting() and do all of my OpenGL calls between them.
I want to use shader programs so that I can manipulate the vertices somewhat within the vertex shader, so I copied Qt's OpenGL Textures Example which uses a shader program to draw 6 textured quads. That example works just fine as is, but when I try to use the same approach within a QGraphicsItem's paint() method, all of my quads just get drawn white. My best guess is that my fragment shader just isn't getting used. I've even tried hardcoding the color within the fragment shader and nothing changes.
Here's the source code of my custom QGraphicsItem class.
class BatchGraphics : public QGraphicsPixmapItem
{
enum {PROGRAM_VERTEX_ATTRIBUTE = 0,
PROGRAM_TEXCOORD_ATTRIBUTE = 1};
public:
BatchGraphics()
: _program(0),
_texture(0),
_dirty(false)
{
}
// Returns the custom bounding rect for this item which encompasses all quads
QRectF boundingRect() const
{
return _boundingRect;
}
// Add a quad to the batch. Only the center point is necessary
void addQuad(int id, float x, float y)
{
_quads.insert(id, QPointF(x, y));
updateBoundingRect();
_dirty = true;
}
// Remove a quad from the batch.
void removeQuad(int id)
{
if (_quads.contains(id))
{
_quads.remove(id);
updateBoundingRect();
_dirty = true;
}
}
// Return the number of quads in the batch
int count() {return _quads.count();}
// Paint the batch using a custom implementation.
void paint(QPainter *painter, const QStyleOptionGraphicsItem *option, QWidget *widget)
{
// If the item is dirty (has been modified, update the geometry)
if (_dirty) {
updateGeometry();
}
if (_program)
{
painter->beginNativePainting();
// Enable GL states
//glEnable(GL_TEXTURE_2D);
// Set the MVP matrix
_program->setUniformValue("matrix", painter->transform());
// Enable and set the vertex and texture attributes
_program->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
_program->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
_program->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, GL_FLOAT, _vertices.constData(), 3, 5*sizeof(GLfloat));
_program->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, GL_FLOAT, _vertices.constData()+2, 2, 5*sizeof(GLfloat));
// Bind the texture
_texture->bind();
// Draw the arrays
glDrawArrays(GL_TRIANGLES, 0, _quads.count()*6); // 6 vertices per quad
painter->endNativePainting();
}
}
private:
// Initialize the shader and texture
void initialize()
{
// Create the OpenGL texture
_texture = new QOpenGLTexture(pixmap().toImage());
// Vertex Shader
QOpenGLShader *vshader = new QOpenGLShader(QOpenGLShader::Vertex);
const char *vsrc =
"attribute highp vec4 vertex;\n"
"attribute mediump vec4 texCoord;\n"
"varying mediump vec4 texc;\n"
"uniform mediump mat4 matrix;\n"
"void main(void)\n"
"{\n"
" gl_Position = matrix * vertex;\n"
" texc = texCoord;\n"
"}\n";
vshader->compileSourceCode(vsrc);
// Fragment Shader
QOpenGLShader *fshader = new QOpenGLShader(QOpenGLShader::Fragment);
const char *fsrc =
"uniform sampler2D texture;\n"
"varying mediump vec4 texc;\n"
"void main(void)\n"
"{\n"
" gl_FragColor = texture2D(texture, texc.st);\n"
"}\n";
fshader->compileSourceCode(fsrc);
// Program
_program = new QOpenGLShaderProgram;
_program->addShader(vshader);
_program->addShader(fshader);
_program->bindAttributeLocation("vertex", PROGRAM_VERTEX_ATTRIBUTE);
_program->bindAttributeLocation("texCoord", PROGRAM_TEXCOORD_ATTRIBUTE);
_program->link();
_program->bind();
_program->setUniformValue("texture", 0);
}
// Update the vertex array. Calls initialize the first time.
void updateGeometry()
{
if (_program == 0) {
initialize();
}
_vertices.clear();
// Half pixmap size
QPointF s = QPointF(pixmap().width()/2, pixmap().height()/2);
// Build vertex data for each quad
foreach (const QPointF& point, _quads)
{
// Top Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 1; // tv
// Top Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 1; // tv
// Bottom Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 0; // tv
// Top Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 1; // tv
// Bottom Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 0; // tv
// Bottom Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 0; // tv
}
_dirty = false;
}
private:
// Updates the bounding rect based on the quads in the batch.
void updateBoundingRect()
{
prepareGeometryChange();
double left = 9999;
double right = -9999;
double top = 9999;
double bottom = -9999;
double w = pixmap().width()/2;
double h = pixmap().width()/2;
foreach (const QPointF& p, _quads)
{
left = qMin(left, p.x()-w);
right = qMax(right, p.x()+w);
top = qMin(top, p.y()-h);
bottom = qMax(bottom, p.y()+h);
}
_boundingRect = QRectF(left, top, (right-left), (bottom-top));
}
private:
QOpenGLShaderProgram* _program;
QOpenGLTexture* _texture;
QRectF _boundingRect;
QMap<int, QPointF> _quads;
QVector<GLfloat> _vertices;
bool _dirty;
};
I understand the basics of the render pipeline and how to use shaders, but as far as dependencies between things and other OpenGL methods that must be called when using certain features I'm pretty clueless on. I can get the quads to be rendered with the texture using a fixed function pipeline approach, but that's old school and like I said, I want to be able to manipulate the vertices in the vertex shader once I get this working.
I'm not doing anything special when creating the QGLWidget, and its QGLFormat ends up being 2.0. I'v also tried calling glEnable(GL_TEXTURE_2D), but that just makes the quads get rendered black instead of white. I've also tried binding the program each time paint() is called, thinking perhaps Qt is binding a different shader somewhere else behind the scenes, but that just causes NOTHING to appear.
Can anyone provide any help please? I can't figure out why this approach works fine in Qt's Textures example but not when I try to do it inside of a QGraphicsItem.

Answer 1

I finally figured it out after looking at Qt's source code and what happens when beginNativePainting(). First, I DID have to bind my shader each time paint() was called, and second I had to get the correct MVP matrix.
I was trying to just pass the QPainter's transform to my shader to act as the modelview projection matrix, but the transform was only the modelview matrix. I needed to get the projection matrix as well, which Qt sets when beginNativePainting() is called.
I got the project and modelview matrices from OpenGL directly and combined them to pass to my shader after binding my texture and presto! It worked!
Here are the relevant changes I had to make:
painter->beginNativePainting();
// Enable GL states
//glEnable(GL_TEXTURE_2D);
// === Begin New Code ======
// Bind my program
_program->bind();
QMatrix4x4 proj;
glGetFloatv(GL_PROJECTION_MATRIX, proj.data());
QMatrix4x4 model;
glGetFloatv(GL_MODELVIEW_MATRIX, model.data());
// Set the MVP matrix
_program->setUniformValue("matrix", proj * model);
// === End New Code ======
// Enable and set the vertex and texture attributes
_program->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
_program->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
_program->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, GL_FLOAT, _vertices.constData(), 3, 5*sizeof(GLfloat));
_program->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, GL_FLOAT, _vertices.constData()+2, 2, 5*sizeof(GLfloat));