How to save a frame using QMediaPlayer? - qt

I want to save an image of a frame from a QMediaPlayer. After reading the documentation, I understood that I should use QVideoProbe. I am using the following code :
QMediaPlayer *player = new QMediaPlayer();
QVideoProbe *probe = new QVideoProbe;
connect(probe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(processFrame(QVideoFrame)));
qDebug()<<probe->setSource(player); // Returns true, hopefully.
player->setVideoOutput(myVideoSurface);
player->setMedia(QUrl::fromLocalFile("observation.mp4"));
player->play(); // Start receving frames as they get presented to myVideoSurface
But unfortunately, probe->setSource(player) always returns false for me, and thus my slot processFrame is not triggered.
What am I doing wrong ? Does anybody have a working example of QVideoProbe ?

You're not doing anything wrong. As #DYangu pointed out, your media object instance does not support monitoring video. I had the same problem (and same for QAudioProbe but it doesn't interest us here). I found a solution by looking at this answer and this one.
The main idea is to subclass QAbstractVideoSurface. Once you've done that, it will call the method QAbstractVideoSurface::present(const QVideoFrame & frame) of your implementation of QAbstractVideoSurface and you will be able to process the frames of your video.
As it is said here, usually you will just need to reimplement two methods :
supportedPixelFormats so that the producer can select an appropriate format for the QVideoFrame
present which allows to display the frame
But at the time, I searched in the Qt source code and happily found this piece of code which helped me to do a full implementation. So, here is the full code for using a "video frame grabber".
VideoFrameGrabber.cpp :
#include "VideoFrameGrabber.h"
#include <QtWidgets>
#include <qabstractvideosurface.h>
#include <qvideosurfaceformat.h>
VideoFrameGrabber::VideoFrameGrabber(QWidget *widget, QObject *parent)
: QAbstractVideoSurface(parent)
, widget(widget)
, imageFormat(QImage::Format_Invalid)
{
}
QList<QVideoFrame::PixelFormat> VideoFrameGrabber::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
Q_UNUSED(handleType);
return QList<QVideoFrame::PixelFormat>()
<< QVideoFrame::Format_ARGB32
<< QVideoFrame::Format_ARGB32_Premultiplied
<< QVideoFrame::Format_RGB32
<< QVideoFrame::Format_RGB24
<< QVideoFrame::Format_RGB565
<< QVideoFrame::Format_RGB555
<< QVideoFrame::Format_ARGB8565_Premultiplied
<< QVideoFrame::Format_BGRA32
<< QVideoFrame::Format_BGRA32_Premultiplied
<< QVideoFrame::Format_BGR32
<< QVideoFrame::Format_BGR24
<< QVideoFrame::Format_BGR565
<< QVideoFrame::Format_BGR555
<< QVideoFrame::Format_BGRA5658_Premultiplied
<< QVideoFrame::Format_AYUV444
<< QVideoFrame::Format_AYUV444_Premultiplied
<< QVideoFrame::Format_YUV444
<< QVideoFrame::Format_YUV420P
<< QVideoFrame::Format_YV12
<< QVideoFrame::Format_UYVY
<< QVideoFrame::Format_YUYV
<< QVideoFrame::Format_NV12
<< QVideoFrame::Format_NV21
<< QVideoFrame::Format_IMC1
<< QVideoFrame::Format_IMC2
<< QVideoFrame::Format_IMC3
<< QVideoFrame::Format_IMC4
<< QVideoFrame::Format_Y8
<< QVideoFrame::Format_Y16
<< QVideoFrame::Format_Jpeg
<< QVideoFrame::Format_CameraRaw
<< QVideoFrame::Format_AdobeDng;
}
bool VideoFrameGrabber::isFormatSupported(const QVideoSurfaceFormat &format) const
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();
return imageFormat != QImage::Format_Invalid
&& !size.isEmpty()
&& format.handleType() == QAbstractVideoBuffer::NoHandle;
}
bool VideoFrameGrabber::start(const QVideoSurfaceFormat &format)
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();
if (imageFormat != QImage::Format_Invalid && !size.isEmpty()) {
this->imageFormat = imageFormat;
imageSize = size;
sourceRect = format.viewport();
QAbstractVideoSurface::start(format);
widget->updateGeometry();
updateVideoRect();
return true;
} else {
return false;
}
}
void VideoFrameGrabber::stop()
{
currentFrame = QVideoFrame();
targetRect = QRect();
QAbstractVideoSurface::stop();
widget->update();
}
bool VideoFrameGrabber::present(const QVideoFrame &frame)
{
if (frame.isValid())
{
QVideoFrame cloneFrame(frame);
cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
const QImage image(cloneFrame.bits(),
cloneFrame.width(),
cloneFrame.height(),
QVideoFrame::imageFormatFromPixelFormat(cloneFrame .pixelFormat()));
emit frameAvailable(image); // this is very important
cloneFrame.unmap();
}
if (surfaceFormat().pixelFormat() != frame.pixelFormat()
|| surfaceFormat().frameSize() != frame.size()) {
setError(IncorrectFormatError);
stop();
return false;
} else {
currentFrame = frame;
widget->repaint(targetRect);
return true;
}
}
void VideoFrameGrabber::updateVideoRect()
{
QSize size = surfaceFormat().sizeHint();
size.scale(widget->size().boundedTo(size), Qt::KeepAspectRatio);
targetRect = QRect(QPoint(0, 0), size);
targetRect.moveCenter(widget->rect().center());
}
void VideoFrameGrabber::paint(QPainter *painter)
{
if (currentFrame.map(QAbstractVideoBuffer::ReadOnly)) {
const QTransform oldTransform = painter->transform();
if (surfaceFormat().scanLineDirection() == QVideoSurfaceFormat::BottomToTop) {
painter->scale(1, -1);
painter->translate(0, -widget->height());
}
QImage image(
currentFrame.bits(),
currentFrame.width(),
currentFrame.height(),
currentFrame.bytesPerLine(),
imageFormat);
painter->drawImage(targetRect, image, sourceRect);
painter->setTransform(oldTransform);
currentFrame.unmap();
}
}
VideoFrameGrabber.h
#ifndef VIDEOFRAMEGRABBER_H
#define VIDEOFRAMEGRABBER_H
#include <QtWidgets>
class VideoFrameGrabber : public QAbstractVideoSurface
{
Q_OBJECT
public:
VideoFrameGrabber(QWidget *widget, QObject *parent = 0);
QList<QVideoFrame::PixelFormat> supportedPixelFormats(
QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const;
bool isFormatSupported(const QVideoSurfaceFormat &format) const;
bool start(const QVideoSurfaceFormat &format);
void stop();
bool present(const QVideoFrame &frame);
QRect videoRect() const { return targetRect; }
void updateVideoRect();
void paint(QPainter *painter);
private:
QWidget *widget;
QImage::Format imageFormat;
QRect targetRect;
QSize imageSize;
QRect sourceRect;
QVideoFrame currentFrame;
signals:
void frameAvailable(QImage frame);
};
#endif //VIDEOFRAMEGRABBER_H
Note : in the .h, you will see I added a signal taking an image as a parameter. This will allow you to process your frame anywhere in your code. At the time, this signal took a QImage as a parameter, but you can of course take a QVideoFrame if you want to.
Now, we are ready to use this video frame grabber:
QMediaPlayer* player = new QMediaPlayer(this);
// no more QVideoProbe
VideoFrameGrabber* grabber = new VideoFrameGrabber(this);
player->setVideoOutput(grabber);
connect(grabber, SIGNAL(frameAvailable(QImage)), this, SLOT(processFrame(QImage)));
Now you just have to declare a slot named processFrame(QImage image) and you will receive a QImage each time you will enter the method present of your VideoFrameGrabber.
I hope that this will help you!

After Qt QVideoProbe documentation:
bool QVideoProbe::setSource(QMediaObject *mediaObject)
Starts monitoring the given mediaObject.
If there is no media object associated with mediaObject, or if it is
zero, this probe will be deactivated and this function wil return
true.
If the media object instance does not support monitoring video, this
function will return false.
Any previously monitored objects will no longer be monitored. Passing
in the same object will be ignored, but monitoring will continue.
So it seems your "media object instance does not support monitoring video"

TL;DR: https://gist.github.com/JC3/a7bab65acbd7659d1e57103d2b0021ba (only file)
I had a similar issue (5.15.2; although in my case I was on Windows, was definitely using the DirectShow back-end, the probe attachment was returning true, the sample grabber was in the graph, but the callback wasn't firing).
I never figured it out but needed to get something working so I kludged one out of a QAbstractVideoSurface, and it's been working well so far. It's a bit simpler than some of the other implementations in this post, and it's all in one file.
Note that Qt 5.15 or higher is required if you intend to both process frames and play them back with this, since the multi-surface QMediaPlayer::setVideoOutput wasn't added until 5.15. If all you want to do is process video you can still use the code below as a template for pre-5.15, just gut the formatSource_ parts.
Code:
VideoProbeSurface.h (the only file; link is to Gist)
#ifndef VIDEOPROBESURFACE_H
#define VIDEOPROBESURFACE_H
#include <QAbstractVideoSurface>
#include <QVideoSurfaceFormat>
class VideoProbeSurface : public QAbstractVideoSurface {
Q_OBJECT
public:
VideoProbeSurface (QObject *parent = nullptr)
: QAbstractVideoSurface(parent)
, formatSource_(nullptr)
{
}
void setFormatSource (QAbstractVideoSurface *source) {
formatSource_ = source;
}
QList<QVideoFrame::PixelFormat> supportedPixelFormats (QAbstractVideoBuffer::HandleType type) const override {
return formatSource_ ? formatSource_->supportedPixelFormats(type)
: QList<QVideoFrame::PixelFormat>();
}
QVideoSurfaceFormat nearestFormat (const QVideoSurfaceFormat &format) const override {
return formatSource_ ? formatSource_->nearestFormat(format)
: QAbstractVideoSurface::nearestFormat(format);
}
bool present (const QVideoFrame &frame) override {
emit videoFrameProbed(frame);
return true;
}
signals:
void videoFrameProbed (const QVideoFrame &frame);
private:
QAbstractVideoSurface *formatSource_;
};
#endif // VIDEOPROBESURFACE_H
I went for the quickest-to-write implementation possible so it just forwards supported pixel formats from another surface (my intent was to both probe and play back to a QVideoWidget) and you get whatever format you get. I just needed to grab subimages into QImages though, which handles most common formats. But you could modify this to force any formats you want (e.g. you might want to just return formats supported by QImage or filter out source formats not supported by QImage), etc.).
Example set up:
QMediaPlayer *player = ...;
QVideoWidget *widget = ...;
// forward surface formats provided by the video widget:
VideoProbeSurface *probe = new VideoProbeSurface(...);
probe->setFormatSource(widget->videoSurface());
// same signal signature as QVideoProbe's signal:
connect(probe, &VideoProbeSurface::videoFrameProbed, ...);
// the key move is to render to both the widget (for viewing)
// and probe (for processing). fortunately, QMediaPlayer can
// take a list:
player->setVideoOutput({ widget->videoSurface(), probe });
Notes
The only really sketchy thing I had to do was const_cast the QVideoFrame on the receiver side (for read-only access), since QVideoFrame::map() isn't const:
if (const_cast<QVideoFrame&>(frame).map(QAbstractVideoBuffer::ReadOnly)) {
...;
const_cast<QVideoFrame&>(frame).unmap();
}
But the real QVideoProbe would make you do the same thing so, I don't know what's up with that -- it's a strange API. I ran some tests with sw, native hw, and copy-back hw renderers and decoders and map/unmap in read mode seem to be functioning OK, so, whatever.
Performance-wise, the video will bog down if you spend too much time in the callback, so design accordingly. However, I didn't test QueuedConnection, so I don't know if that'd still have the issue (although the fact that the signal parameter is a reference would make me wary of trying it, as well as conceivable issues with the GPU releasing the memory before the slot ends up being called). I don't know how QVideoProbe behaves in this regard, either. I do know that, at least on my machine, I can pack and queue Full HD (1920 x 1080) resolution QImages to a thread pool for processing without slowing down the video.
You probably also want to implement some sort of auto-unmapper utility object for exception safe unmap(), etc. But again, that's not unique to this, same thing you'd have to do with QVideoProbe.
So hopefully that helps somebody else.
Example QImage Use
PS, example of packing arbitrarily-formatted QVideoFrames into a QImage in:
void MyVideoProcessor::onFrameProbed(const QVideoFrame &frame) {
if (const_cast<QVideoFrame&>(frame).map(QAbstractVideoBuffer::ReadOnly)) {
auto imageFormat = QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat());
QImage image(frame.bits(), frame.width(), frame.height(), frame.bytesPerLine(), imageFormat);
// *if* you want to use this elsewhere you must force detach:
image = image.copy();
// but if you don't need to use it past unmap(), you can just
// use the original image instead of a copy.
// <---- now do whatever with the image, e.g. save() it.
// if you *haven't* copied the image, then, before unmapping,
// kill any internal data pointers just to be safe:
image = QImage();
const_cast<QVideoFrame&>(frame).unmap();
}
}
Notes about that:
Constructing a QImage directly from the data is fast and essentially free: no copies are done.
The data buffers are only technically valid between map and unmap, so if you intend to use the QImage outside of that scope, you'll want to use copy() (or anything else that forces a detach) to force a deep copy.
You also probably want to ensure that the original not-copied QImage is destructed before calling unmap. It's unlikely to cause problems but it's always a good idea to minimize how many invalid pointers are hanging around at any given time, and also the QImage docs say "The buffer must remain valid throughout the life of the QImage and all copies that have not been modified or otherwise detached from the original buffer". Best to be strict about it.

Related

More than one QGLWidget simultaneous doesn't work for a video usage

I have a problem if I want to display a video using QGLWidget. With one instance it works, but it doesn't and widgets are black with multi-occurrence (usage in QGridLayout for example).
I subclass QGLWidget that way to play a video (by refreshing the QPixmap I want to show using my setCurrentImage method):
class GLWidget : public QGLWidget
{
Q_OBJECT
public:
GLWidget(QGLWidget* shareWidget = 0, QWidget* parent = 0)
: QGLWidget(parent, shareWidget)
{
//...
// I do this because the QPixmap to refresh is produced in a different thread than UI thread.
connect(this, SIGNAL(updated()), this SLOT(update()));
}
//...
void setCurrentImage(const QPixmap& pixmap)
{
m_mutex.lock();
m_pixmap = pixmap;
m_mutex.unlock();
emit updated();
}
protected:
void initializeGL()
{
static const int coords[4][2] = {{ +1, -1 }, { -1, -1 }, { -1, +1 }, { +1, +1 }};
for(int j = 0; j < 4; ++j){
m_texCoords.append(QVector2D(j == 0 || j == 3, j == 0 || j == 1));
m_vertices.append(QVector2D(0.5 * coords[j][0], 0.5 * coords[j][1]));
}
glEnable(GL_DEPTH_TEST);
glEnable(GL_CULL_FACE);
glEnable(GL_TEXTURE_2D);
#define PROGRAM_VERTEX_ATTRIBUTE 0
#define PROGRAM_TEXCOORD_ATTRIBUTE 1
QGLShader* vShader = new QGLShader(QGLShader::Vertex, this);
const char* szVShaderCode = /*...*/;
vShader->compileSourceCode(szVShaderCode);
QGLShader *fShader = new QGLShader(QGLShader::Fragment, this);
const char* szFShaderCode = /*...*/;
fShader->compileSourceCode(szFShaderCode);
m_pProgram = new QGLShaderProgram(this);
m_pProgram->addShader(vShader);
m_pProgram->addShader(fShader);
m_pProgram->bindAttributeLocation("vertex", PROGRAM_VERTEX_ATTRIBUTE);
m_pProgram->bindAttributeLocation("texCoord", PROGRAM_TEXCOORD_ATTRIBUTE);
m_pProgram->link();
m_pProgram->bind();
m_pProgram->setUniformValue("texture", 0);
}
void paintGL()
{
qglClearColor(Qt::black);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// I use some uniform values to "modify" image
m_pProgram->setUniformValue(/*...*/);
//...
QMatrix4x4 m;
m.ortho(-0.5f, +0.5f, +0.5f, -0.5f, 4.0f, 15.0f);
m.translate(0.0f, 0.0f, -10.0f);
m_pProgram->setUniformValue("matrix", m);
m_pProgram->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
m_pProgram->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
m_pProgram->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, m_vertices.constData());
m_pProgram->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, m_texCoords.constData());
glBindTexture(GL_TEXTURE_2D, m_texture);
glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
}
void resizeGL(int iWidth, int iHeight)
{
glViewport(0, 0, iWidth, iHeight);
}
private slots:
void update()
{
QPixmap pixmap;
m_mutex.lock();
pixmap = m_pixmap;
m_mutex.unlock();
m_texture = bindTexture(pixmap, GL_TEXTURE_2D);
updateGL();
}
private:
QMutex m_mutex;
QPixmap m_pixmap;
QVector<QVector2D> m_vertices;
QVector<QVector2D> m_texCoords;
QGLShaderProgram* m_pProgram;
};
Then, to simulate a multi thread rendering (each thread produces its own video images), I wrote these little classes.
The image producer class:
class ImageProducer : public QThread
{
Q_OBJECT
public:
ImageProducer(QGLWidget* pGLWidget)
: QThread(pGLWidget), m_pGLWidget(pGLWidget)
{
m_pixmap = QPixmap("fileName.jpg");
m_bMustStop = false;
}
protected:
void run()
{
while(!m_bMustStop){
static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(m_pixmap);
// Simulate a frame rate
msleep(1000 / /*FRAME_RATE*/);
}
}
private:
QGLWidget* m_pGLWidget;
QPixmap m_pixmap;
bool m_bMustStop;
};
And the rendering class:
#define ROWS 1
#define COLS 1
class Window : public QWidget
{
public:
Window()
{
QGridLayout* pMainLayout = new QGridLayout(this);
setLayout(pMainLayout);
for(int i = 0; i < ROWS; ++i){
for(int j = 0; j < COLS; ++j){
QGLWidget* pGLWidget = new GLWidget();
pMainLayout->addWidget(pGLWidget, i, j);
ImageProducer* pImageProducer = new ImageProducer(pGLWidget);
pImageProducer->start();
}
}
}
};
Ok stop with code samples ^^ Problem is with ROWS = 1 and COLS = 1 (see Window class) it works, but I have black widgets with other values... I'm lost, what do I miss?
Thanks!
EDIT: (context is always multi-GLWidget instance, all works fine with only one)
Strange thing I just discover: I overrode QGLWidget's mouseMoveEvent (so, in my GLWidget class) which simply calls updateGL();. And fact is when I press and move the mouse with the current code nothing happens. But if I replace (in my ImageProducer's run() method):
static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(m_pixmap);
By
static_cast<GLWidget*>(m_pGLWidget)->setCurrentImage(QPixmap("fileName.jpg"));
The image is refreshed in the current component as long as I move the mouse. When I release it, or do it in an other components, the background become black again.
I am writing down the answer after a chat with the OP, for the benefit of others. Here are the main topics that lead to a resolution of the issue.
Note: At the time of writing, the Qt OpenGL module is deprecated and its usage is discouraged. According to the official documentation the suggested approach is to use the OpenGL* classes in the GUI module.
Textures
The main issue with the code above is essentially due to the way the textures are handled.
First of all, the bindTexture() method does not bind in OpenGL terms, but actually create the OpenGL texture and upload the data passed as argument to it (QImage or QPixmap). That is confusing and can lead to serious issues. In the code posted by the OP, he is essentially leaking memory allocating a new texture at each frame update.
To ensure you are not leaking memory, you should at the least release the previously allocated texture, calling QGLWidget::deleteTexture.
However, it is worth to note that there are better approaches to avoid unnecessary memory fragmentation and inefficiency. In this case, the best approach is to allocate the texture once and simply update its content when necessary. This may not be possible directly with the API offered by the old Qt OpenGL module, but you can always mix Qt and native OpenGL code.
Context handling
Another issue is the way OpenGL context is handled. As a rule of thumb, one should always ensure the correct context is current before issuing commands to the OpenGL implementation. Sometimes, Qt does this for you automatically (i.e. before calling the paintGL() method).
In this case, we need to explicitly make the QGLWidget's context the current context before calling the bindTexture method, otherwise it will effect the last context that was made current. That is why only the last widget was showing something untill somewhone was triggering a makeCurrent call by interacting with the other widgets.
Threading
There are a couple of issues here. First of all, it is not safe to use a QPixmap object in a thread other than the GUI thread. QPixmap is designed to optimize pixmap blitting on screen. In this case, the pixmap is really just the frame to be uploaded to the OpenGL implementation, which handles all the rendering. So it is safe to use a QImage instead.
The other issue is that the GLWidget::setCurrentImage() and thus, the bindTexture method, are called directly from the run() method of the ImageProducer thread. This can't be because we need to make the widget context current, but it is not possible to call makeCurrent() from a thread other than the GUI thread (more details here).
A possible approach to solve this issue is to add a signal to ImageProducer to notify the frame has been updated, and connect this signal to the setCurrentImage() slot. Qt's signal-slot mechanism will ensure that the setCurrentImage is executed in the GLWidget thread, that is the GUI thread.
Code
Here are the suggested (and tested) modifications to the code posted above.
ImageProduer class
add the signal:
void imageUpdated(const QImage &image);
and emit it when the frame is updated:
void ImageProducer::run()
{
while(!m_bMustStop){
QImage frame(m_pixmap.width(), m_pixmap.height(), m_pixmap.format());
QPainter painter(&frame);
painter.drawImage(QPoint(), m_pixmap);
painter.setFont(QFont("sans-serif", 22));
painter.setPen(Qt::white);
painter.drawText(20, 50, QString("Frame: %1").arg(QString::number(_frameCount)));
painter.end();
emit imageUpdated(frame);
msleep(1000 / FRAME_RATE);
_frameCount++;
}
}
GLWidget class
ensure the setCurrentImage method handle the OpenGL context and destroy the old texture:
void GLWidget::setCurrentImage(const QImage& pixmap)
{
m_mutex.lock();
m_pixmap = pixmap;
m_mutex.unlock();
makeCurrent();
deleteTexture(m_texture);
m_texture = bindTexture(pixmap, GL_TEXTURE_2D);
doneCurrent();
emit updated();
}

How to render programmatically a vtk item in qml?

So far, I understand that we have two threads in QML, our main application thread, and our "scene graph" thread : http://doc.qt.io/qt-5/qtquick-visualcanvas-scenegraph.html
I've implemented my own vtkQmlItem with the help of this link : http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html
and I've noticed that my vtkscene is only rendered when the afterrendering signal is emitted by the qml flow.
So far, everything is ok and works perfectly, I can see my vtk scene and can even interract with it.
But I would like to also programmatically render my vtk scene as well, since I want to do an animation by moving the camera around a vtk object.
Calling renderer->render() directly shows a lot of vtk error, and does not seem to be the good way to do this.
Calling this->window()->update() seems to put the event in the eventLoop, when I want it to be handled instantly. The only way I've managed to make it work instantly is by using QApplication::processEvents(), which is a hack I don't like and would love another solution.
So the pseudocode of the working solution that I don't like is the following :
for (int i = 0; i < 50; i++)
{
ChangeCameraPosition(i); // Change the position and orientation of the vtk camera
this->window()->update();
QApplication::processEvents(); // The hack I don't like
QThread::msleep(500);
}
For people looking for a solution for this using Qt QuickControls 2 and VTK 8, you can find a working example in this repository https://github.com/nicanor-romero/QtVtk with building instructions in the README.
the problem is actually a bit complicated and if nothing changed in the past few months, there is still no support for QtQuick in VTK, which means no simple few lines solution. You can find support classes for QtWidgets in VTK/GUISupport/QtOpenGL/ and use them as a template to derive support for qml. But mainly I recommend checking this thread for a discussion about this topic.
The key point is that QtQuick holds the openGL context for the qml window you are trying to render into in a dedicated thread and it won't let anything else get that context. So in order to render into it from VTK, you have to do it within that thread. This means:
1) Create your own vtkRenderWindow that overrides the Render() method such that it is ensured it happens in the qml's render thread.
2) Make that render window render into a framebuffer object provided by the qtquick (instance of QQuickFramebufferObject).
3) Interconnect vtk's rendering signals with the qt's rendering methods -> e.g. when the vtk render window calls makeCurrent, the qt's rendering thread "wakes up".
Here is my implementation based on Taylor Braun-Jones' template linked above. It might not be perfect, but it works for me (I have removed some parts specific to my app so it might not compile straight away, but it should put you on a path to some working solution):
qmlVtk.h:
#include <vtkEventQtSlotConnect.h>
#include <vtkGenericOpenGLRenderWindow.h>
#include <vtkRenderer.h>
#include <QtQuick/QQuickFramebufferObject>
// Use the OpenGL API abstraction from Qt instead of from VTK because vtkgl.h
// and other Qt OpenGL-related headers do not play nice when included in the
// same compilation unit
#include <QOpenGLFunctions>
#include <qqmlapplicationengine.h>
class QVTKFramebufferObjectRenderer;
class QVTKInteractorAdapter;
class vtkInternalOpenGLRenderWindow;
class QVTKFramebufferObjectRenderer;
class QVTKFrameBufferObjectItem : public QQuickFramebufferObject
{
Q_OBJECT
public:
QVTKFrameBufferObjectItem(QQuickItem *parent = 0);
~QVTKFrameBufferObjectItem();
Renderer *createRenderer() const;
vtkSmartPointer<vtkInternalOpenGLRenderWindow> GetRenderWindow() const;
protected:
// Called once before the FBO is created for the first time. This method is
// called from render thread while the GUI thread is blocked.
virtual void init();
vtkSmartPointer<vtkInternalOpenGLRenderWindow> m_win;
QVTKInteractorAdapter* m_irenAdapter;
vtkSmartPointer<vtkEventQtSlotConnect> mConnect;
friend class QVTKFramebufferObjectRenderer;
// Convert the position of the event from openGL coordinate to native coordinate
QMouseEvent openGLToNative(QMouseEvent const& event);
virtual void mouseMoveEvent(QMouseEvent * event);
virtual void mousePressEvent(QMouseEvent * event);
virtual void mouseReleaseEvent(QMouseEvent * event);
virtual void mouseDoubleClickEvent(QMouseEvent * event);
virtual void wheelEvent(QWheelEvent *event);
virtual void keyPressEvent(QKeyEvent* event);
virtual void keyReleaseEvent(QKeyEvent* event);
virtual void focusInEvent(QFocusEvent * event);
virtual void focusOutEvent(QFocusEvent * event);
protected Q_SLOTS:
// slot to make this vtk render window current
virtual void MakeCurrent();
// slot called when vtk wants to know if the context is current
virtual void IsCurrent(vtkObject* caller, unsigned long vtk_event, void* client_data, void* call_data);
// slot called when vtk wants to start the render
virtual void Start();
// slot called when vtk wants to end the render
virtual void End();
// slot called when vtk wants to know if a window is direct
virtual void IsDirect(vtkObject* caller, unsigned long vtk_event, void* client_data, void* call_data);
// slot called when vtk wants to know if a window supports OpenGL
virtual void SupportsOpenGL(vtkObject* caller, unsigned long vtk_event, void* client_data, void* call_data);
};
/// <summary>
/// An extension of vktGenericOpenGLRenderWindow to work with Qt.
/// Serves to write VTK-generated render calls to a framebuffer provided and maintained by Qt. It is meant to be used within Qt render loop, i.e. using Qt's render thread.
/// </summary>
/// <seealso cref="vtkGenericOpenGLRenderWindow" />
/// <seealso cref="QOpenGLFunctions" />
class vtkInternalOpenGLRenderWindow : public vtkGenericOpenGLRenderWindow, protected QOpenGLFunctions
{
public:
static vtkInternalOpenGLRenderWindow* New();
vtkTypeMacro(vtkInternalOpenGLRenderWindow, vtkGenericOpenGLRenderWindow)
virtual void OpenGLInitState();
// Override to use deferred rendering - Tell the QSG that we need to
// be rendered which will then, at the appropriate time, call
// InternalRender to do the actual OpenGL rendering.
virtual void Render();
// Do the actual OpenGL rendering
void InternalRender();
// Provides a convenient way to set the protected FBO ivars from an existing
// FBO that was created and owned by Qt's FBO abstraction class
// QOpenGLFramebufferObject
void SetFramebufferObject(QOpenGLFramebufferObject *fbo);
QVTKFramebufferObjectRenderer *QtParentRenderer;
protected:
vtkInternalOpenGLRenderWindow();
~vtkInternalOpenGLRenderWindow()
{
// Prevent superclass destructors from destroying the framebuffer object.
// QOpenGLFramebufferObject owns the FBO and manages it's lifecyle.
this->OffScreenRendering = 0;
}
};
qmlVtk.cpp:
#include "QVTKFramebufferObjectItem.h"
#include <QQuickFramebufferObject>
#include <QQuickWindow>
#include <QOpenGLFramebufferObject>
#include <QVTKInteractorAdapter.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkObjectFactory.h>
#include <vtkSmartPointer.h>
#include <vtkCamera.h>
#include <vtkProperty.h>
#include <qglfunctions.h>
class QVTKFramebufferObjectRenderer : public QQuickFramebufferObject::Renderer
{
friend class vtkInternalOpenGLRenderWindow;
public:
QVTKFramebufferObjectRenderer(vtkSmartPointer<vtkInternalOpenGLRenderWindow> rw) :
m_framebufferObject(0)
{
m_vtkRenderWindow = rw;
m_vtkRenderWindow->QtParentRenderer = this;
}
~QVTKFramebufferObjectRenderer()
{
m_vtkRenderWindow->QtParentRenderer = 0;
glFrontFace(GL_CCW); // restore default settings
}
virtual void synchronize(QQuickFramebufferObject * item)
{
// the first synchronize call - right before the the framebufferObject
// is created for the first time
if (!m_framebufferObject)
{
QVTKFrameBufferObjectItem *vtkItem = static_cast<QVTKFrameBufferObjectItem*>(item);
vtkItem->init();
}
}
virtual void render()
{
m_vtkRenderWindow->InternalRender(); // vtkXOpenGLRenderWindow renders the scene to the FBO
}
QOpenGLFramebufferObject *createFramebufferObject(const QSize &size)
{
QOpenGLFramebufferObjectFormat format;
format.setAttachment(QOpenGLFramebufferObject::Depth);
m_framebufferObject = new QOpenGLFramebufferObject(size, format);
m_vtkRenderWindow->SetFramebufferObject(m_framebufferObject);
return m_framebufferObject;
}
vtkSmartPointer<vtkInternalOpenGLRenderWindow> m_vtkRenderWindow;
QOpenGLFramebufferObject *m_framebufferObject;
};
vtkStandardNewMacro(vtkInternalOpenGLRenderWindow);
vtkInternalOpenGLRenderWindow::vtkInternalOpenGLRenderWindow() :
QtParentRenderer(0)
{
vtkOpenGLRenderWindow::OpenGLInitContext();
}
void vtkInternalOpenGLRenderWindow::OpenGLInitState()
{
this->MakeCurrent();
vtkOpenGLRenderWindow::OpenGLInitState();
// Before any of the gl* functions in QOpenGLFunctions are called for a
// given OpenGL context, an initialization must be run within that context
initializeOpenGLFunctions();
glFrontFace(GL_CW); // to compensate for the switched Y axis
}
void vtkInternalOpenGLRenderWindow::InternalRender()
{
vtkOpenGLRenderWindow::Render();
}
//
// vtkInternalOpenGLRenderWindow Definitions
//
void vtkInternalOpenGLRenderWindow::Render()
{
this->QtParentRenderer->update();
}
void vtkInternalOpenGLRenderWindow::SetFramebufferObject(QOpenGLFramebufferObject *fbo)
{
// QOpenGLFramebufferObject documentation states that "The color render
// buffer or texture will have the specified internal format, and will
// be bound to the GL_COLOR_ATTACHMENT0 attachment in the framebuffer
// object"
this->BackLeftBuffer = this->FrontLeftBuffer = this->BackBuffer = this->FrontBuffer =
static_cast<unsigned int>(GL_COLOR_ATTACHMENT0);
// Save GL objects by static casting to standard C types. GL* types
// are not allowed in VTK header files.
QSize fboSize = fbo->size();
this->Size[0] = fboSize.width();
this->Size[1] = fboSize.height();
this->NumberOfFrameBuffers = 1;
this->FrameBufferObject = static_cast<unsigned int>(fbo->handle());
this->DepthRenderBufferObject = 0; // static_cast<unsigned int>(depthRenderBufferObject);
this->TextureObjects[0] = static_cast<unsigned int>(fbo->texture());
this->OffScreenRendering = 1;
this->OffScreenUseFrameBuffer = 1;
this->Modified();
}
void QVTKFrameBufferObjectItem::Start()
{
m_win->OpenGLInitState();
}
void QVTKFrameBufferObjectItem::End()
{
}
void QVTKFrameBufferObjectItem::MakeCurrent()
{
this->window()->openglContext()->makeCurrent(this->window());
}
void QVTKFrameBufferObjectItem::IsCurrent(vtkObject*, unsigned long, void*, void* call_data)
{
bool* ptr = reinterpret_cast<bool*>(call_data);
*ptr = this->window()->openglContext();
}
void QVTKFrameBufferObjectItem::IsDirect(vtkObject*, unsigned long, void*, void* call_data)
{
int* ptr = reinterpret_cast<int*>(call_data);
*ptr = QGLFormat::fromSurfaceFormat(this->window()->openglContext()->format()).directRendering();
}
void QVTKFrameBufferObjectItem::SupportsOpenGL(vtkObject*, unsigned long, void*, void* call_data)
{
int* ptr = reinterpret_cast<int*>(call_data);
*ptr = QGLFormat::hasOpenGL();
}
QVTKFrameBufferObjectItem::QVTKFrameBufferObjectItem(QQuickItem *parent) : QQuickFramebufferObject(parent)
{
setAcceptedMouseButtons(Qt::AllButtons);
m_irenAdapter = new QVTKInteractorAdapter(this);
m_win = vtkSmartPointer<vtkInternalOpenGLRenderWindow>::New();
// make a connection between the vtk signals and qt slots so that an initialized and madeCurrent opengl context is given to the vtk
// we probably need only the Start(), MakeCurrent() and End() one, but just to be sure...
mConnect = vtkSmartPointer<vtkEventQtSlotConnect>::New();
mConnect->Connect(m_win, vtkCommand::WindowMakeCurrentEvent, this, SLOT(MakeCurrent()));
mConnect->Connect(m_win, vtkCommand::WindowIsCurrentEvent, this, SLOT(IsCurrent(vtkObject*, unsigned long, void*, void*)));
mConnect->Connect(m_win, vtkCommand::StartEvent, this, SLOT(Start()));
mConnect->Connect(m_win, vtkCommand::EndEvent, this, SLOT(End()));
mConnect->Connect(m_win, vtkCommand::WindowIsDirectEvent, this, SLOT(IsDirect(vtkObject*, unsigned long, void*, void*)));
mConnect->Connect(m_win, vtkCommand::WindowSupportsOpenGLEvent, this, SLOT(SupportsOpenGL(vtkObject*, unsigned long, void*, void*)));
}
QVTKFrameBufferObjectItem::~QVTKFrameBufferObjectItem()
{
mConnect->Disconnect(); // disconnect all slots
if (m_irenAdapter)
delete m_irenAdapter;
}
QQuickFramebufferObject::Renderer *QVTKFrameBufferObjectItem::createRenderer() const
{
return new QVTKFramebufferObjectRenderer(m_win);
}
vtkSmartPointer<vtkInternalOpenGLRenderWindow> QVTKFrameBufferObjectItem::GetRenderWindow() const
{
return m_win;
}
void QVTKFrameBufferObjectItem::init()
{
}
// theoretically not needed now - the Y is being flipped in render and devicePixelRatio will almost always be = 1 on a PC anyway...but lets keep it to be sure
QMouseEvent QVTKFrameBufferObjectItem::openGLToNative(QMouseEvent const& event)
{
QPointF localPos(event.localPos());
localPos.setX(localPos.x() * window()->devicePixelRatio());
localPos.setY(localPos.y() * window()->devicePixelRatio());
QMouseEvent nativeEvent(event.type(), localPos, event.button(), event.buttons(), event.modifiers());
return nativeEvent;
}
void QVTKFrameBufferObjectItem::mouseMoveEvent(QMouseEvent * event)
{
m_win->GetInteractor()->SetSize(this->width(), this->height());
QMouseEvent nativeEvent = openGLToNative(*event);
m_irenAdapter->ProcessEvent(&nativeEvent, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::mousePressEvent(QMouseEvent * event)
{
m_win->GetInteractor()->SetSize(this->width(), this->height());
QMouseEvent nativeEvent = openGLToNative(*event);
m_irenAdapter->ProcessEvent(&nativeEvent, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::mouseReleaseEvent(QMouseEvent * event)
{
m_win->GetInteractor()->SetSize(this->width(), this->height());
QMouseEvent nativeEvent = openGLToNative(*event);
m_irenAdapter->ProcessEvent(&nativeEvent, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::wheelEvent(QWheelEvent *event)
{
m_irenAdapter->ProcessEvent(event, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::keyPressEvent(QKeyEvent* event)
{
m_irenAdapter->ProcessEvent(event, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::keyReleaseEvent(QKeyEvent* event)
{
m_irenAdapter->ProcessEvent(event, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::focusInEvent(QFocusEvent * event)
{
m_irenAdapter->ProcessEvent(event, this->m_win->GetInteractor());
}
void QVTKFrameBufferObjectItem::focusOutEvent(QFocusEvent * event)
{
m_irenAdapter->ProcessEvent(event, this->m_win->GetInteractor());
}
To use it, define an instance of the framebuffer in your qml form and stretch it across the window you want to render into, e.g. like this (assuming you registered the QVTKFrameBufferObjectItem as a QVTKFrameBuffer in qml e.g. like this qmlRegisterType<QVTKFrameBufferObjectItem>("VtkQuick", 1, 0, "QVTKFrameBuffer");):
import VtkQuick 1.0
QVTKFrameBuffer
{
id: renderBuffer
anchors.fill : parent
Component.onCompleted :
{
myCppDisplay.framebuffer = renderBuffer // tell the c++ side of your app that this is the framebuffer into which it should render
}
}
You then use the vtkRenderWindow you get by myCppDisplay.framebuffer.GetRenderWindow() the same way you would use any other vtkRenderWindow if you were rendering into a vtk-managed window, i.e. you can assign vtkRenderer to it, assign actors to that renderer, call theWindow.Render() as you wish and it will all be rendered into the qml component to which you assigned the framebuffer.
Two notes: 1) the vtk and qt use different coordinate system, you need to flip the y-coordinate...I am doing it by assigning a scale transformation to the camera, but there is plenty of other ways to do it:
vtkSmartPointer<vtkTransform> scale = vtkSmartPointer<vtkTransform>::New();
scale->Scale(1, -1, 1);
renderer->GetActiveCamera()->SetUserTransform(scale);
2) things get quite tricky once you start using multiple threads - you have to make sure that you are not trying to render in two different threads, because they would compete for that one QtQuick's rendering thread. This does not mean only not calling renderWindow.Render() in parallel - that is easy to avoid - but you have to realize that that qt thread is used also for rendering the GUI, so you might get into trouble this way (updating GUI while doing VTK rendering).
I tried implementing this example and have successfully compiled and ran it as well but I am facing some challenges with respect to the code.
I am not able to figure out where these lines are defined as they are giving me runtime errors and are not getting recognized as well.
(Defined in qmlVtk.cpp -> SetFramebufferObject() function)
this->NumberOfFrameBuffers = 1;
this->FrameBufferObject = static_cast<unsigned int>(fbo->handle());
this->DepthRenderBufferObject = 0;
this->TextureObjects[0] = static_cast<unsigned int>(fbo->texture());
this->OffScreenRendering = 1;
this->OffScreenUseFrameBuffer = 1;
Also, can anyone guide me as how to integrate QML with OpenGLWidget.

QT container, with specified order and no repetitions

I need somthing similar to QSet, but I need the items to be saved on the order I inserted them
is there such thing?
I am not aware of anything like that out of the box in neither Qt nor STL. Boost has something like that I think but it is not that hard to do this yourself.
You could do a wrapper around QHash like this:
template<typename T>
class MySet : QHash<T, int>
{
public:
using QHash<T, int>::QHash;
QVector<T> values() //this 'hides' the base QHash::values() of QHash
{
QVector<T> vec(count());
for(auto it = cbegin(); it != end(); ++it)
{
vec[it.value()] = it.key();
}
return vec;
}
void insert(const T &value)
{
if(!contains(value))
{
insert(value, m_Data.count());
}
}
};
The usage is quite similar to QSet:
MySet<QString> set;
set.insert("1");
set.insert("2");
set.insert("3");
qDebug() << set.values();
And that prints the values in order. If you need more complete support like iterators also iterating in your desired order you would have to reimplement more functionality but the gist of it would be the same. After all QSet is internally QHash as well. Note that the above does not support removal without modification.
Maybe a QList or a QVector could help.
QList<QString> stringList;
//By the way, Qt provides QStringList as a typedef for QList<QString>
stringList.append("A");
stringList.append("B");
qDebug() << stringList.at(0); //A
qDebug() << stringList.at(1); //B

How to monitor changes to an arbitrary widget?

I am starting a QT5 application with a rather complex design based on Qt Widgets. It runs on Beagleboard with a touchscreen. I will have a rather weird local invention instead of the LCD display. It's a laser painting on acrylic plate. It has no driver yet. To actually update a screen I must create a screenshot of the window as bitmap, turn it to grayscale and feed to a proprietary library, which will handle the laser. It should look cute, when ready. Unfortunately, the laser blinks on update, so I cannot just make screenshots on timer, or it will be jerky like hell.
I need to run a function every time a meaningful update of GUI happens, while preferably ignore things like button being pressed and released. Is there some way to create a hook without subclassing every single Qt Widget I will use? The only way to do this I know is to override paintEvent of everything. I want a simpler solution.
Possible assumptions are: the application will be running under X server with dummy display, will be the only GUI app running. Some updates happen without user input.
The code below does it. It doesn't dig too deeply into the internals of Qt, it merely leverages the fact that backing store devices are usually QImages. It could be modified to accommodate OpenGL-based backing stores as well.
The WidgetMonitor class is used to monitor the widgets for content changes. An entire top-level window is monitored no matter which particular widget is passed to the monitor(QWidget*) method. You only need to call the monitor method for one widget in the window you intend to monitor - any widget will do. The changes are sent out as a QImage of window contents.
The implementation installs itself as an event filter in the target window widget and all of its children, and monitors the repaint events. It attempts to coalesce the repaint notifications by using the zero-length timer. The additions and removals of children are tracked automagically.
When you run the example, it creates two windows: a source window, and a destination window. They may be overlapped so you need to separate them. As you resize the source window, the size of the destination's rendition of it will also change appropriately. Any changes to the source children (time label, button state) propagate automatically to the destination.
In your application, the destination could be an object that takes the QImage contents, converts them to grayscale, resizes appropriately, and passes them to your device.
I do not quite understand how your laser device works if it can't gracefully handle updates. I presume that it is a raster-scanning laser that runs continuously in a loop that looks roughly like this:
while (1) {
for (line = 0; line < nLines; ++line) {
drawLine();
}
}
You need to modify this loop so that it works as follows:
newImage = true;
QImage localImage;
while (1) {
if (newImage) localImage = newImage;
for (line = 0; line < localImage.height(); ++line) {
drawLine(line, localImage);
}
}
You'd be flipping the newImage flag from the notification slot connected to the WidgetMonitor. You may well find out that leveraging QImage, and Qt's functionality in general, in your device driver code, will make it much easier to develop. Qt provides portable timers, threads, collections, etc. I presume that your "driver" is completely userspace, and communicates via a serial port or ethernet to the micro controller that actually controls the laser device.
If you will be writing a kernel driver for the laser device, then the interface would be probably very similar, except that you end up writing the image bitmap to an open device handle.
// https://github.com/KubaO/stackoverflown/tree/master/questions/surface-20737882
#include <QtWidgets>
#include <array>
const char kFiltered[] = "WidgetMonitor_filtered";
class WidgetMonitor : public QObject {
Q_OBJECT
QVector<QPointer<QWidget>> m_awake;
QBasicTimer m_timer;
int m_counter = 0;
void queue(QWidget *window) {
Q_ASSERT(window && window->isWindow());
if (!m_awake.contains(window)) m_awake << window;
if (!m_timer.isActive()) m_timer.start(0, this);
}
void filter(QObject *obj) {
if (obj->isWidgetType() && !obj->property(kFiltered).toBool()) {
obj->installEventFilter(this);
obj->setProperty(kFiltered, true);
}
}
void unfilter(QObject *obj) {
if (obj->isWidgetType() && obj->property(kFiltered).toBool()) {
obj->removeEventFilter(this);
obj->setProperty(kFiltered, false);
}
}
bool eventFilter(QObject *obj, QEvent *ev) override {
switch (ev->type()) {
case QEvent::Paint: {
if (!obj->isWidgetType()) break;
if (auto *window = static_cast<QWidget *>(obj)->window()) queue(window);
break;
}
case QEvent::ChildAdded: {
auto *cev = static_cast<QChildEvent *>(ev);
if (auto *child = qobject_cast<QWidget *>(cev->child())) monitor(child);
break;
}
default:
break;
}
return false;
}
void timerEvent(QTimerEvent *ev) override {
if (ev->timerId() != m_timer.timerId()) return;
qDebug() << "painting: " << m_counter++ << m_awake;
for (auto w : m_awake)
if (auto *img = dynamic_cast<QImage *>(w->backingStore()->paintDevice()))
emit newContents(*img, w);
m_awake.clear();
m_timer.stop();
}
public:
explicit WidgetMonitor(QObject *parent = nullptr) : QObject{parent} {}
explicit WidgetMonitor(QWidget *w, QObject *parent = nullptr) : QObject{parent} {
monitor(w);
}
Q_SLOT void monitor(QWidget *w) {
w = w->window();
if (!w) return;
filter(w);
for (auto *obj : w->findChildren<QWidget *>()) filter(obj);
queue(w);
}
Q_SLOT void unMonitor(QWidget *w) {
w = w->window();
if (!w) return;
unfilter(w);
for (auto *obj : w->findChildren<QWidget *>()) unfilter(obj);
m_awake.removeAll(w);
}
Q_SIGNAL void newContents(const QImage &, QWidget *w);
};
class TestWidget : public QWidget {
QVBoxLayout m_layout{this};
QLabel m_time;
QBasicTimer m_timer;
void timerEvent(QTimerEvent *ev) override {
if (ev->timerId() != m_timer.timerId()) return;
m_time.setText(QTime::currentTime().toString());
}
public:
explicit TestWidget(QWidget *parent = nullptr) : QWidget{parent} {
m_layout.addWidget(&m_time);
m_layout.addWidget(new QLabel{"Static Label"});
m_layout.addWidget(new QPushButton{"A Button"});
m_timer.start(1000, this);
}
};
int main(int argc, char **argv) {
QApplication app{argc, argv};
TestWidget src;
QLabel dst;
dst.setFrameShape(QFrame::Box);
for (auto *w : std::array<QWidget *, 2>{&dst, &src}) {
w->show();
w->raise();
}
QMetaObject::invokeMethod(&dst, [&] { dst.move(src.frameGeometry().topRight()); },
Qt::QueuedConnection);
WidgetMonitor mon(&src);
src.setWindowTitle("Source");
dst.setWindowTitle("Destination");
QObject::connect(&mon, &WidgetMonitor::newContents, [&](const QImage &img) {
dst.resize(img.size());
dst.setPixmap(QPixmap::fromImage(img));
});
return app.exec();
}
#include "main.moc"

Qt QAudioOutput StateChnged always on ActiveState (Qt 5.1 Windows MinGW)

I have a derived class from QObject that has a bunch of concatenated WAVE files in a QByteArray as a member variable.
I want to play specific files in that array (provided that I have the offset of it) through a QAudioOuput.
Here is the code of the PlaySound Method:
void DRMUtils::PlaySound(int offset){
mAudioFormat = new QAudioFormat(GetFormat(offset));
mAudioOut = new QAudioOutput(*mAudioFormat);
mBufferedAudio = new QBuffer();
mBufferedAudio->setData(GetSound(offset));
mBufferedAudio->open(QIODevice::ReadOnly);
connect(mAudioOut, SIGNAL(stateChanged(QAudio::State)), this, SLOT(handleAudioStateChanged(QAudio::State)));
mAudioOut->start(mBufferedAudio);
}
I get the file format from the 44 first bytes (WAVE standard) with GetFormat(offset) and I get the data in QByteArray format with GetSound(offset)
Everything seems to work fine (I can hear the sample play), but the state of QAudioFormat never changes from "ActiveState"
Here is my slot code:
void DRMUtils::handleAudioStateChanged(QAudio::State newState)
{
qDebug() << "State: " << newState;
switch (newState) {
case QAudio::IdleState:
// Finished playing
mAudioOut->stop();
mBufferedAudio->close();
delete mAudioOut;
delete mBufferedAudio;
delete mAudioFormat;
qDebug() << "DELETED!";
break;
case QAudio::StoppedState:
// Stopped for other
qDebug() << "STOPPED!";
if (mAudioOut->error() != QAudio::NoError) {
// Error handling
qDebug() << "STOPPED ERROR!";
}
break;
default:
// ... other cases as appropriate
qDebug() << "DEFAULT!";
break;
}
}
My Debug output is always:
State: ActiveState
DEFAULT!
Do I have to "cap" the mBufferedAudio in some way that QAudioOutput "knows" when the sample is completed?
Also, I anybody can tell me when why when I initialize mAudioOut like this (declaring the parent):
mAudioOut = new QAudioOutput(*mAudioFormat, this);
instead of (not declaring the parent):
mAudioOut = new QAudioOutput(*mAudioFormat);
I don't get any output to the speakers (with parent declared).
Thanks
May be this gets deleted? So mAudioOut is deleted too when the parent is specified. It would also explain why you don't receive state changes (if the object gets deleted, the slot cannot be called anymore).

Resources