OpenNI + OpenCV + Qt - qt

I'm trying to make an app using Kinect (OpenNI), processing the image (OpenCV) with a GUI.
I tested de OpenNI+OpenCV and OpenCV+Qt
Normally when we use OpenCV+Qt we can make a QWidget to show the content of the camera (VideoCapture) .. Capture a frame and update this querying for new frames to device.
With OpenNI and OpenCV i see examples using a for cycle to pull data from Kinect Sensors (image, depth) , but i don't know how to make this pulling routing mora straightforward. I mean, similar to the OpenCV frame querying.
The idea is embed in a QWidget the images captured from Kinect. The QWidget will have (for now) 2 buttons "Start Kinect" and "Quit" ..and below the Painting section to show the data captured.
Any thoughs?

You can try the QTimer class to query the kinect at fixed time intervals. In my application I use the code below.
void UpperBodyGestures::refreshUsingTimer()
{
QTimer *timer = new QTimer(this);
connect(timer, SIGNAL(timeout()), this, SLOT(MainEventFunction()));
timer->start(30);
}
void UpperBodyGestures::on_pushButton_Kinect_clicked()
{
InitKinect();
ui.pushButton_Kinect->setEnabled(false);
}
// modify the main function to call refreshUsingTimer function
UpperBodyGestures w;
w.show();
w.refreshUsingTimer();
return a.exec();
Then to query the frame you can use the label widget. I'm posting an example code below:
// Query the depth data from Openni
const XnDepthPixel* pDepth = depthMD.Data();
// Convert it to opencv for manipulation etc
cv::Mat DepthBuf(480,640,CV_16UC1,(unsigned char*)g_Depth);
// Normalize Depth image to 0-255 range (cant remember max range number so assuming it as 10k)
DepthBuf = DepthBuf / 10000 *255;
DepthBuf.convertTo(DepthBuf,CV_8UC1);
// Convert opencv image to a Qimage object
QImage qimage((const unsigned char*)DepthBuf.data, DepthBuf.size().width, DepthBuf.size().height, DepthBuf.step, QImage::Format_RGB888);
// Display the Qimage in the defined mylabel object
ui.myLabel->setPixmap(pixmap.fromImage(qimage,0).scaled(QSize(300,300), Qt::KeepAspectRatio, Qt::FastTransformation));

Related

QT QVideoFrame memcpy zero-copy alternative

I get raw video data from the V4L2 driver using VIDIOC_DQBUF and I want to render this frames in qt using QVideoFrame as described here: https://blog.katastros.com/a?ID=9f708708-c5b3-4cb3-bbce-400cc8b8000c
This code works well but has huge performance issues.
Here is the problematik code part when doing this:
QVideoFrame f(size, QSize(width, height), width, QVideoFrame::Format_YUV420P);
if (f.map(QAbstractVideoBuffer::WriteOnly)) {
memcpy(f.bits(), data, size);
f.setStartTime(0);
f.unmap();
emit newFrameAvailable(f);
}
The memcpy operation for my 4K video reduces the framerate from 35fps to 5fps on my arm based embedded system.
This constructor is supposed to constructs a video frame from a buffer with the given pixel format and size in pixels. However I cannot find any example of this:
QVideoFrame::QVideoFrame(QAbstractVideoBuffer *buffer, const QSize &size, QVideoFrame::PixelFormat format)
I just need to pass valid buffer to QVideoFrame. I don't need to map or unmap the QVideoFrame. Like this:
unsigned char * pBuffer = get_pointer_to_a_frame();
QVideoFrame frame((QAbstractVideoBuffer *) pBuffer, QSize(width, height), QVideoFrame::Format_YUV420P);
frame.setStartTime(0);
emit newFrameAvailable(frame);
Any zero-copy QVideoFrame usage will wellcome.

Using QGLFramebufferObject and shaders without QGLWidget

I want to create an OGL data processor using QGLFunctions shaders and framebuffers. I don't need any widgets. But to create valid Shader and framebuffer instances, I need a valid QGLContext with support for the appropriate glExtensions.
With zero context, of course, nothing works. With context of zero QPaintDevice too. With Qpixmap as device it creates a valid context, but it lacks glExtensions for Shader and framebuffer.
#include <QGLFramebufferObject>
#include <QGLShaderProgram>
#include <QtOpenGL/QGLFunctions>
// ...
void GLProcessor::init()
{
auto format = QGLFormat::defaultFormat();
if (!context()){
m_context = new QGLContext(format, new QPixmap(1, 1));
bool ok = m_context->create();
qDebug() << "CREATING CONTEXT "<< ok;
Q_ASSERT(context()->isValid());
}
context()->makeCurrent();
initializeGLFunctions(context());
m_binFBO = new QGLFramebufferObject(lowsize ,lowsize ,QGLFramebufferObject::NoAttachment, GL_TEXTURE_2D, GL_RED);
m_outFBO = new QGLFramebufferObject(lowsize ,1 ,QGLFramebufferObject::NoAttachment, GL_TEXTURE_2D, GL_RED);
setupShaders();
// ...
}
There is an option, of course, to do as always is to get the context from the QGLWidget and hide it. But somehow inelegant. PS CUDA, OpenCL, AMP and so on I don't need. For my tasks need OpenGL.
How do I use shaders and framebuffers in qt4 without creating a QGLWidget?

what is the best way to show tile map and some other object in graphicsview?

recently i start to learn Qt and now i'm working on GCS project that it must have a map with some tiled imges and and some graphics item like Plan,the path and also on over off all some gauge.
so we have 3 kind of item:
Tiled map in the background so that its change by scrolling .
in the middle there is a picture of airplane that move by gps changes and also its way .
on the all on off these items there 3 or 4 gauge like speed meter, horizontal gauge and altimeter gauge there are must be solid in somewhere of graphicsview and not change when scrolling down/up or left right
The question is what is the best way to implement this ?
here is first look of my project:
in first look gauge are not over map but i want to be ! i want to have bigger map screen with gauges include it !
And here is map updater code :
void mainMap::update()
{
m_scene->clear();
QString TilePathTemp;
QImage *imageTemp = new QImage();
int X_Start=visibleRect().topLeft().x()/256;
int X_Num=qCeil((float)visibleRect().bottomRight().x()/256.0f-(float)visibleRect().topLeft().x()/256.0f);
int Y_Start=visibleRect().topLeft().y()/256;
int Y_Num=qCeil((float)visibleRect().bottomRight().y()/256.0f-(float)visibleRect().topLeft().y()/256.0f);
LastCenterPoint->setX(visibleRect().center().x());
LastCenterPoint->setY(visibleRect().center().y());
X_Start=(X_Start-X_MAP_MARGIN)>0?(X_Start-X_MAP_MARGIN):0;
Y_Start=(Y_Start-Y_MAP_MARGIN)>0?(Y_Start-Y_MAP_MARGIN):0;
X_Num+=X_MAP_MARGIN;
Y_Num+=Y_MAP_MARGIN;
qDebug()<<"XS:"<<X_Start<<" Num:"<<X_Num;
qDebug()<<"YS:"<<Y_Start<<" Num:"<<Y_Num;
for(int x=X_Start;x<=X_Start+X_Num;x++){
for(int y=Y_Start;y<=Y_Start+Y_Num;y++){
if(Setting->value("MapType",gis::Hybrid).toInt()==gis::Hybrid) TilePathTemp=Setting->value("MapPath","/Users/M410/Documents/Map").toString()+"/Hybrid/gh_"+QString::number(x)+"_"+QString::number(y)+"_"+QString::number(ZoomLevel)+".jpeg" ;
else if(Setting->value("MapType",gis::Sattelite).toInt()==gis::Sattelite) TilePathTemp=Setting->value("MapPath","/Users/M410/Documents/Map").toString()+"/Sattelite/gs_"+QString::number(x)+"_"+QString::number(y)+"_"+QString::number(ZoomLevel)+".jpeg" ;
else if(Setting->value("MapType",gis::Street).toInt()==gis::Street) TilePathTemp=Setting->value("MapPath","/Users/M410/Documents/Map").toString()+"/Street/gm_"+QString::number(x)+"_"+QString::number(y)+"_"+QString::number(ZoomLevel)+".jpeg" ;
QFileInfo check_file(TilePathTemp);
// check if file exists and if yes: Is it really a file and no directory?
if (check_file.exists() && check_file.isFile()) {
// qDebug()<<"Exist!";
imageTemp->load(TilePathTemp);
QPixmap srcImage = QPixmap::fromImage(*imageTemp);
//QPixmap srcImage("qrc:/Map/File1.jpeg");
QGraphicsPixmapItem* item = new QGraphicsPixmapItem(srcImage);
item->setPos(QPointF(x*256, y*256));
m_scene->addItem(item);
// centerOn( width() / 2.0f , height() / 2.0f );
} else {
qDebug()<<"NOT Exist!";
}
}
}
Really, you should consider using QML. The advantage of using QML instead of QGraphicsView is you can iterate a lot faster than if you were working directly in C++. The primary downside is generally increased memory usage and incompatibility with QWidgets.
So if you need unique graphics, and very little "standard widget" stuff, you should use QML first and then QGraphicsView ONLY IF requirements dictate it.
Specific to your project though, Qt has a Map type which could be useful: https://doc.qt.io/qt-5/qml-qtlocation-map.html

Optimizing QPainter drawing & Converting QVideoFrame straight to QPixMap

Some background info about my issue. My goal is to optimize drawing of images coming from webcam, the images come as QVideoFrame and are currently loaded in to QImage and drawn from there. This solution works fine, but drawing QImage is very slow on X11. Drawing one image takes about 20ms which doesn't sound like much but when you do this for every frame this cut's the framerate of the camerafeed to half.
I did some research and testing, drawing QPixMaps in X11 can be done about 10 times faster than drawing QImages.
This is how the drawing process is done currently
if(mVFcurrentFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage image(mVFcurrentFrame.bits(), mVFcurrentFrame.width(), mVFcurrentFrame.height(), mVFcurrentFrame.bytesPerLine(), imageFormat);
painter->drawImage(0,0,image); //Takes about 20ms
mVFcurrentFrame.unmap();
}
What i have tried so far:
Converting the QImage to QPixMap, this works but the conversion is as slow as painting the Qimage
Loading the QVideoFrame straight to QPixMap with QPixMap::loadFromData(), can't make it work.
So my question is, can i convert QVideoFrame straight to QPixMap and draw it instead of using QImage and how would you do the QVideoFrame to QPixmap conversion without using QImage in between?
I have tried using QPixMap::loadFromData() method to load the video frame but so far i have been unable to make it work.
If this isn't possible could i thread the QImage to QPixMap conversion or optimize the drawing in some other way?
This is my problem too.
camera frames are shown very slowly in QLabel.
my code is here:
QCamera *camera = new QCamera(this);
camera->setCaptureMode(QCamera::CaptureViewfinder);
QVideoProbe *videoProbe = new QVideoProbe(this);
bool ret = videoProbe->setSource(camera);
if (ret) {
connect(videoProbe, SIGNAL(videoFrameProbed(const QVideoFrame &)),
this, SLOT(present(const QVideoFrame &)));
}
camera->start();
...
...
bool MainWindow::present(const QVideoFrame &frame)
{
QVideoFrame cloneFrame(frame);
if(cloneFrame.map(QAbstractVideoBuffer::ReadOnly))
{
QImage img(
cloneFrame.size(), QImage::Format_ARGB32);
qt_convert_NV21_to_ARGB32(cloneFrame.bits(),
(quint32 *)img.bits(),
cloneFrame.width(),
cloneFrame.height());
label->setPixmap(QPixmap::fromImage(img));
cloneFrame.unmap();
}
return true;
}

Cannot see IplImage in QGraphicsView

I'm trying to display a 3D scene(OpenGL-OpenCV) in a QGraphicsView object in QT. The scene has 5 planes: top, bottom, right, left and front. I'm taking images from my webcam and mapping them to the front plane. I have successfully displayed 4 of 5 planes, the front plane is missing.
I followed this tutorial to load the OpenGL scene: http://doc.trolltech.com/qq/qq26-openglcanvas.html
However, I don't know how to treat the IplImage to be displayed in the QT Object. Do you guys have any suggestion?
This is something that I salvaged out of a blog posting,
This will provide you with a QImage that you can display using Qt.
you should tailor it to fit your needs.
QImage img;
constructor()
{
// setup capture device
CvCapture *cvCapture = cvCreateCapture(0);
}
getQImageFromIplImage()
{
// this frame gets a frame from capture device
IplImage *frame = new IplImage();
frame = cvQueryFrame(cvCapture);
// create an IplImage with 8bit color depth
IplImage *iplImg = cvCreateImage(cvSize(frame->width, frame->height),IPL_DEPTH_8U, 3);
// copy image captured from capture device to new image, converting pixel data from OpenCV's default BGR format to Qt's RGB format
cvCvtColor(frame, iplImg, CV_BGR2RGB);
// create a this newly converted RGB pixel data with a QImage
qImg = QImage((uchar *)iplImg->imageData, iplImg->width, iplImg->height, QImage::Format_RGB888);
}
for full code, check out:
http://www.morethantechnical.com/2009/03/05/qt-opencv-combined-for-face-detecting-qwidgets/

Resources