QPixmap.loadFromData() does not load image from QByteArray - qt

I'm creating a socket-based program to send a screenshot from one user to another user. I need to convert a screenshot to a byte array before sending. After I convert my screenshot to a QByteArray I insert 4 bytes to the beginning of the array to mark that it is a picture (it is the number 20 to tell me it is a picture and not text or something else).
After I send the byte array via a socket to other user, when it is received I read the first 4 bytes to know what it is. Since it was a picture I then convert it from a QByteArray to QPixmap to show it on a label. I use secondPixmap.loadFromData(byteArray,"JPEG") to load it but it not load any picture.
This is a sample of my code:
void MainWindow::shootScreen()
{
originalPixmap = QPixmap(); // clear image for low memory situations
// on embedded devices.
originalPixmap = QGuiApplication::primaryScreen()->grabWindow(0);
scaledPixmap = originalPixmap.scaled(500, 500);
QByteArray bArray;
QBuffer buffer(&bArray);
buffer.open(QIODevice::WriteOnly);
originalPixmap.save(&buffer,"JPEG",5);
qDebug() << bArray.size() << "diz0";
byteArray= QByteArray();
QDataStream ds(&byteArray,QIODevice::ReadWrite);
int32_t c = 20;
ds << c;
ds<<bArray;
}
void MainWindow::updateScreenshotLabel()
{
this->ui->label->setPixmap(secondPixmap.scaled(this->ui->label->size(), Qt::KeepAspectRatio, Qt::SmoothTransformation));
}
void MainWindow::on_pushButton_clicked()
{
shootScreen();
}
void MainWindow::on_pushButton_2_clicked()
{
secondPixmap = QPixmap();
QDataStream ds(&byteArray,QIODevice::ReadOnly);
qint32 code;
ds>>code;
secondPixmap.loadFromData(byteArray,"JPEG");
updateScreenshotLabel();
}

Your MainWindow::on_pushButton_2_clicked implementation looks odd. You have...
QDataStream ds(&byteArray,QIODevice::ReadOnly);
which creates a read-only QDataStream that will read it's input data from byteArray. But later you have...
secondPixmap.loadFromData(byteArray,"JPEG");
which attempts to read the QPixmap directly from the same QByteArray -- bypassing the QDataStream completely.
You can also make use of the QPixmap static members that read from/write to a QDataStream. So I think you're looking for something like...
QDataStream ds(&byteArray,QIODevice::ReadOnly);
qint32 code;
ds >> code;
if (code == 20)
ds >> secondPixmap;
And likewise for your MainWindow::shootScreen implementation. You could reduce your code a fair bit by making use of QDataStream & operator<<(QDataStream &stream, const QPixmap &pixmap).

Related

How can i play music using QByteArray data?

I have a QByteArray which created like this:
QByteArray data;
QFile file("/path/to/music.mp3");
if (file.open(QIODevice::ReadOnly))
{
data = file.readAll();
}
And i get it somewhere else, how could i play it using QMediaPlayer without save it to file ?
If you have .mp3 file directly, you can call it by directly setting the URL to QMediaPlayer.
You can find below example in documentation.
QMediaPlayer* player = new QMediaPlayer;
connect(player, SIGNAL(positionChanged(qint64)), this, SLOT(positionChanged(qint64)));
player->setMedia(QUrl::fromLocalFile("/path/to/music.mp3"));
player->setVolume(50);
player->play();
https://doc.qt.io/qt-5/qmediaplayer.html#setMedia
If for obvious reasons, you have to go with QByteArray, May be you can try as said below (Not tried and tested):
//BYTE ARRAY
QByteArray data;
if (file.open(QIODevice::ReadOnly))
{
data = file.readAll();
}
//CREATE A BUFFER OBJECT WITH BYTE ARRAY
QBuffer buffer(&data);
buffer.open(QIODevice::ReadOnly);
//CREATE MEDIA PLAYER OBJECT
QMediaPlayer* player = new QMediaPlayer;
//SET MEDIA CONTENT AND BUFFER.
player->setMedia(QUrl::fromLocalFile("/path/to/music.mp3"),&buffer);
player->play();

QByteArray to int always give 0

I try to send data throw qtcpsocket and use QByteArray. I send previosly size of data and try convert to int like this QByteArray to Int conversion.
But always get 0. Code convertion example:
QString ss = "bca";
int aaa;
QByteArray b;
QDataStream stream(&b, QIODevice::ReadWrite);
stream << ss.toUtf8().size();
stream >> aaa;
In this example aaa is always 0, but ss.toUtf8().size() isnt. What im doing wrong?
QString ss = "bca";
int aaa;
QByteArray b;
QDataStream stream(&b, QIODevice::ReadWrite);
stream << ss.toUtf8().size();
stream.device().seek(0); // add this code
stream >> aaa;
The inner pointer of QByteArray is at the end, so nothing can be read.

QFile write a WAV header writes only 4 byte data

I'm writing a WAV recorder, using QFile as backbone. However, when I fill my Wav struct, and try to write it to my QFile it writes only "RIFF", I viewed it with unix's od -cb 1.wav. Here is the samle code:
wavwriter.cpp
Wav::Wav(const char *fname, QFile* parent)
: QFile(fname, parent),
m_fname(fname)
{
setFileName(fname);
bool res = this->open(QIODevice::ReadWrite);
if (res) {
std::cout << "File opened for RW\n";
}
}
Wav::~Wav()
{
}
void Wav::writeHeader(const WavHdr* hdr)
{
write((char*)hdr);
flush();
}
void Wav::appendData(const QByteArray &data)
{
m_data.append(data);
}
QByteArray Wav::getWavData()
{
return m_data;
}
And the usage is as follows:
WavHdr hdr;
hdr.bits_per_sample = 8;
hdr.riff[0] = 'R';
hdr.riff[1] = 'I';
hdr.riff[2] = 'F';
hdr.riff[3] = 'F';
hdr.sample_rate = 8;
hdr.fmt[0] = 'f';
hdr.fmt[1] = 'm';
hdr.fmt[2] = 't';
m_wavs[i]->writeHeader(&hdr);
The WavHdr has the following setup:
struct WavHdr
{
char riff[4];
qint32 file_size;
char wave[4];
char fmt[4];
char len[3];
qint16 type;
quint16 format;
qint32 sample_rate;
qint32 sr_bs_channs;
quint8 bits_per_sample;
char data[4];
qint32 fsize;
};
You can't dump WavHdr to disk directly.
The way you use the write method only makes sense for zero-terminated strings. It will stop writing at the first zero-valued byte. A WavHdr is not a null-terminated string.
You cannot assume that the struct has any particular representation in memory. The compiler is free to arrange that structure the way it sees fit. Not only it can pad and align the members arbitrarily, it can also rearrange them. So that's a non-portable anti-pattern: it may happen to work on some compilers, on others it will be thoroughly broken.
Your WavHdr is wrong.
See here for reference. I've included a correct header structure below.
You probably wish to use a QSaveFile.
When saving files, you usually intend the file writing to be atomic: either it succeeds and you get a complete, valid WAV file, or it fails and nothing changes on disk (e.g. an existing file is not overwritten and corrupted). That's what QSaveFile is designed for.
You probably want your wave class to use an I/O device, but not be one.
I/O can be done with just an instance of a QIODevice*: you'll then be able to easily write the data to in-memory buffers, files, network sockets, etc. The user of your class should be free to choose what particular device to use.
Instead, use QDataStream to write the header in a portable way:
struct WavHdr
{
constexpr static quint32 k_riff_id = 0x46464952;
constexpr static quint32 k_wave_format = 0x45564157;
constexpr static quint32 k_fmt_id = 0x20746d66;
constexpr static quint32 k_data_id = 0x61746164;
// RIFF
quint32 chunk_id = k_riff_id;
quint32 chunk_size;
quint32 chunk_format = k_wave_format;
// fmt
quint32 fmt_id = k_fmt_id;
quint32 fmt_size;
quint16 audio_format;
quint16 num_channels;
quint32 sample_rate;
quint32 byte_rate;
quint16 block_align;
quint16 bits_per_sample;
// data
quint32 data_id = k_data_id;
quint32 data_size;
};
bool write(QIODevice * dev, const WavHdr & h) {
QDataStream s{dev};
s.setByteOrder(QDataStream::LittleEndian); // for RIFF
s << h.chunk_id << h.chunk_size
<< h.chunk_format;
s << h.fmt_id << h.fmt_size
<< h.audio_format
<< h.num_channels
<< h.sample_rate
<< h.byte_rate
<< h.block_align
<< h.bits_per_sample;
s << h.data_id << h.data_size;
return s.status() == QDataStream::Ok;
}

How to save a frame using QMediaPlayer?

I want to save an image of a frame from a QMediaPlayer. After reading the documentation, I understood that I should use QVideoProbe. I am using the following code :
QMediaPlayer *player = new QMediaPlayer();
QVideoProbe *probe = new QVideoProbe;
connect(probe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(processFrame(QVideoFrame)));
qDebug()<<probe->setSource(player); // Returns true, hopefully.
player->setVideoOutput(myVideoSurface);
player->setMedia(QUrl::fromLocalFile("observation.mp4"));
player->play(); // Start receving frames as they get presented to myVideoSurface
But unfortunately, probe->setSource(player) always returns false for me, and thus my slot processFrame is not triggered.
What am I doing wrong ? Does anybody have a working example of QVideoProbe ?
You're not doing anything wrong. As #DYangu pointed out, your media object instance does not support monitoring video. I had the same problem (and same for QAudioProbe but it doesn't interest us here). I found a solution by looking at this answer and this one.
The main idea is to subclass QAbstractVideoSurface. Once you've done that, it will call the method QAbstractVideoSurface::present(const QVideoFrame & frame) of your implementation of QAbstractVideoSurface and you will be able to process the frames of your video.
As it is said here, usually you will just need to reimplement two methods :
supportedPixelFormats so that the producer can select an appropriate format for the QVideoFrame
present which allows to display the frame
But at the time, I searched in the Qt source code and happily found this piece of code which helped me to do a full implementation. So, here is the full code for using a "video frame grabber".
VideoFrameGrabber.cpp :
#include "VideoFrameGrabber.h"
#include <QtWidgets>
#include <qabstractvideosurface.h>
#include <qvideosurfaceformat.h>
VideoFrameGrabber::VideoFrameGrabber(QWidget *widget, QObject *parent)
: QAbstractVideoSurface(parent)
, widget(widget)
, imageFormat(QImage::Format_Invalid)
{
}
QList<QVideoFrame::PixelFormat> VideoFrameGrabber::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
Q_UNUSED(handleType);
return QList<QVideoFrame::PixelFormat>()
<< QVideoFrame::Format_ARGB32
<< QVideoFrame::Format_ARGB32_Premultiplied
<< QVideoFrame::Format_RGB32
<< QVideoFrame::Format_RGB24
<< QVideoFrame::Format_RGB565
<< QVideoFrame::Format_RGB555
<< QVideoFrame::Format_ARGB8565_Premultiplied
<< QVideoFrame::Format_BGRA32
<< QVideoFrame::Format_BGRA32_Premultiplied
<< QVideoFrame::Format_BGR32
<< QVideoFrame::Format_BGR24
<< QVideoFrame::Format_BGR565
<< QVideoFrame::Format_BGR555
<< QVideoFrame::Format_BGRA5658_Premultiplied
<< QVideoFrame::Format_AYUV444
<< QVideoFrame::Format_AYUV444_Premultiplied
<< QVideoFrame::Format_YUV444
<< QVideoFrame::Format_YUV420P
<< QVideoFrame::Format_YV12
<< QVideoFrame::Format_UYVY
<< QVideoFrame::Format_YUYV
<< QVideoFrame::Format_NV12
<< QVideoFrame::Format_NV21
<< QVideoFrame::Format_IMC1
<< QVideoFrame::Format_IMC2
<< QVideoFrame::Format_IMC3
<< QVideoFrame::Format_IMC4
<< QVideoFrame::Format_Y8
<< QVideoFrame::Format_Y16
<< QVideoFrame::Format_Jpeg
<< QVideoFrame::Format_CameraRaw
<< QVideoFrame::Format_AdobeDng;
}
bool VideoFrameGrabber::isFormatSupported(const QVideoSurfaceFormat &format) const
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();
return imageFormat != QImage::Format_Invalid
&& !size.isEmpty()
&& format.handleType() == QAbstractVideoBuffer::NoHandle;
}
bool VideoFrameGrabber::start(const QVideoSurfaceFormat &format)
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();
if (imageFormat != QImage::Format_Invalid && !size.isEmpty()) {
this->imageFormat = imageFormat;
imageSize = size;
sourceRect = format.viewport();
QAbstractVideoSurface::start(format);
widget->updateGeometry();
updateVideoRect();
return true;
} else {
return false;
}
}
void VideoFrameGrabber::stop()
{
currentFrame = QVideoFrame();
targetRect = QRect();
QAbstractVideoSurface::stop();
widget->update();
}
bool VideoFrameGrabber::present(const QVideoFrame &frame)
{
if (frame.isValid())
{
QVideoFrame cloneFrame(frame);
cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
const QImage image(cloneFrame.bits(),
cloneFrame.width(),
cloneFrame.height(),
QVideoFrame::imageFormatFromPixelFormat(cloneFrame .pixelFormat()));
emit frameAvailable(image); // this is very important
cloneFrame.unmap();
}
if (surfaceFormat().pixelFormat() != frame.pixelFormat()
|| surfaceFormat().frameSize() != frame.size()) {
setError(IncorrectFormatError);
stop();
return false;
} else {
currentFrame = frame;
widget->repaint(targetRect);
return true;
}
}
void VideoFrameGrabber::updateVideoRect()
{
QSize size = surfaceFormat().sizeHint();
size.scale(widget->size().boundedTo(size), Qt::KeepAspectRatio);
targetRect = QRect(QPoint(0, 0), size);
targetRect.moveCenter(widget->rect().center());
}
void VideoFrameGrabber::paint(QPainter *painter)
{
if (currentFrame.map(QAbstractVideoBuffer::ReadOnly)) {
const QTransform oldTransform = painter->transform();
if (surfaceFormat().scanLineDirection() == QVideoSurfaceFormat::BottomToTop) {
painter->scale(1, -1);
painter->translate(0, -widget->height());
}
QImage image(
currentFrame.bits(),
currentFrame.width(),
currentFrame.height(),
currentFrame.bytesPerLine(),
imageFormat);
painter->drawImage(targetRect, image, sourceRect);
painter->setTransform(oldTransform);
currentFrame.unmap();
}
}
VideoFrameGrabber.h
#ifndef VIDEOFRAMEGRABBER_H
#define VIDEOFRAMEGRABBER_H
#include <QtWidgets>
class VideoFrameGrabber : public QAbstractVideoSurface
{
Q_OBJECT
public:
VideoFrameGrabber(QWidget *widget, QObject *parent = 0);
QList<QVideoFrame::PixelFormat> supportedPixelFormats(
QAbstractVideoBuffer::HandleType handleType = QAbstractVideoBuffer::NoHandle) const;
bool isFormatSupported(const QVideoSurfaceFormat &format) const;
bool start(const QVideoSurfaceFormat &format);
void stop();
bool present(const QVideoFrame &frame);
QRect videoRect() const { return targetRect; }
void updateVideoRect();
void paint(QPainter *painter);
private:
QWidget *widget;
QImage::Format imageFormat;
QRect targetRect;
QSize imageSize;
QRect sourceRect;
QVideoFrame currentFrame;
signals:
void frameAvailable(QImage frame);
};
#endif //VIDEOFRAMEGRABBER_H
Note : in the .h, you will see I added a signal taking an image as a parameter. This will allow you to process your frame anywhere in your code. At the time, this signal took a QImage as a parameter, but you can of course take a QVideoFrame if you want to.
Now, we are ready to use this video frame grabber:
QMediaPlayer* player = new QMediaPlayer(this);
// no more QVideoProbe
VideoFrameGrabber* grabber = new VideoFrameGrabber(this);
player->setVideoOutput(grabber);
connect(grabber, SIGNAL(frameAvailable(QImage)), this, SLOT(processFrame(QImage)));
Now you just have to declare a slot named processFrame(QImage image) and you will receive a QImage each time you will enter the method present of your VideoFrameGrabber.
I hope that this will help you!
After Qt QVideoProbe documentation:
bool QVideoProbe::setSource(QMediaObject *mediaObject)
Starts monitoring the given mediaObject.
If there is no media object associated with mediaObject, or if it is
zero, this probe will be deactivated and this function wil return
true.
If the media object instance does not support monitoring video, this
function will return false.
Any previously monitored objects will no longer be monitored. Passing
in the same object will be ignored, but monitoring will continue.
So it seems your "media object instance does not support monitoring video"
TL;DR: https://gist.github.com/JC3/a7bab65acbd7659d1e57103d2b0021ba (only file)
I had a similar issue (5.15.2; although in my case I was on Windows, was definitely using the DirectShow back-end, the probe attachment was returning true, the sample grabber was in the graph, but the callback wasn't firing).
I never figured it out but needed to get something working so I kludged one out of a QAbstractVideoSurface, and it's been working well so far. It's a bit simpler than some of the other implementations in this post, and it's all in one file.
Note that Qt 5.15 or higher is required if you intend to both process frames and play them back with this, since the multi-surface QMediaPlayer::setVideoOutput wasn't added until 5.15. If all you want to do is process video you can still use the code below as a template for pre-5.15, just gut the formatSource_ parts.
Code:
VideoProbeSurface.h (the only file; link is to Gist)
#ifndef VIDEOPROBESURFACE_H
#define VIDEOPROBESURFACE_H
#include <QAbstractVideoSurface>
#include <QVideoSurfaceFormat>
class VideoProbeSurface : public QAbstractVideoSurface {
Q_OBJECT
public:
VideoProbeSurface (QObject *parent = nullptr)
: QAbstractVideoSurface(parent)
, formatSource_(nullptr)
{
}
void setFormatSource (QAbstractVideoSurface *source) {
formatSource_ = source;
}
QList<QVideoFrame::PixelFormat> supportedPixelFormats (QAbstractVideoBuffer::HandleType type) const override {
return formatSource_ ? formatSource_->supportedPixelFormats(type)
: QList<QVideoFrame::PixelFormat>();
}
QVideoSurfaceFormat nearestFormat (const QVideoSurfaceFormat &format) const override {
return formatSource_ ? formatSource_->nearestFormat(format)
: QAbstractVideoSurface::nearestFormat(format);
}
bool present (const QVideoFrame &frame) override {
emit videoFrameProbed(frame);
return true;
}
signals:
void videoFrameProbed (const QVideoFrame &frame);
private:
QAbstractVideoSurface *formatSource_;
};
#endif // VIDEOPROBESURFACE_H
I went for the quickest-to-write implementation possible so it just forwards supported pixel formats from another surface (my intent was to both probe and play back to a QVideoWidget) and you get whatever format you get. I just needed to grab subimages into QImages though, which handles most common formats. But you could modify this to force any formats you want (e.g. you might want to just return formats supported by QImage or filter out source formats not supported by QImage), etc.).
Example set up:
QMediaPlayer *player = ...;
QVideoWidget *widget = ...;
// forward surface formats provided by the video widget:
VideoProbeSurface *probe = new VideoProbeSurface(...);
probe->setFormatSource(widget->videoSurface());
// same signal signature as QVideoProbe's signal:
connect(probe, &VideoProbeSurface::videoFrameProbed, ...);
// the key move is to render to both the widget (for viewing)
// and probe (for processing). fortunately, QMediaPlayer can
// take a list:
player->setVideoOutput({ widget->videoSurface(), probe });
Notes
The only really sketchy thing I had to do was const_cast the QVideoFrame on the receiver side (for read-only access), since QVideoFrame::map() isn't const:
if (const_cast<QVideoFrame&>(frame).map(QAbstractVideoBuffer::ReadOnly)) {
...;
const_cast<QVideoFrame&>(frame).unmap();
}
But the real QVideoProbe would make you do the same thing so, I don't know what's up with that -- it's a strange API. I ran some tests with sw, native hw, and copy-back hw renderers and decoders and map/unmap in read mode seem to be functioning OK, so, whatever.
Performance-wise, the video will bog down if you spend too much time in the callback, so design accordingly. However, I didn't test QueuedConnection, so I don't know if that'd still have the issue (although the fact that the signal parameter is a reference would make me wary of trying it, as well as conceivable issues with the GPU releasing the memory before the slot ends up being called). I don't know how QVideoProbe behaves in this regard, either. I do know that, at least on my machine, I can pack and queue Full HD (1920 x 1080) resolution QImages to a thread pool for processing without slowing down the video.
You probably also want to implement some sort of auto-unmapper utility object for exception safe unmap(), etc. But again, that's not unique to this, same thing you'd have to do with QVideoProbe.
So hopefully that helps somebody else.
Example QImage Use
PS, example of packing arbitrarily-formatted QVideoFrames into a QImage in:
void MyVideoProcessor::onFrameProbed(const QVideoFrame &frame) {
if (const_cast<QVideoFrame&>(frame).map(QAbstractVideoBuffer::ReadOnly)) {
auto imageFormat = QVideoFrame::imageFormatFromPixelFormat(frame.pixelFormat());
QImage image(frame.bits(), frame.width(), frame.height(), frame.bytesPerLine(), imageFormat);
// *if* you want to use this elsewhere you must force detach:
image = image.copy();
// but if you don't need to use it past unmap(), you can just
// use the original image instead of a copy.
// <---- now do whatever with the image, e.g. save() it.
// if you *haven't* copied the image, then, before unmapping,
// kill any internal data pointers just to be safe:
image = QImage();
const_cast<QVideoFrame&>(frame).unmap();
}
}
Notes about that:
Constructing a QImage directly from the data is fast and essentially free: no copies are done.
The data buffers are only technically valid between map and unmap, so if you intend to use the QImage outside of that scope, you'll want to use copy() (or anything else that forces a detach) to force a deep copy.
You also probably want to ensure that the original not-copied QImage is destructed before calling unmap. It's unlikely to cause problems but it's always a good idea to minimize how many invalid pointers are hanging around at any given time, and also the QImage docs say "The buffer must remain valid throughout the life of the QImage and all copies that have not been modified or otherwise detached from the original buffer". Best to be strict about it.

How to send data from server to client as QByteArray/QDataStream

In the fortuneserver sample of Qt, a QString is sent by the method sendFortune(). Therefore one QString is selected from the QStringList fortunes:
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out.setVersion(QDataStream::Qt_4_0);
out << (quint16)0;
out << fortunes.at(qrand() % fortunes.size());
out.device()->seek(0);
out << (quint16)(block.size() - sizeof(quint16));
QTcpSocket *clientConnection = tcpServer->nextPendingConnection();
clientConnection->write(block);
Is it also possible to send another type of data, like files, images or multiple strings? Or is it just possible to send a single string?
My second question: What is the advantage of sending data as QByteArry and why do I have to define (quint16) by setting up the QDataStream?
You don't send the data as QDataStream, QDataStream is a class that impersonates a stream, a way to transfer data, like wire.
QByteArray represents a storage for your data.
So, you can send data as QByteArray.
You can try QTcpSocket's member function called "int write(QByteArray)", like in the example you provided. Just take the image, the file, any other data and convert it to QByteArray. Here is where you will need QDataStream. Bind the stream to bytearray like this.
QByteArray dat;
QDataStream out(&dat, QIODevice::WriteOnly);
and use out to fill dat.
out << myImage << myImage2;
When you had finished filling the QByteArray, send it:
mySocket.write(dat);
don't forget to check the return value.
Read the docs and you will succeed.
To know if you have read all the data send by the other side of the socket, I use the commitTransaction() function from QDataStream:
Client::Client()
{
....
connect(tcpSocket, &QIODevice::readyRead, this, &Client::readData);
....
}
void Client::readData()
{
in.startTransaction();
QString data;
in >> data;
if (!in.commitTransaction())
{
qDebug() << TAG << "incomplete: " << data;
// readyRead will be called again when there is more data
return;
}
// data is complete, do something with it
....

Resources