Related
For a list of colors, I have a corresponding list of QImage, format mono. The mono images have been processed in such a way that a single pixel can be black from all images.
I would like to merge them into a color image.
I had 3 ideas.
1. Use image composition modes. I was unable to get it to work. (Editing to remove it, to clean up post...)
2. Use the mono images as masks for each of the colors when added to the destination.
I have no idea how to implement it.
3. Iterating through pixels - slow, the documentation says that pixel manipulation functions are slow...
This works:
// Creating destination image
// m_colors: list of (n+1) QCcolor (last one corresponding to background)
// m_images: list of n QImage, Format_Mono, all of the same size (as the destination)
// using indexed image for destination since I have a limited palette
QImage preview = QImage(rect.size().toSize(), QImage::Format_Indexed8);
int previewWidth = preview.size().width();
int previewHeight = preview.size().height();
int colorsSize = m_colors.size();
for(int k = 0; k < colorsSize; ++k)
{
preview.setColor(k, m_colors.at(k).rgb());
}
--colorsSize;
// combining images
for(int j = 0; j < previewHeight; ++j)
{
for(int i = 0; i < previewWidth; ++i)
{
// set background color
preview.setPixel(i, j, colorsSize);
for(int k = 0; k < colorsSize; ++k)
{
QImage im = m_images.at(k);
if(!im.isNull())
{
if(m_images.at(k).pixelIndex(i, j) == 0)
{
preview.setPixel(i, j, k);
}
}
}
}
}
I should at least improve this using scanLine(), but don't know how... I can only find examples that use scanLine() with 32 bit images, not 8 or 2.
Is it actually possible to use scanLine() with 8 or 2 bit images ?
I don't understand the documentation - does it mean that only 32 bit images can be read/written using scanLine() or that regardless of image type, the function will work the same way, and I only use one of the 4 bytes ?
Would be more effective to use 32 bit images instead of 8 or 2 bit ?
If I use 32 bit image for destination, and try to use scanLine() to write data, still how can I improve my reading of the mono images ?
Please help me improve my algorithm, either to improve the version that I get to work iterating through all pixels of images, or perhaps using some tools like combining images using composition.
Is it actually possible to use scanLine() with 8 or 2 bit images ?
Yes it is.
Would be more effective to use 32 bit images instead of 8 or 2 bit ?
You will have to measure, and it will depend on the specific code. I used 8 bit code here for simplicity and because your code did.
If I use 32 bit image for destination, and try to use scanLine() to write data, still how can I improve my reading of the mono images ?
It is probably not a good idea to copy the image in the inner loop
QImage im = m_images.at(k)
and to then not use that copy for the next access.
if(m_images.at(k).pixelIndex(i, j) == 0)
It should speed up your painting if your inner loop iterates over an image instead of iterating over the destination pixels in the inner loop.
If the image is monochrome, then the scan line will point to packed color information which will need to be unpacked. It is easier (and maybe faster) to let convertToFormat convert the image and then use scanLine to read the unpacked information. In the example below the images are all 8 bit.
#include<vector>
#include <QtGui/QImage>
#include <QtGui/QColor>
static char * img1[] = {
"5 5 2 1", "a c #000000", "b c #ffffff",
"aabba", "aabba", "aabba", "aabba", "aabba"
};
static char * img2[] = {
"5 5 2 1","a c #000000","b c #ffffff",
"aaaaa", "aaaaa", "bbbbb", "bbbbb", "aaaaa"
};
int main( int argc, char* arg[] )
{
auto images = std::vector<QImage>( 2 );
images[0] = QImage( img1 );
images[1] = QImage( img2 );
auto colors = std::vector<QColor>( 2 );
colors[0] = QColor( Qt::red );
colors[1] = QColor( Qt::green );
QImage combined = QImage( images[0].size(), QImage::Format_Indexed8 );
combined.setColor( 0, Qt::black );
combined.fill(0);
for( int k = 1, num = images.size(); k <= num; ++k )
{
combined.setColor( k, colors[k-1].rgb() );
QImage img= images[k-1];
for( int i = 0, height = img.height(); i < height ; ++i )
{
uchar* src = img.scanLine(i);
uchar* dst = combined.scanLine(i);
for( int j = 0, width = mono.width(); j < width; ++j )
{
if( src[j] != 0 )
dst[j] = k;
}
}
}
I need to extract frames from a video in my Qt based application. Using ffmpeg libraries I am able to fetch frames as AVFrames which I need to convert to QImage to use in other parts of my application. This conversion needs to be efficient. So far it seems sws_scale() is the right function to use but I am not sure what source and destination pixel formats are to be specified.
Came up with the following 2-step process that first converts a decoded AVFame to another AVFrame in RGB colorspace and then to QImage. It works and is reasonably fast.
src_frame = get_decoded_frame();
AVFrame *pFrameRGB = avcodec_alloc_frame(); // intermediate pframe
if(pFrameRGB==NULL) {
;// Handle error
}
int numBytes= avpicture_get_size(PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
uint8_t *buffer = (uint8_t*)malloc(numBytes);
avpicture_fill((AVPicture*)pFrameRGB, buffer, PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
int dst_fmt = PIX_FMT_RGB24;
int dst_w = is->video_st->codec->width;
int dst_h = is->video_st->codec->height;
// TODO: cache following conversion context for speedup,
// and recalculate only on dimension changes
SwsContext *img_convert_ctx_temp;
img_convert_ctx_temp = sws_getContext(
is->video_st->codec->width, is->video_st->codec->height,
is->video_st->codec->pix_fmt,
dst_w, dst_h, (PixelFormat)dst_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGB32);
sws_scale(img_convert_ctx_temp,
src_frame->data, src_frame->linesize, 0, is->video_st->codec->height,
pFrameRGB->data,
pFrameRGB->linesize);
uint8_t *src = (uint8_t *)(pFrameRGB->data[0]);
for (int y = 0; y < dst_h; y++)
{
QRgb *scanLine = (QRgb *) myImage->scanLine(y);
for (int x = 0; x < dst_w; x=x+1)
{
scanLine[x] = qRgb(src[3*x], src[3*x+1], src[3*x+2]);
}
src += pFrameRGB->linesize[0];
}
If you find a more efficient approach, let me know in the comments
I know, it's too late, but maybe someone will find it useful. From here I got the clue of doing the same conversion, which looks a bit shorter.
So I created QImage which is reused for every decoded frame:
QImage img( width, height, QImage::Format_RGB888 );
Created frameRGB:
frameRGB = av_frame_alloc();
//Allocate memory for the pixels of a picture and setup the AVPicture fields for it.
avpicture_alloc( ( AVPicture *) frameRGB, AV_PIX_FMT_RGB24, width, height);
After the the first frame is decoded I create conversion context SwsContext this way (it will be used for all the next frames):
mImgConvertCtx = sws_getContext( codecContext->width, codecContext->height, codecContext->pix_fmt, width, height, AV_PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);
And finally for every decoded frame conversion is performed:
if( 1 == framesFinished && nullptr != imgConvertCtx )
{
//conversion frame to frameRGB
sws_scale(imgConvertCtx, frame->data, frame->linesize, 0, codecContext->height, frameRGB->data, frameRGB->linesize);
//setting QImage from frameRGB
for( int y = 0; y < height; ++y )
memcpy( img.scanLine(y), frameRGB->data[0]+y * frameRGB->linesize[0], mWidth * 3 );
}
See the link for the specifics.
A simpler approach, I think:
void takeSnapshot(AVCodecContext* dec_ctx, AVFrame* frame)
{
SwsContext* img_convert_ctx;
img_convert_ctx = sws_getContext(dec_ctx->width,
dec_ctx->height,
dec_ctx->pix_fmt,
dec_ctx->width,
dec_ctx->height,
AV_PIX_FMT_RGB24,
SWS_BICUBIC, NULL, NULL, NULL);
AVFrame* frameRGB = av_frame_alloc();
avpicture_alloc((AVPicture*)frameRGB,
AV_PIX_FMT_RGB24,
dec_ctx->width,
dec_ctx->height);
sws_scale(img_convert_ctx,
frame->data,
frame->linesize, 0,
dec_ctx->height,
frameRGB->data,
frameRGB->linesize);
QImage image(frameRGB->data[0],
dec_ctx->width,
dec_ctx->height,
frameRGB->linesize[0],
QImage::Format_RGB888);
image.save("capture.png");
}
Today, I have tested directly pass the image->bit() to swscale and finally it works, so it doesn't need to copy to memory. For example:
/* 1. Get frame and QImage to show */
struct my_frame *frame = get_frame(source);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGBA8888);
/* 2. Convert and write into image buffer */
uint8_t *dst[] = {myImage->bits()};
int linesizes[4];
av_image_fill_linesizes(linesizes, AV_PIX_FMT_RGBA, frame->width);
sws_scale(myswscontext, frame->data, (const int*)frame->linesize,
0, frame->height, dst, linesizes);
I just discovered that scanLine is just seeking thru the buffer.. all you need is use AV_PIX_FMT_RGB32 for the AVFrame and QImage::FORMAT_RGB32 for the QImage.
Then after decoding just do a memcpy
memcpy(img.scanLine(0), pFrameRGB->data[0], pFrameRGB->linesize[0] * pFrameRGB->height());
I had problems with the other proposed solutions as :
They did not mention freeing either AVFrame, SwsContext or the allocated buffers, which caused massive memory leaks (I had thousands of frames to handle). These problems couldn't all be solved easily as QImage relies on the underlying data, and does not copy it. If freeing the buffer directly, the QImage points to freed data and breaks. This could be solved by using QImage's cleanupFunction to free the buffer once the image is no longer needed, but with other problems it wasn't good anyways.
In some cases one of the suggestions, of passing QImage.bits directly to sws_scale, would not work as QImage are minimum 32 bit aligned. Therefore for certain dimensions it would not match the expected width by sws_scale and output each line shifted a little bit.
A third problem is that they used deprecated AVPicture elements.
I listed the problem in another question Converting an AVFrame to QImage with conversion of pixel format and in the end found a solution using a temporary buffer, which could be copied to the QImage, and then safely freed.
So see my answer for a fully working, efficient, and with no deprecated function calls, implementation : https://stackoverflow.com/a/68212609/7360943
I've found very similiar topic: how to convert an opencv cv::Mat to qimage , but it does not solve my problem.
I have function converting cv::Mat to QImage
QImage cvMatToQImg(cv::Mat& mat)
{
cv::Mat rgb;
if(mat.channels()==1)
{
cv::cvtColor(mat,rgb,CV_GRAY2BGR);
cv::cvtColor(rgb,rgb,CV_BGR2BGRA);
QImage temp = QImage((unsigned char*)(rgb.data), rgb.cols,
rgb.rows,QImage::Format_ARGB32 );
QImage returnImage = temp.copy();
return returnImage;
}
And it's works for my but I want to make it more efficient.
First: why changing 2 cvtColor functions with:
cv::cvtColor(mat,rgb,CV_GRAY2BGRA)
fails on
QImage returnImage = temp.copy()
with segfault.
Then how to eliminate copying of QImage. When I simply return temp image, I'm getting segfault.
Any other optimalizations can be done there? It's very often used function so I want to make it as fast as possible.
Your solution to the problem is not efficient, in particular it is less efficient then the code I posted on the other question you link to.
Your problem is that you have to convert from grayscale to color, or RGBA. As soon as you need this conversation, naturally a copy of the data is needed.
My solution does the conversion between grayscale and color, as well as between cv::Mat and QImage at the same time. That's why it is the most efficient you can get.
In your solution, you first try to convert and then want to build QImage around OpenCV data directly to spare a second copy. But, the data you point to is temporary. As soon as you leave the function, the cv::Mat free's its associated memory and that's why it is not valid anymore also within the QImage. You could manually increase the reference counter of the cv::Mat beforehand, but that opens the door for a memory leak afterwards.
In the end, you attempt a dirty solution to a problem better solved in a clean fashion.
It may be easiest to roll your own solution. Below is the current OpenCV implementation for going from gray to RGBA format:
template<typename _Tp>
struct Gray2RGB
{
typedef _Tp channel_type;
Gray2RGB(int _dstcn) : dstcn(_dstcn) {}
void operator()(const _Tp* src, _Tp* dst, int n) const
{
if( dstcn == 3 )
for( int i = 0; i < n; i++, dst += 3 )
{
dst[0] = dst[1] = dst[2] = src[i];
}
else
{
_Tp alpha = ColorChannel<_Tp>::max();
for( int i = 0; i < n; i++, dst += 4 )
{
dst[0] = dst[1] = dst[2] = src[i];
dst[3] = alpha;
}
}
}
int dstcn;
};
Here is where the actual cvtColor call occurs:
case CV_GRAY2BGR: case CV_GRAY2BGRA:
if( dcn <= 0 ) dcn = 3;
CV_Assert( scn == 1 && (dcn == 3 || dcn == 4));
_dst.create(sz, CV_MAKETYPE(depth, dcn));
dst = _dst.getMat();
if( depth == CV_8U )
CvtColorLoop(src, dst, Gray2RGB<uchar>(dcn));
This code is contained in the color.cpp file in the imgproc library.
As you can see, since you are not setting the dstCn parameter in your cvtColor calls, it defaults to dcn = 3. To go straight from gray to BGRA, set dstCn to 4. Since OpenCV's default color order is BGR, you'll still need to swap the color channels for it to look right (assuming you get your image data from an OpenCV function). So, it may be worth it to implement your own converter possibly following the above example, or using ypnos answer here.
Also, have a look at my other answer involving how to integrate OpenCV with Qt.
The problem is that both the cv::Mat and QImage data isn't necessarily contiguous.
New data rows in opencv start on a 32bit boundary (not sure about QImage - I think it's system dependant) so you can't copy a memeory block unless your rows happen to be exact multiples of 4bytes
See How to output this 24 bit image in Qt
I'm trying to print a image from a Dicom file. I pass the raw data to a convertToFormat_RGB888 function. As far as I know, Qt can't handle monochrome 16 bits images.
Here's the original image (converted to jpg here):
http://imageshack.us/photo/my-images/839/16bitc.jpg/
bool convertToFormat_RGB888(gdcm::Image const & gimage, char *buffer, QImage* &imageQt)
Inside this function, I get inside this...
...
else if (gimage.GetPixelFormat() == gdcm::PixelFormat::UINT16)
{
short *buffer16 = (short*)buffer;
unsigned char *ubuffer = new unsigned char[dimX*dimY*3];
unsigned char *pubuffer = ubuffer;
for (unsigned int i = 0; i < dimX*dimY; i++)
{
*pubuffer++ = *buffer16;
*pubuffer++ = *buffer16;
*pubuffer++ = *buffer16;
buffer16++;
}
imageQt = new QImage(ubuffer, dimX, dimY, QImage::Format_RGB888);
...
This code is a little adaptation from here:
gdcm.sourceforge.net/2.0/html/ConvertToQImage_8cxx-example.html
But the original one I got a execution error. Using mine at least I get an image, but it's not the same.
Here is the new image (converted to jpg here):
http://imageshack.us/photo/my-images/204/8bitz.jpg/
What am I doing wrong?
Thanks.
Try to get values of pixels from buffer manually and pass it to QImage::setPixel. It can be simplier.
You are assigning 16-bit integer to 8-bit variables here:
*pubuffer++ = *buffer16;
The result is undefined and most compilers just move the lower 8 bits to the destination. You want the upper 8 bits
*pubuffer++ = (*buffer16) >> 8;
The other issue is endianness. Depending to the endianness of the source data, you may need to call one of the QtEndian functions.
Lastly, you don't really need to use any of the 32 or 24-bit Qt image formats. Use 8-bit QImage::Format_Indexed8 and set the color table to grays.
this command line:
QImage:: QImage (uchar * data, int width, int height, int bytesPerLine, Format format)
Would use is it so?
QImage image = new QImage (buffer, 600, 400, jpg)
the bytesPerLine not they mean well, will the photo occupies kb?
thanks
If you do not want to use the bytesPerLine parameter, there is a
QImage::QImage ( uchar * data, int width, int height, Format format )
constructor.
However, Format is not what you might think. Theformatparameter specifies an enum value which decides over the bit depth etc. I.e. enteringjpgor"jpg"there won't work. Check Format-enum for a list of possible values.
I will try to answer the best I can considering the fact that your question is very unclear to me.
From the Qt documentation:
bytesPerLine specifies the number of bytes per line (stride)
Also consider that the format argument, which you specified as jpg, must be given as one of the enum values specified in here.
Best regards
That's how you would use this constructor:
int imageWidth = 800;
int imageHeight = 600;
int bytesPerPixel = 4; // 4 for RGBA, 3 for RGB
int format = QImage::Format_ARGB32; // this is the pixel format - check Qimage::Format enum type for more options
QImage image(yourData, imageWidth, imageHeight, imageWidth * bytesPerPixel, format);
You don't specify the image format (png, jpeg, etc.) but the pixel format (RGB, RGBA, etc.)