gdi+ Graphics::DrawImage really slow~~ - gdi+

I am using a GDI+ Graphic to draw a 4000*3000 image to screen, but it is really slow. It takes about 300ms. I wish it just occupy less than 10ms.
Bitmap *bitmap = Bitmap::FromFile("XXXX",...);
//--------------------------------------------
// this part takes about 300ms, terrible!
int width = bitmap->GetWidth();
int height = bitmap->GetHeight();
DrawImage(bitmap,0,0,width,height);
//------------------------------------------
I cannot use CachedBitmap, because I want to edit the bitmap later.
How can I improve it? Or is any thing wrong?
This native GDI function also draws the image into the screen, and it just take 1 ms:
SetStretchBltMode(hDC, COLORONCOLOR);
StretchDIBits(hDC, rcDest.left, rcDest.top,
rcDest.right-rcDest.left, rcDest.bottom-rcDest.top,
0, 0, width, height,
BYTE* dib, dibinfo, DIB_RGB_COLORS, SRCCOPY);
//--------------------------------------------------------------
If I want to use StretchDIBits, I need to pass BITMAPINFO, But how can I get BITMAPINFO from a Gdi+ Bitmap Object? I did the experiment by FreeImage lib, I call StretchDIBits using FreeImageplus object, it draw really fast. But now I need to draw Bitmap, and write some algorithm on Bitmap's bits array, how can I get BITMAPINFO if I have an Bitmap object? It's really annoying -___________-|

If you're using GDI+, the TextureBrush class is what you need for rendering images fast. I've written a couple of 2d games with it, getting around 30 FPS or so.
I've never written .NET code in C++, so here's a C#-ish example:
Bitmap bmp = new Bitmap(...)
TextureBrush myBrush = new TextureBrush(bmp)
private void Paint(object sender, PaintEventArgs e):
{
//Don't draw the bitmap directly.
//Only draw TextureBrush inside the Paint event.
e.Graphics.FillRectangle(myBrush, ...)
}

You have a screen of 4000 x 3000 resolution? Wow!
If not, you should draw only the visible part of the image, it would be much faster...
[EDIT after first comment] My remark is indeed a bit stupid, I suppose DrawImage will mask/skip unneeded pixels.
After your edit (showing StretchDIBits), I guess a possible source of speed difference might come from the fact that StretchDIBits is hardware accelerated ("If the driver cannot support the JPEG or PNG file image" is a hint...) while DrawImage might be (I have no proof for that!) coded in C, relying on CPU power instead of GPU's one...
If I recall correctly, DIB images are fast (despite being "device independent"). See High Speed Win32 Animation: "use CreateDIBSection to do high speed animation". OK, it applies to DIB vs. GDI, in old Windows version (1996!) but I think it is still true.
[EDIT] Maybe Bitmap::GetHBITMAP function might help you to use StretchDIBits (not tested...).

Just a thought; instead of retrieving the width and height of the image before drawing, why not cache these values when you load the image?

Explore the impact of explicitly setting the interpolation mode to NearestNeighbor (where, in your example, it looks like interpolation is not actually needed! But 300ms is the kind of cost of doing high-quality interpolation when no interpolation is needed, so its worth a try)
Another thing to explore is changing the colour depth of the bitmap.

Unfortunately when I had a similar problem, I found that GDI+ is known to be much slower than GDI and not generally hardware accelerated, but now Microsoft have moved on to WPF they will not come back to improve GDI+!
All the graphics card manufacturers have moved onto 3D performance and don't seem interested in 2D acceleration, and there's no clear source of information on which functions are or can be hardware accelerated or not. Very frustrating because having written an app in .NET using GDI+, I am not happy to change to a completely different technology to speed it up to reasonable levels.

i don't think they'll make much of a different, but since you're not actually needing to resize the image, try using the overload of DrawImage that doesn't (attempt) to resize:
DrawImage(bitmap,0,0);
Like i said, i doubt it will make any difference, because i'm sure that DrawImage checks the Width and Height of the bitmap, and if there's no resizing needed, just calls this overload. (i would hope it doesn't bother going through all 12 million pixels performing no actual work).
Update: My ponderings are wrong. i had since found out, but guys comment reminded me of my old answer: you want to specify the destination size; even though it matches the source size:
DrawImage(bitmap, 0, 0, bitmap.GetWidth, bitmap.GetHeight);
The reason is because of dpi differences between the dpi of bitmap and the dpi of the destination. GDI+ will perform scaling to get the image to come out the right "size" (i.e. in inches)
What i've learned on my own since last October is that you really want to draw a "cached" version of your bitmap. There is a CachedBitmap class in GDI+. There are some tricks to using it. But in there end i have a function bit of (Delphi) code that does it.
The caveat is that the CachedBitmap can become invalid - meaning it can't be used to draw. This happens if the user changes resolutions or color depths (e.g. Remote Desktop). In that case the DrawImage will fail, and you have to re-created the CachedBitmap:
class procedure TGDIPlusHelper.DrawCachedBitmap(image: TGPImage;
var cachedBitmap: TGPCachedBitmap;
Graphics: TGPGraphics; x, y: Integer; width, height: Integer);
var
b: TGPBitmap;
begin
if (image = nil) then
begin
//i've chosen to not throw exceptions during paint code - it gets very nasty
Exit;
end;
if (graphics = nil) then
begin
//i've chosen to not throw exceptions during paint code - it gets very nasty
Exit;
end;
//Check if we have to invalidate the cached image because of size mismatch
//i.e. if the user has "zoomed" the UI
if (CachedBitmap <> nil) then
begin
if (CachedBitmap.BitmapWidth <> width) or (CachedBitmap.BitmapHeight <> height) then
FreeAndNil(CachedBitmap); //nil'ing it will force it to be re-created down below
end;
//Check if we need to create the "cached" version of the bitmap
if CachedBitmap = nil then
begin
b := TGDIPlusHelper.ResizeImage(image, width, height);
try
CachedBitmap := TGPCachedBitmap.Create(b, graphics);
finally
b.Free;
end;
end;
if (graphics.DrawCachedBitmap(cachedBitmap, x, y) <> Ok) then
begin
//The calls to DrawCachedBitmap failed
//The API is telling us we have to recreate the cached bitmap
FreeAndNil(cachedBitmap);
b := TGDIPlusHelper.ResizeImage(image, width, height);
try
CachedBitmap := TGPCachedBitmap.Create(b, graphics);
finally
b.Free;
end;
graphics.DrawCachedBitmap(cachedBitmap, x, y);
end;
end;
The cachedBitmap is passed in by reference. The first call to DrawCachedBitmap it cached version will be created. You then pass it in subsequent calls, e.g.:
Image imgPrintInvoice = new Image.FromFile("printer.png");
CachedBitmap imgPrintInvoiceCached = null;
...
int glyphSize = 16 * (GetCurrentDpi() / 96);
DrawCachedBitmap(imgPrintInvoice , ref imgPrintInvoiceCached , graphics,
0, 0, glyphSize, glyphSize);
i use the routine to draw glyphs on buttons, taking into account the current DPI. The same could have been used by the Internet Explorer team to draw images when the user is running high dpi (ie is very slow drawing zoomed images, because they use GDI+).

/*
First sorry for ma English, and the code is partly in polish, but it's simple to understand.
I had the same problem and I found the best solution. Here it is.
Dont use: Graphics graphics(hdc); graphics.DrawImage(gpBitmap, 0, 0); It is slow.
Use: GetHBITMAP(Gdiplus::Color(), &g_hBitmap) for HBITMAP and draw using my function ShowBitmapStretch().
You can resize it and it is much faster! Artur Czekalski / Poland
*/
//--------Global-----------
Bitmap *g_pGDIBitmap; //for loading picture
int gRozXOkna, gRozYOkna; //size of working window
int gRozXObrazu, gRozYObrazu; //Size of picture X,Y
HBITMAP g_hBitmap = NULL; //for displaying on window
//------------------------------------------------------------------------------
int ShowBitmapStretch(HDC hdc, HBITMAP hBmp, int RozX, int RozY, int RozXSkal, int RozYSkal, int PozX, int PozY)
{
if (hBmp == NULL) return -1;
HDC hdc_mem = CreateCompatibleDC(hdc); //utworzenie kontekstu pamięciowego
if (NULL == hdc_mem) return -2;
//Trzeba połączyć BMP z hdc_mem, tzn. umieścić bitmapę w naszym kontekście pamięciowym
if (DeleteObject(SelectObject(hdc_mem, hBmp)) == NULL) return -3;
SetStretchBltMode(hdc, COLORONCOLOR); //important! for smoothness
if (StretchBlt(hdc, PozX, PozY, RozXSkal, RozYSkal, hdc_mem, 0, 0, RozX, RozY, SRCCOPY) == 0) return -4;
if (DeleteDC(hdc_mem) == 0) return -5;
return 0; //OK
}
//---------------------------------------------------------------------------
void ClearBitmaps(void)
{
if (g_hBitmap) { DeleteObject(g_hBitmap); g_hBitmap = NULL; }
if (g_pGDIBitmap) { delete g_pGDIBitmap; g_pGDIBitmap = NULL; }
}
//---------------------------------------------------------------------------
void MyOpenFile(HWND hWnd, szFileName)
{
ClearBitmaps(); //Important!
g_pGDIBitmap = new Bitmap(szFileName); //load a picture from file
if (g_pGDIBitmap == 0) return;
//---Checking if picture was loaded
gRozXObrazu = g_pGDIBitmap->GetWidth();
gRozYObrazu = g_pGDIBitmap->GetHeight();
if (gRozXObrazu == 0 || gRozYObrazu == 0) return;
//---Uworzenie bitmapy do wyświatlaia; DO IT ONCE HERE!
g_pGDIBitmap->GetHBITMAP(Gdiplus::Color(), &g_hBitmap); //creates a GDI bitmap from this Bitmap object
if (g_hBitmap == 0) return;
//---We need to force the window to redraw itself
InvalidateRect(hWnd, NULL, TRUE);
UpdateWindow(hWnd);
}
//---------------------------------------------------------------------------
void MyOnPaint(HDC hdc, HWND hWnd) //in case WM_PAINT; DO IT MANY TIMES
{
if (g_hBitmap)
{
double SkalaX = 1.0, SkalaY = 1.0; //scale
if (gRozXObrazu > gRozXOkna || gRozYObrazu > gRozYOkna || //too big picture, więc zmniejsz;
(gRozXObrazu < gRozXOkna && gRozYObrazu < gRozYOkna)) //too small picture, można powiększyć
{
SkalaX = (double)gRozXOkna / (double)gRozXObrazu; //np. 0.7 dla zmniejszania; FOR DECREASE
SkalaY = (double)gRozYOkna / (double)gRozYObrazu; //np. 1.7 dla powiększania; FOR INCREASE
if (SkalaY < SkalaX) SkalaX = SkalaY; //ZAWSZE wybierz większe skalowanie, czyli mniejszą wartość i utaw w SkalaX
}
if (ShowBitmapStretch(hdc, g_hBitmap, gRozXObrazu, gRozYObrazu, (int)(gRozXObrazu*SkalaX), (int)(gRozYObrazu*SkalaX), 0, 0, msg) < 0) return;

Try using copy of Bitmap from file. FromFile function on some files returns "slow" image, but its copy will draw faster.
Bitmap *bitmap = Bitmap::FromFile("XXXX",...);
Bitmap *bitmap2 = new Bitmap(bitmap); // make copy
DrawImage(bitmap2,0,0,width,height);

I have made some researching and wasn't able to find a way to render images with GDI/GDI+ more faster than
Graphics.DrawImage/DrawImageUnscaled
and at the same time simple like it.
Till I discovered
ImageList.Draw(GFX,Point,Index)
and yeah it's really so fast and simple.

Related

Qt, Is there a more efficient way to crop out part of an Qimage?

I am making a simple editor where the user can click on points of an image and crop out a shape. My implementation is terribly inefficient and as I'm new to qt, I have trouble deciphering all the functions on qt's docs.
QPolygonF polygon(points);
std::map<std::string, int> map = pointsHandler.getOutsideVals();
for(int i = map["Left"]; i < map["Right"]; i++){
for(int j = map["Top"]; j < map["Bottom"]; j++){
for(int n = 0; n < points.size(); n++){
if(polygon.containsPoint(QPointF(i,j), Qt::OddEvenFill)){
image.setPixelColor(QPoint(i - xOffset, j - yOffset), Qt::transparent);
}
}
}
}
painter.drawImage(xOffset,yOffset, image);
Currently how I'm doing it is looping through a rectangle given by the outer most points of the polygon. If a point is in the polygon or not I change the pixel value to be transparent. The polygon is made from the users clicked points which I then store the outer most values in a map. When I crop out large portions, it takes far to long and was I looking for some advice to make this more efficient. Thank you.
EDIT
I am now using setClipPath mentioned by G.M. and have no performance issues, however the way I found to get the job done now seems like a waste of memory. Using setClipPath(...) the best work around I found was to make multiple Qt class objects on the stack, it works great just seems like I'm working around to much stuff. Here's the updated code.
QPolygon clipPolygon = QPolygonF(points).toPolygon();
QRegion clippedRegion(clipPolygon, Qt::OddEvenFill);
QRect translatedImageRect = image.rect().translated(QPoint(xOffset, yOffset));
QRegion unClippedRegion = QRegion(translatedImageRect).subtracted(clippedRegion);
painter.save();
painter.setClipRegion(unClippedRegion, Qt::ReplaceClip);
painter.drawImage(xOffset, yOffset, image);
painter.restore();
It works great, just feel like I'm wasting memory.
You can use QPainter to make a rectangle of your image transparent.
QImage image("/home/tim/Bilder/Example.png");
QPainter painter(&image);
painter.setCompositionMode(QPainter::CompositionMode_Source);
painter.fillRect(0, 0, 10, 10, Qt::transparent);
painter.end();
image.save("/home/tim/Bilder/changed.png", "PNG");

JavaFx ImageViewer from unsigned short array

I want to display an image received in a short[] of pixels from a server.
The server(C++) writes the image as an unsigned short[] of pixels (12 bit depth).
My java application gets the image by a CORBA call to this server.
Since java does not have ushort, the pixels are stored as (signed) short[].
This is the code I'm using to obtain a BufferedImage from the array:
private WritableImage loadImage(short[] pixels, int width, int height) {
int[] intPixels = new int[pixels.length];
for (int i = 0; i < pixels.length; i++) {
intPixels[i] = (int) pixels[i];
}
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
WritableRaster raster = (WritableRaster) image.getData();
raster.setPixels(0, 0, width, height, intPixels);
return SwingFXUtils.toFXImage(image, null);
}
And later:
WritableImage orgImage = convertShortArrayToImage2(image.data, image.size_x, image.size_y);
//load it into the widget
Platform.runLater(() -> {
imgViewer.setImage(orgImage);
});
I've checked that width=1280 and height=1024 and the pixels array is 1280x1024, that matches with the raster height and width.
However I'm getting an array out of bounds error in the line:
raster.setPixels(0, 0, width, height, intPixels);
I have try with ALL ImageTypes , and all of them produce the same error except for:
TYPE_USHORT_GRAY: Which I thought it would be the one, but shows an all-black image
TYPE_BYTE_GRAY: which show the image in negative(!) and with a lot of grain(?)
TYPE_BYTE_INDEXED: which likes the above what colorized in a funny way
I also have tried shifting bits when converting from shot to int, without any difference:
intPixels[i] = (int) pixels[i] & 0xffff;
So..I'm quite frustrated after looking for days a solution in the internet. Any help is very welcome
Edit. The following is an example of the images received, converted to jpg on the server side. Not sure if it is useful since I think it is made from has pixel rescaling (sqrt) :
Well, finally I solved it.
Probably not the best solution but it works and could help someone in ether....
Being the image grayscale 12 bit depth, I used BufferedImage of type TYPE_BYTE_GRAY, but I had to downsample to 8 bit scaling the array of pixels. from 0-4095 to 0-255.
I had an issue establishing the higher and lower limits of the scale. I tested with avg of the n higher/lower limits, which worked reasonably well, until someone sent me a link to a java program translating the zscale algorithm (used in DS9 tool for example) for getting the limits of the range of greyscale vlues to be displayed:
find it here
from that point I modified the previous code and it worked like a charm:
//https://github.com/Caltech-IPAC/firefly/blob/dev/src/firefly/java/edu/caltech/ipac/visualize/plot/Zscale.java
Zscale.ZscaleRetval retval = Zscale.cdl_zscale(pixels, width, height,
bitsVal, contrastVal, opt_sizeVal, len_stdlineVal, blankValueVal);
double Z1 = retval.getZ1();
double Z2 = retval.getZ2();
try {
int[] ints = new int[pixels.length];
for (int i = 0; i < pixels.length; i++) {
if (pixels[i] < Z1) {
pixels[i] = (short) Z1;
} else if (pixels[i] > Z2) {
pixels[i] = (short) Z2;
}
ints[i] = ((int) ((pixels[i] - Z1) * 255 / (Z2 - Z1)));
}
BufferedImage bImg
= new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
bImg.getRaster().setPixels(0, 0, width, height, ints);
return SwingFXUtils.toFXImage(bImg, null);
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
return null;

Efficient conversion of AVFrame to QImage

I need to extract frames from a video in my Qt based application. Using ffmpeg libraries I am able to fetch frames as AVFrames which I need to convert to QImage to use in other parts of my application. This conversion needs to be efficient. So far it seems sws_scale() is the right function to use but I am not sure what source and destination pixel formats are to be specified.
Came up with the following 2-step process that first converts a decoded AVFame to another AVFrame in RGB colorspace and then to QImage. It works and is reasonably fast.
src_frame = get_decoded_frame();
AVFrame *pFrameRGB = avcodec_alloc_frame(); // intermediate pframe
if(pFrameRGB==NULL) {
;// Handle error
}
int numBytes= avpicture_get_size(PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
uint8_t *buffer = (uint8_t*)malloc(numBytes);
avpicture_fill((AVPicture*)pFrameRGB, buffer, PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
int dst_fmt = PIX_FMT_RGB24;
int dst_w = is->video_st->codec->width;
int dst_h = is->video_st->codec->height;
// TODO: cache following conversion context for speedup,
// and recalculate only on dimension changes
SwsContext *img_convert_ctx_temp;
img_convert_ctx_temp = sws_getContext(
is->video_st->codec->width, is->video_st->codec->height,
is->video_st->codec->pix_fmt,
dst_w, dst_h, (PixelFormat)dst_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGB32);
sws_scale(img_convert_ctx_temp,
src_frame->data, src_frame->linesize, 0, is->video_st->codec->height,
pFrameRGB->data,
pFrameRGB->linesize);
uint8_t *src = (uint8_t *)(pFrameRGB->data[0]);
for (int y = 0; y < dst_h; y++)
{
QRgb *scanLine = (QRgb *) myImage->scanLine(y);
for (int x = 0; x < dst_w; x=x+1)
{
scanLine[x] = qRgb(src[3*x], src[3*x+1], src[3*x+2]);
}
src += pFrameRGB->linesize[0];
}
If you find a more efficient approach, let me know in the comments
I know, it's too late, but maybe someone will find it useful. From here I got the clue of doing the same conversion, which looks a bit shorter.
So I created QImage which is reused for every decoded frame:
QImage img( width, height, QImage::Format_RGB888 );
Created frameRGB:
frameRGB = av_frame_alloc();
//Allocate memory for the pixels of a picture and setup the AVPicture fields for it.
avpicture_alloc( ( AVPicture *) frameRGB, AV_PIX_FMT_RGB24, width, height);
After the the first frame is decoded I create conversion context SwsContext this way (it will be used for all the next frames):
mImgConvertCtx = sws_getContext( codecContext->width, codecContext->height, codecContext->pix_fmt, width, height, AV_PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);
And finally for every decoded frame conversion is performed:
if( 1 == framesFinished && nullptr != imgConvertCtx )
{
//conversion frame to frameRGB
sws_scale(imgConvertCtx, frame->data, frame->linesize, 0, codecContext->height, frameRGB->data, frameRGB->linesize);
//setting QImage from frameRGB
for( int y = 0; y < height; ++y )
memcpy( img.scanLine(y), frameRGB->data[0]+y * frameRGB->linesize[0], mWidth * 3 );
}
See the link for the specifics.
A simpler approach, I think:
void takeSnapshot(AVCodecContext* dec_ctx, AVFrame* frame)
{
SwsContext* img_convert_ctx;
img_convert_ctx = sws_getContext(dec_ctx->width,
dec_ctx->height,
dec_ctx->pix_fmt,
dec_ctx->width,
dec_ctx->height,
AV_PIX_FMT_RGB24,
SWS_BICUBIC, NULL, NULL, NULL);
AVFrame* frameRGB = av_frame_alloc();
avpicture_alloc((AVPicture*)frameRGB,
AV_PIX_FMT_RGB24,
dec_ctx->width,
dec_ctx->height);
sws_scale(img_convert_ctx,
frame->data,
frame->linesize, 0,
dec_ctx->height,
frameRGB->data,
frameRGB->linesize);
QImage image(frameRGB->data[0],
dec_ctx->width,
dec_ctx->height,
frameRGB->linesize[0],
QImage::Format_RGB888);
image.save("capture.png");
}
Today, I have tested directly pass the image->bit() to swscale and finally it works, so it doesn't need to copy to memory. For example:
/* 1. Get frame and QImage to show */
struct my_frame *frame = get_frame(source);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGBA8888);
/* 2. Convert and write into image buffer */
uint8_t *dst[] = {myImage->bits()};
int linesizes[4];
av_image_fill_linesizes(linesizes, AV_PIX_FMT_RGBA, frame->width);
sws_scale(myswscontext, frame->data, (const int*)frame->linesize,
0, frame->height, dst, linesizes);
I just discovered that scanLine is just seeking thru the buffer.. all you need is use AV_PIX_FMT_RGB32 for the AVFrame and QImage::FORMAT_RGB32 for the QImage.
Then after decoding just do a memcpy
memcpy(img.scanLine(0), pFrameRGB->data[0], pFrameRGB->linesize[0] * pFrameRGB->height());
I had problems with the other proposed solutions as :
They did not mention freeing either AVFrame, SwsContext or the allocated buffers, which caused massive memory leaks (I had thousands of frames to handle). These problems couldn't all be solved easily as QImage relies on the underlying data, and does not copy it. If freeing the buffer directly, the QImage points to freed data and breaks. This could be solved by using QImage's cleanupFunction to free the buffer once the image is no longer needed, but with other problems it wasn't good anyways.
In some cases one of the suggestions, of passing QImage.bits directly to sws_scale, would not work as QImage are minimum 32 bit aligned. Therefore for certain dimensions it would not match the expected width by sws_scale and output each line shifted a little bit.
A third problem is that they used deprecated AVPicture elements.
I listed the problem in another question Converting an AVFrame to QImage with conversion of pixel format and in the end found a solution using a temporary buffer, which could be copied to the QImage, and then safely freed.
So see my answer for a fully working, efficient, and with no deprecated function calls, implementation : https://stackoverflow.com/a/68212609/7360943

cv::Mat to QImage conversion

I've found very similiar topic: how to convert an opencv cv::Mat to qimage , but it does not solve my problem.
I have function converting cv::Mat to QImage
QImage cvMatToQImg(cv::Mat& mat)
{
cv::Mat rgb;
if(mat.channels()==1)
{
cv::cvtColor(mat,rgb,CV_GRAY2BGR);
cv::cvtColor(rgb,rgb,CV_BGR2BGRA);
QImage temp = QImage((unsigned char*)(rgb.data), rgb.cols,
rgb.rows,QImage::Format_ARGB32 );
QImage returnImage = temp.copy();
return returnImage;
}
And it's works for my but I want to make it more efficient.
First: why changing 2 cvtColor functions with:
cv::cvtColor(mat,rgb,CV_GRAY2BGRA)
fails on
QImage returnImage = temp.copy()
with segfault.
Then how to eliminate copying of QImage. When I simply return temp image, I'm getting segfault.
Any other optimalizations can be done there? It's very often used function so I want to make it as fast as possible.
Your solution to the problem is not efficient, in particular it is less efficient then the code I posted on the other question you link to.
Your problem is that you have to convert from grayscale to color, or RGBA. As soon as you need this conversation, naturally a copy of the data is needed.
My solution does the conversion between grayscale and color, as well as between cv::Mat and QImage at the same time. That's why it is the most efficient you can get.
In your solution, you first try to convert and then want to build QImage around OpenCV data directly to spare a second copy. But, the data you point to is temporary. As soon as you leave the function, the cv::Mat free's its associated memory and that's why it is not valid anymore also within the QImage. You could manually increase the reference counter of the cv::Mat beforehand, but that opens the door for a memory leak afterwards.
In the end, you attempt a dirty solution to a problem better solved in a clean fashion.
It may be easiest to roll your own solution. Below is the current OpenCV implementation for going from gray to RGBA format:
template<typename _Tp>
struct Gray2RGB
{
typedef _Tp channel_type;
Gray2RGB(int _dstcn) : dstcn(_dstcn) {}
void operator()(const _Tp* src, _Tp* dst, int n) const
{
if( dstcn == 3 )
for( int i = 0; i < n; i++, dst += 3 )
{
dst[0] = dst[1] = dst[2] = src[i];
}
else
{
_Tp alpha = ColorChannel<_Tp>::max();
for( int i = 0; i < n; i++, dst += 4 )
{
dst[0] = dst[1] = dst[2] = src[i];
dst[3] = alpha;
}
}
}
int dstcn;
};
Here is where the actual cvtColor call occurs:
case CV_GRAY2BGR: case CV_GRAY2BGRA:
if( dcn <= 0 ) dcn = 3;
CV_Assert( scn == 1 && (dcn == 3 || dcn == 4));
_dst.create(sz, CV_MAKETYPE(depth, dcn));
dst = _dst.getMat();
if( depth == CV_8U )
CvtColorLoop(src, dst, Gray2RGB<uchar>(dcn));
This code is contained in the color.cpp file in the imgproc library.
As you can see, since you are not setting the dstCn parameter in your cvtColor calls, it defaults to dcn = 3. To go straight from gray to BGRA, set dstCn to 4. Since OpenCV's default color order is BGR, you'll still need to swap the color channels for it to look right (assuming you get your image data from an OpenCV function). So, it may be worth it to implement your own converter possibly following the above example, or using ypnos answer here.
Also, have a look at my other answer involving how to integrate OpenCV with Qt.
The problem is that both the cv::Mat and QImage data isn't necessarily contiguous.
New data rows in opencv start on a 32bit boundary (not sure about QImage - I think it's system dependant) so you can't copy a memeory block unless your rows happen to be exact multiples of 4bytes
See How to output this 24 bit image in Qt

Large dimensioned images lead into an OufOfMemoryException

I am using OpenMap and have to load very large dimensioned images.
I tried to load these images as big raster which fails with an OufOfMemoryException. In debug mode the layer constructor tells me that the image dimensions are too large.
In an OpenMap mailing list I found the MyJAIPlugin, which allows me to load and display GeoTiff files.
How can I show a 300mb GeoTiff in OpenMap?
I had a nearly same situation by loading hd maps with at least 690mb filesize.
I also used the JAIPlugIn from the mailing list and internaly they use the OMScalingRaster witch works with a BufferedImage. These limits your image size and causes the debug message.
I've solved it by modifieing the OMScalingRaster. I've changed the BufferedImage to a TiledImage to handle large images and fixed the upcoming errors. Here it's important you change the scaleTo(Projection thisProj)-method, to scale with JAI.
Now i can load the file and it's rendered on the map. But if you are zooming out too much, it will throw a OutOfMemoryException because in my modification i make a subimage of the part of the image that will be visible and give it as BufferedImage to the OMRaster.
Here is the mod. at the end of the scaleTo-method:
// Now we can grab the bit we want out of the source
// and
// scale it to fit the intersection.
// Calc width adjustment
float widthAdj = (float) ((double) iRect.width
/ (double) clipRect.width);
// Calc height adjustment
float heightAdj = (float) ((double) iRect.height
/ (double) clipRect.height);
// Create the transform
// JAI-Version
ParameterBlock pb = new ParameterBlock();
pb.addSource(sourceImage.getSubImage(clipRect.x,
clipRect.y,
clipRect.width,
clipRect.height).getAsBufferedImage());
pb.add(widthAdj); // The xScale
pb.add(heightAdj); // The yScale
pb.add(0.0F); // The x translation
pb.add(0.0F); // The y translation
RenderedOp newImage = JAI.create("scale",pb, null);
bitmap = newImage.getAsBufferedImage();
point1.setLocation(iRect.x, iRect.y);
// setVisible(currentVisibility);
}
} else {
bitmap = null;
}
}
For the other errors by replacing BufferedImage with TiledImage use the equivalent TiledImage-methods. But to save memory you should use the TiledImage-constructor with the sharedDataBuffer flag = true.
For Exsample this mod. can handle maps (compressed 690mb) with a scaling of 1:50000 and i can zoom out to 1:600000 before the layer says it run out of memory.

Resources