JavaFx ImageViewer from unsigned short array - javafx

I want to display an image received in a short[] of pixels from a server.
The server(C++) writes the image as an unsigned short[] of pixels (12 bit depth).
My java application gets the image by a CORBA call to this server.
Since java does not have ushort, the pixels are stored as (signed) short[].
This is the code I'm using to obtain a BufferedImage from the array:
private WritableImage loadImage(short[] pixels, int width, int height) {
int[] intPixels = new int[pixels.length];
for (int i = 0; i < pixels.length; i++) {
intPixels[i] = (int) pixels[i];
}
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
WritableRaster raster = (WritableRaster) image.getData();
raster.setPixels(0, 0, width, height, intPixels);
return SwingFXUtils.toFXImage(image, null);
}
And later:
WritableImage orgImage = convertShortArrayToImage2(image.data, image.size_x, image.size_y);
//load it into the widget
Platform.runLater(() -> {
imgViewer.setImage(orgImage);
});
I've checked that width=1280 and height=1024 and the pixels array is 1280x1024, that matches with the raster height and width.
However I'm getting an array out of bounds error in the line:
raster.setPixels(0, 0, width, height, intPixels);
I have try with ALL ImageTypes , and all of them produce the same error except for:
TYPE_USHORT_GRAY: Which I thought it would be the one, but shows an all-black image
TYPE_BYTE_GRAY: which show the image in negative(!) and with a lot of grain(?)
TYPE_BYTE_INDEXED: which likes the above what colorized in a funny way
I also have tried shifting bits when converting from shot to int, without any difference:
intPixels[i] = (int) pixels[i] & 0xffff;
So..I'm quite frustrated after looking for days a solution in the internet. Any help is very welcome
Edit. The following is an example of the images received, converted to jpg on the server side. Not sure if it is useful since I think it is made from has pixel rescaling (sqrt) :

Well, finally I solved it.
Probably not the best solution but it works and could help someone in ether....
Being the image grayscale 12 bit depth, I used BufferedImage of type TYPE_BYTE_GRAY, but I had to downsample to 8 bit scaling the array of pixels. from 0-4095 to 0-255.
I had an issue establishing the higher and lower limits of the scale. I tested with avg of the n higher/lower limits, which worked reasonably well, until someone sent me a link to a java program translating the zscale algorithm (used in DS9 tool for example) for getting the limits of the range of greyscale vlues to be displayed:
find it here
from that point I modified the previous code and it worked like a charm:
//https://github.com/Caltech-IPAC/firefly/blob/dev/src/firefly/java/edu/caltech/ipac/visualize/plot/Zscale.java
Zscale.ZscaleRetval retval = Zscale.cdl_zscale(pixels, width, height,
bitsVal, contrastVal, opt_sizeVal, len_stdlineVal, blankValueVal);
double Z1 = retval.getZ1();
double Z2 = retval.getZ2();
try {
int[] ints = new int[pixels.length];
for (int i = 0; i < pixels.length; i++) {
if (pixels[i] < Z1) {
pixels[i] = (short) Z1;
} else if (pixels[i] > Z2) {
pixels[i] = (short) Z2;
}
ints[i] = ((int) ((pixels[i] - Z1) * 255 / (Z2 - Z1)));
}
BufferedImage bImg
= new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
bImg.getRaster().setPixels(0, 0, width, height, ints);
return SwingFXUtils.toFXImage(bImg, null);
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
return null;

Related

Update part of a JavaFx WritableImage with a AWT BufferedImage

I'm trying to efficiently update a large WritableImage with multiple smaller tiles which are received as AWT BufferedImage instances.
Things I've tried:
I can use SwingFXUtils to convert the BufferedImage to a WritableImage, however there is no simple API on Writable image to take the pixels from another WritableImage that I can figure out.
I can get the pixels out of BufferedImage as an int [] using getData().getPixels(....).
However WritableImage expects data to be tightly packed in a single integers i.e. {ARGB, ARGB, ...} whereas BufferedImage.getData.getPixels returns data as series of integers, i.e. {R, G, B, A, R, G, B, A, ...).
I could obviously coerce the data into the correct format, but this doesn't feel very efficient.
I've after an approach / pointers on which API to use, I'm happy coding myself.
There is actually a version of PixelWriter.setPixels which takes another PixelReader as input.
private void update(Image input, WritableImage output, int x, int y) {
int width = input.getWidth(null);
int height = input.getHeight(null);
WritableImage temp = new WritableImage(width, height); // same size as input
SwingFXUtils.toFXImage(fragment, temp);
output.getPixelWriter().setPixels(x, y, width, height, temp.getPixelReader(), 0, 0);
}

using glDrawPixels to render bitmap raw data

I receive raw image data from server. The server uses MS Dib() function which returns in BGR format. Now, what i want to do is to read this raw data and use glDrawPixels to draw it in Linux.
I was advised that GetClrTabAddress function in MS and alike shall be used to get me the RGB values for each index of 800 by 600 image sent to me.
I do not know how to get these values using indices. Could anyone give some tips.
void func(QByteArray)
{
window_width = 800;
window_height = 600;
size = window_width * window_height;
pixels = new float[size*3];
memcpy(pixels, bytes, bytes.size());
}
void GlWidget::paintGL()
{
//! [5]
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glDrawPixels(window_width,window_height,GL_RGB,GL_FLOAT,pixels);
}
You can use GL_BGR in glDrawPixels, which will do the conversion for you and will probably be faster since AFAIK the GPU will do the work.
QByteArray sounds like you should be using unsigned bytes/chars instead of floats, which means GL_UNSIGNED_BYTE.
I'd assert(size*3*sizeof(float) == bytes.size());.
In this case make sure to set glPixelStorei(GL_UNPACK_ALIGNMENT, 1) if your width doesn't align to the default 4-byte boundry. With GL_BGR very pixel is 3 bytes and by default each row of your pixels is assumed to be padded to the next 4-byte boundary.
[EDIT]
OK, it looks like the image uses a palette. This means every value inthe QByteArray maps to an rgb value in another array. I'm not 100% sure where the palette is and maybe it can be computed implicitly, but you mentioned GetClrTabAddress which sounds promising.
The code will then look something like this
for(int i = 0; i < size; ++i)
{
unsigned char index = btmp[i];
//and something like..
memcpy(bytes + i * 3, GetClrTabAddress() + index * 3, 3);
//or
bytes[i*3+0] = someOtherPaletteData[index].red;
bytes[i*3+1] = someOtherPaletteData[index].green;
bytes[i*3+2] = someOtherPaletteData[index].blue;
}

Efficient conversion of AVFrame to QImage

I need to extract frames from a video in my Qt based application. Using ffmpeg libraries I am able to fetch frames as AVFrames which I need to convert to QImage to use in other parts of my application. This conversion needs to be efficient. So far it seems sws_scale() is the right function to use but I am not sure what source and destination pixel formats are to be specified.
Came up with the following 2-step process that first converts a decoded AVFame to another AVFrame in RGB colorspace and then to QImage. It works and is reasonably fast.
src_frame = get_decoded_frame();
AVFrame *pFrameRGB = avcodec_alloc_frame(); // intermediate pframe
if(pFrameRGB==NULL) {
;// Handle error
}
int numBytes= avpicture_get_size(PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
uint8_t *buffer = (uint8_t*)malloc(numBytes);
avpicture_fill((AVPicture*)pFrameRGB, buffer, PIX_FMT_RGB24,
is->video_st->codec->width, is->video_st->codec->height);
int dst_fmt = PIX_FMT_RGB24;
int dst_w = is->video_st->codec->width;
int dst_h = is->video_st->codec->height;
// TODO: cache following conversion context for speedup,
// and recalculate only on dimension changes
SwsContext *img_convert_ctx_temp;
img_convert_ctx_temp = sws_getContext(
is->video_st->codec->width, is->video_st->codec->height,
is->video_st->codec->pix_fmt,
dst_w, dst_h, (PixelFormat)dst_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGB32);
sws_scale(img_convert_ctx_temp,
src_frame->data, src_frame->linesize, 0, is->video_st->codec->height,
pFrameRGB->data,
pFrameRGB->linesize);
uint8_t *src = (uint8_t *)(pFrameRGB->data[0]);
for (int y = 0; y < dst_h; y++)
{
QRgb *scanLine = (QRgb *) myImage->scanLine(y);
for (int x = 0; x < dst_w; x=x+1)
{
scanLine[x] = qRgb(src[3*x], src[3*x+1], src[3*x+2]);
}
src += pFrameRGB->linesize[0];
}
If you find a more efficient approach, let me know in the comments
I know, it's too late, but maybe someone will find it useful. From here I got the clue of doing the same conversion, which looks a bit shorter.
So I created QImage which is reused for every decoded frame:
QImage img( width, height, QImage::Format_RGB888 );
Created frameRGB:
frameRGB = av_frame_alloc();
//Allocate memory for the pixels of a picture and setup the AVPicture fields for it.
avpicture_alloc( ( AVPicture *) frameRGB, AV_PIX_FMT_RGB24, width, height);
After the the first frame is decoded I create conversion context SwsContext this way (it will be used for all the next frames):
mImgConvertCtx = sws_getContext( codecContext->width, codecContext->height, codecContext->pix_fmt, width, height, AV_PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);
And finally for every decoded frame conversion is performed:
if( 1 == framesFinished && nullptr != imgConvertCtx )
{
//conversion frame to frameRGB
sws_scale(imgConvertCtx, frame->data, frame->linesize, 0, codecContext->height, frameRGB->data, frameRGB->linesize);
//setting QImage from frameRGB
for( int y = 0; y < height; ++y )
memcpy( img.scanLine(y), frameRGB->data[0]+y * frameRGB->linesize[0], mWidth * 3 );
}
See the link for the specifics.
A simpler approach, I think:
void takeSnapshot(AVCodecContext* dec_ctx, AVFrame* frame)
{
SwsContext* img_convert_ctx;
img_convert_ctx = sws_getContext(dec_ctx->width,
dec_ctx->height,
dec_ctx->pix_fmt,
dec_ctx->width,
dec_ctx->height,
AV_PIX_FMT_RGB24,
SWS_BICUBIC, NULL, NULL, NULL);
AVFrame* frameRGB = av_frame_alloc();
avpicture_alloc((AVPicture*)frameRGB,
AV_PIX_FMT_RGB24,
dec_ctx->width,
dec_ctx->height);
sws_scale(img_convert_ctx,
frame->data,
frame->linesize, 0,
dec_ctx->height,
frameRGB->data,
frameRGB->linesize);
QImage image(frameRGB->data[0],
dec_ctx->width,
dec_ctx->height,
frameRGB->linesize[0],
QImage::Format_RGB888);
image.save("capture.png");
}
Today, I have tested directly pass the image->bit() to swscale and finally it works, so it doesn't need to copy to memory. For example:
/* 1. Get frame and QImage to show */
struct my_frame *frame = get_frame(source);
QImage *myImage = new QImage(dst_w, dst_h, QImage::Format_RGBA8888);
/* 2. Convert and write into image buffer */
uint8_t *dst[] = {myImage->bits()};
int linesizes[4];
av_image_fill_linesizes(linesizes, AV_PIX_FMT_RGBA, frame->width);
sws_scale(myswscontext, frame->data, (const int*)frame->linesize,
0, frame->height, dst, linesizes);
I just discovered that scanLine is just seeking thru the buffer.. all you need is use AV_PIX_FMT_RGB32 for the AVFrame and QImage::FORMAT_RGB32 for the QImage.
Then after decoding just do a memcpy
memcpy(img.scanLine(0), pFrameRGB->data[0], pFrameRGB->linesize[0] * pFrameRGB->height());
I had problems with the other proposed solutions as :
They did not mention freeing either AVFrame, SwsContext or the allocated buffers, which caused massive memory leaks (I had thousands of frames to handle). These problems couldn't all be solved easily as QImage relies on the underlying data, and does not copy it. If freeing the buffer directly, the QImage points to freed data and breaks. This could be solved by using QImage's cleanupFunction to free the buffer once the image is no longer needed, but with other problems it wasn't good anyways.
In some cases one of the suggestions, of passing QImage.bits directly to sws_scale, would not work as QImage are minimum 32 bit aligned. Therefore for certain dimensions it would not match the expected width by sws_scale and output each line shifted a little bit.
A third problem is that they used deprecated AVPicture elements.
I listed the problem in another question Converting an AVFrame to QImage with conversion of pixel format and in the end found a solution using a temporary buffer, which could be copied to the QImage, and then safely freed.
So see my answer for a fully working, efficient, and with no deprecated function calls, implementation : https://stackoverflow.com/a/68212609/7360943

Graphics.MeasureCharacterRanges giving wrong size calculations

I'm trying to render some text into a specific part of an image in a Web Forms app. The text will be user entered, so I want to vary the font size to make sure it fits within the bounding box.
I have code that was doing this fine on my proof-of-concept implementation, but I'm now trying it against the assets from the designer, which are larger, and I'm getting some odd results.
I'm running the size calculation as follows:
StringFormat fmt = new StringFormat();
fmt.Alignment = StringAlignment.Center;
fmt.LineAlignment = StringAlignment.Near;
fmt.FormatFlags = StringFormatFlags.NoClip;
fmt.Trimming = StringTrimming.None;
int size = __startingSize;
Font font = __fonts.GetFontBySize(size);
while (GetStringBounds(text, font, fmt).IsLargerThan(__textBoundingBox))
{
context.Trace.Write("MyHandler.ProcessRequest",
"Decrementing font size to " + size + ", as size is "
+ GetStringBounds(text, font, fmt).Size()
+ " and limit is " + __textBoundingBox.Size());
size--;
if (size < __minimumSize)
{
break;
}
font = __fonts.GetFontBySize(size);
}
context.Trace.Write("MyHandler.ProcessRequest", "Writing " + text + " in "
+ font.FontFamily.Name + " at " + font.SizeInPoints + "pt, size is "
+ GetStringBounds(text, font, fmt).Size()
+ " and limit is " + __textBoundingBox.Size());
I then use the following line to render the text onto an image I'm pulling from the filesystem:
g.DrawString(text, font, __brush, __textBoundingBox, fmt);
where:
__fonts is a PrivateFontCollection,
PrivateFontCollection.GetFontBySize is an extension method that returns a FontFamily
RectangleF __textBoundingBox = new RectangleF(150, 110, 212, 64);
int __minimumSize = 8;
int __startingSize = 48;
Brush __brush = Brushes.White;
int size starts out at 48 and decrements within that loop
Graphics g has SmoothingMode.AntiAlias and TextRenderingHint.AntiAlias set
context is a System.Web.HttpContext (this is an excerpt from the ProcessRequest method of an IHttpHandler)
The other methods are:
private static RectangleF GetStringBounds(string text, Font font,
StringFormat fmt)
{
CharacterRange[] range = { new CharacterRange(0, text.Length) };
StringFormat myFormat = fmt.Clone() as StringFormat;
myFormat.SetMeasurableCharacterRanges(range);
using (Graphics g = Graphics.FromImage(new Bitmap(
(int) __textBoundingBox.Width - 1,
(int) __textBoundingBox.Height - 1)))
{
g.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.AntiAlias;
g.TextRenderingHint = System.Drawing.Text.TextRenderingHint.AntiAlias;
Region[] regions = g.MeasureCharacterRanges(text, font,
__textBoundingBox, myFormat);
return regions[0].GetBounds(g);
}
}
public static string Size(this RectangleF rect)
{
return rect.Width + "×" + rect.Height;
}
public static bool IsLargerThan(this RectangleF a, RectangleF b)
{
return (a.Width > b.Width) || (a.Height > b.Height);
}
Now I have two problems.
Firstly, the text sometimes insists on wrapping by inserting a line-break within a word, when it should just fail to fit and cause the while loop to decrement again. I can't see why it is that Graphics.MeasureCharacterRanges thinks that this fits within the box when it shouldn't be word-wrapping within a word. This behaviour is exhibited irrespective of the character set used (I get it in Latin alphabet words, as well as other parts of the Unicode range, like Cyrillic, Greek, Georgian and Armenian). Is there some setting I should be using to force Graphics.MeasureCharacterRanges only to be word-wrapping at whitespace characters (or hyphens)? This first problem is the same as post 2499067.
Secondly, in scaling up to the new image and font size, Graphics.MeasureCharacterRanges is giving me heights that are wildly off. The RectangleF I am drawing within corresponds to a visually apparent area of the image, so I can easily see when the text is being decremented more than is necessary. Yet when I pass it some text, the GetBounds call is giving me a height that is almost double what it's actually taking.
Using trial and error to set the __minimumSize to force an exit from the while loop, I can see that 24pt text fits within the bounding box, yet Graphics.MeasureCharacterRanges is reporting that the height of that text, once rendered to the image, is 122px (when the bounding box is 64px tall and it fits within that box). Indeed, without forcing the matter, the while loop iterates to 18pt, at which point Graphics.MeasureCharacterRanges returns a value that fits.
The trace log excerpt is as follows:
Decrementing font size to 24, as size is 193×122 and limit is 212×64
Decrementing font size to 23, as size is 191×117 and limit is 212×64
Decrementing font size to 22, as size is 200×75 and limit is 212×64
Decrementing font size to 21, as size is 192×71 and limit is 212×64
Decrementing font size to 20, as size is 198×68 and limit is 212×64
Decrementing font size to 19, as size is 185×65 and limit is 212×64
Writing VENNEGOOR of HESSELINK in DIN-Black at 18pt, size is 178×61 and limit is 212×64
So why is Graphics.MeasureCharacterRanges giving me a wrong result? I could understand it being, say, the line height of the font if the loop stopped around 21pt (which would visually fit, if I screenshot the results and measure it in Paint.Net), but it's going far further than it should be doing because, frankly, it's returning the wrong damn results.
I have a similar problem. I want to know how big the text I'm drawing is going to be, and where it's going to appear, EXACTLY. I haven't had the line-break problem, so I don't think I can help you there. I had the same problems you had with all the various measuring techniques available, including ending up with MeasureCharacterRanges, which worked okay for the left and right, but not at all for the height and top. (Playing with the baseline can work well for some rare applications though.)
I've ended up with a very inelegant, inefficient, but working solution, at least for my use case. I draw the text on a bitmap, check the bits to see where they ended up, and that's my range. Since I'm mostly drawing small fonts and short strings, it's been fast enough for me (especially with the memoization I added). Maybe this won't be exactly what you need, but maybe it can lead you down the right track anyway.
Note it requires compiling the project to allow unsafe code at the moment, as I'm trying to squeeze out every bit of efficiency from it, but that constraint could be removed if you wanted to. Also, it's not as thread safe as it could be right now, you could easily add that if you needed it.
Dictionary<Tuple<string, Font, Brush>, Rectangle> cachedTextBounds = new Dictionary<Tuple<string, Font, Brush>, Rectangle>();
/// <summary>
/// Determines bounds of some text by actually drawing the text to a bitmap and
/// reading the bits to see where it ended up. Bounds assume you draw at 0, 0. If
/// drawing elsewhere, you can easily offset the resulting rectangle appropriately.
/// </summary>
/// <param name="text">The text to be drawn</param>
/// <param name="font">The font to use when drawing the text</param>
/// <param name="brush">The brush to be used when drawing the text</param>
/// <returns>The bounding rectangle of the rendered text</returns>
private unsafe Rectangle RenderedTextBounds(string text, Font font, Brush brush) {
// First check memoization
Tuple<string, Font, Brush> t = new Tuple<string, Font, Brush>(text, font, brush);
try {
return cachedTextBounds[t];
}
catch(KeyNotFoundException) {
// not cached
}
// Draw the string on a bitmap
Rectangle bounds = new Rectangle();
Size approxSize = TextRenderer.MeasureText(text, font);
using(Bitmap bitmap = new Bitmap((int)(approxSize.Width*1.5), (int)(approxSize.Height*1.5))) {
using(Graphics g = Graphics.FromImage(bitmap))
g.DrawString(text, font, brush, 0, 0);
// Unsafe LockBits code takes a bit over 10% of time compared to safe GetPixel code
BitmapData bd = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
byte* row = (byte*)bd.Scan0;
// Find left, looking for first bit that has a non-zero alpha channel, so it's not clear
for(int x = 0; x < bitmap.Width; x++)
for(int y = 0; y < bitmap.Height; y++)
if(((byte*)bd.Scan0)[y*bd.Stride + 4*x + 3] != 0) {
bounds.X = x;
goto foundX;
}
foundX:
// Right
for(int x = bitmap.Width - 1; x >= 0; x--)
for(int y = 0; y < bitmap.Height; y++)
if(((byte*)bd.Scan0)[y*bd.Stride + 4*x + 3] != 0) {
bounds.Width = x - bounds.X + 1;
goto foundWidth;
}
foundWidth:
// Top
for(int y = 0; y < bitmap.Height; y++)
for(int x = 0; x < bitmap.Width; x++)
if(((byte*)bd.Scan0)[y*bd.Stride + 4*x + 3] != 0) {
bounds.Y = y;
goto foundY;
}
foundY:
// Bottom
for(int y = bitmap.Height - 1; y >= 0; y--)
for(int x = 0; x < bitmap.Width; x++)
if(((byte*)bd.Scan0)[y*bd.Stride + 4*x + 3] != 0) {
bounds.Height = y - bounds.Y + 1;
goto foundHeight;
}
foundHeight:
bitmap.UnlockBits(bd);
}
cachedTextBounds[t] = bounds;
return bounds;
}
Ok so 4 years late but this question EXACTLY matched my symptoms and I've actually worked out the cause.
There is most certainly a bug in MeasureString AND MeasureCharacterRanges.
The simple answer is:
Make sure you divide your width restriction (int width in MeasureString or the Size.Width property of the boundingRect in MeasureCharacterRanges) by 0.72. When you get your results back multiply each dimension by 0.72 to get the REAL result
int measureWidth = Convert.ToInt32((float)width/0.72);
SizeF measureSize = gfx.MeasureString(text, font, measureWidth, format);
float actualHeight = measureSize.Height * (float)0.72;
or
float measureWidth = width/0.72;
Region[] regions = gfx.MeasureCharacterRanges(text, font, new RectangleF(0,0,measureWidth, format);
float actualHeight = 0;
if(regions.Length>0)
{
actualHeight = regions[0].GetBounds(gfx).Size.Height * (float)0.72;
}
The explanation (that I can figure out) is that something to do with the context is triggering a conversion in the Measure methods (that doesn't trigger in the DrawString method) for inch->point (*72/100). When you pass in the ACTUAL width limitation it is adjusting this value so the MEASURED width limitation is, in effect, shorter than it should be. Your text then wraps earlier than it is supposed to and so you get a longer height result than expected. Unfortunately the conversion applies to the actual height result as well so it's a good idea to 'unconvert' that value too.
Could you try removing the following line?
fmt.FormatFlags = StringFormatFlags.NoClip;
Overhanging parts of glyphs, and
unwrapped text reaching outside the
formatting rectangle are allowed to
show. By default all text and glyph
parts reaching outside the formatting
rectangle are clipped.
That's the best I can come up with for this :(
I also had some problems with the MeasureCharacterRanges method. It was giving me inconsistent sizes for the same string and even the same Graphics object. Then I discovered that it depends on the value of the layoutRect parametr - I can't see why, in my opinion it's a bug in the .NET code.
For example if layoutRect was completely empty (all values set to zero), I got correct values for the string "a" - the size was {Width=8.898438, Height=18.10938} using 12pt Ms Sans Serif font.
However, when I set the value of the 'X' property of the rectangle to a non-integer number (like 1.2), it gave me {Width=9, Height=19}.
So I really think there is a bug when you use a layout rectangle with non-integer X coordinate.
To convert from points to dpi as in screen resolution you need to divide by 72 and multiply by DPI, for example:
graphics.DpiY * text.Width / 72
Red Nightengale was really close, because graphics.DpiY is usually 96 for screen resolutions.

gdi+ Graphics::DrawImage really slow~~

I am using a GDI+ Graphic to draw a 4000*3000 image to screen, but it is really slow. It takes about 300ms. I wish it just occupy less than 10ms.
Bitmap *bitmap = Bitmap::FromFile("XXXX",...);
//--------------------------------------------
// this part takes about 300ms, terrible!
int width = bitmap->GetWidth();
int height = bitmap->GetHeight();
DrawImage(bitmap,0,0,width,height);
//------------------------------------------
I cannot use CachedBitmap, because I want to edit the bitmap later.
How can I improve it? Or is any thing wrong?
This native GDI function also draws the image into the screen, and it just take 1 ms:
SetStretchBltMode(hDC, COLORONCOLOR);
StretchDIBits(hDC, rcDest.left, rcDest.top,
rcDest.right-rcDest.left, rcDest.bottom-rcDest.top,
0, 0, width, height,
BYTE* dib, dibinfo, DIB_RGB_COLORS, SRCCOPY);
//--------------------------------------------------------------
If I want to use StretchDIBits, I need to pass BITMAPINFO, But how can I get BITMAPINFO from a Gdi+ Bitmap Object? I did the experiment by FreeImage lib, I call StretchDIBits using FreeImageplus object, it draw really fast. But now I need to draw Bitmap, and write some algorithm on Bitmap's bits array, how can I get BITMAPINFO if I have an Bitmap object? It's really annoying -___________-|
If you're using GDI+, the TextureBrush class is what you need for rendering images fast. I've written a couple of 2d games with it, getting around 30 FPS or so.
I've never written .NET code in C++, so here's a C#-ish example:
Bitmap bmp = new Bitmap(...)
TextureBrush myBrush = new TextureBrush(bmp)
private void Paint(object sender, PaintEventArgs e):
{
//Don't draw the bitmap directly.
//Only draw TextureBrush inside the Paint event.
e.Graphics.FillRectangle(myBrush, ...)
}
You have a screen of 4000 x 3000 resolution? Wow!
If not, you should draw only the visible part of the image, it would be much faster...
[EDIT after first comment] My remark is indeed a bit stupid, I suppose DrawImage will mask/skip unneeded pixels.
After your edit (showing StretchDIBits), I guess a possible source of speed difference might come from the fact that StretchDIBits is hardware accelerated ("If the driver cannot support the JPEG or PNG file image" is a hint...) while DrawImage might be (I have no proof for that!) coded in C, relying on CPU power instead of GPU's one...
If I recall correctly, DIB images are fast (despite being "device independent"). See High Speed Win32 Animation: "use CreateDIBSection to do high speed animation". OK, it applies to DIB vs. GDI, in old Windows version (1996!) but I think it is still true.
[EDIT] Maybe Bitmap::GetHBITMAP function might help you to use StretchDIBits (not tested...).
Just a thought; instead of retrieving the width and height of the image before drawing, why not cache these values when you load the image?
Explore the impact of explicitly setting the interpolation mode to NearestNeighbor (where, in your example, it looks like interpolation is not actually needed! But 300ms is the kind of cost of doing high-quality interpolation when no interpolation is needed, so its worth a try)
Another thing to explore is changing the colour depth of the bitmap.
Unfortunately when I had a similar problem, I found that GDI+ is known to be much slower than GDI and not generally hardware accelerated, but now Microsoft have moved on to WPF they will not come back to improve GDI+!
All the graphics card manufacturers have moved onto 3D performance and don't seem interested in 2D acceleration, and there's no clear source of information on which functions are or can be hardware accelerated or not. Very frustrating because having written an app in .NET using GDI+, I am not happy to change to a completely different technology to speed it up to reasonable levels.
i don't think they'll make much of a different, but since you're not actually needing to resize the image, try using the overload of DrawImage that doesn't (attempt) to resize:
DrawImage(bitmap,0,0);
Like i said, i doubt it will make any difference, because i'm sure that DrawImage checks the Width and Height of the bitmap, and if there's no resizing needed, just calls this overload. (i would hope it doesn't bother going through all 12 million pixels performing no actual work).
Update: My ponderings are wrong. i had since found out, but guys comment reminded me of my old answer: you want to specify the destination size; even though it matches the source size:
DrawImage(bitmap, 0, 0, bitmap.GetWidth, bitmap.GetHeight);
The reason is because of dpi differences between the dpi of bitmap and the dpi of the destination. GDI+ will perform scaling to get the image to come out the right "size" (i.e. in inches)
What i've learned on my own since last October is that you really want to draw a "cached" version of your bitmap. There is a CachedBitmap class in GDI+. There are some tricks to using it. But in there end i have a function bit of (Delphi) code that does it.
The caveat is that the CachedBitmap can become invalid - meaning it can't be used to draw. This happens if the user changes resolutions or color depths (e.g. Remote Desktop). In that case the DrawImage will fail, and you have to re-created the CachedBitmap:
class procedure TGDIPlusHelper.DrawCachedBitmap(image: TGPImage;
var cachedBitmap: TGPCachedBitmap;
Graphics: TGPGraphics; x, y: Integer; width, height: Integer);
var
b: TGPBitmap;
begin
if (image = nil) then
begin
//i've chosen to not throw exceptions during paint code - it gets very nasty
Exit;
end;
if (graphics = nil) then
begin
//i've chosen to not throw exceptions during paint code - it gets very nasty
Exit;
end;
//Check if we have to invalidate the cached image because of size mismatch
//i.e. if the user has "zoomed" the UI
if (CachedBitmap <> nil) then
begin
if (CachedBitmap.BitmapWidth <> width) or (CachedBitmap.BitmapHeight <> height) then
FreeAndNil(CachedBitmap); //nil'ing it will force it to be re-created down below
end;
//Check if we need to create the "cached" version of the bitmap
if CachedBitmap = nil then
begin
b := TGDIPlusHelper.ResizeImage(image, width, height);
try
CachedBitmap := TGPCachedBitmap.Create(b, graphics);
finally
b.Free;
end;
end;
if (graphics.DrawCachedBitmap(cachedBitmap, x, y) <> Ok) then
begin
//The calls to DrawCachedBitmap failed
//The API is telling us we have to recreate the cached bitmap
FreeAndNil(cachedBitmap);
b := TGDIPlusHelper.ResizeImage(image, width, height);
try
CachedBitmap := TGPCachedBitmap.Create(b, graphics);
finally
b.Free;
end;
graphics.DrawCachedBitmap(cachedBitmap, x, y);
end;
end;
The cachedBitmap is passed in by reference. The first call to DrawCachedBitmap it cached version will be created. You then pass it in subsequent calls, e.g.:
Image imgPrintInvoice = new Image.FromFile("printer.png");
CachedBitmap imgPrintInvoiceCached = null;
...
int glyphSize = 16 * (GetCurrentDpi() / 96);
DrawCachedBitmap(imgPrintInvoice , ref imgPrintInvoiceCached , graphics,
0, 0, glyphSize, glyphSize);
i use the routine to draw glyphs on buttons, taking into account the current DPI. The same could have been used by the Internet Explorer team to draw images when the user is running high dpi (ie is very slow drawing zoomed images, because they use GDI+).
/*
First sorry for ma English, and the code is partly in polish, but it's simple to understand.
I had the same problem and I found the best solution. Here it is.
Dont use: Graphics graphics(hdc); graphics.DrawImage(gpBitmap, 0, 0); It is slow.
Use: GetHBITMAP(Gdiplus::Color(), &g_hBitmap) for HBITMAP and draw using my function ShowBitmapStretch().
You can resize it and it is much faster! Artur Czekalski / Poland
*/
//--------Global-----------
Bitmap *g_pGDIBitmap; //for loading picture
int gRozXOkna, gRozYOkna; //size of working window
int gRozXObrazu, gRozYObrazu; //Size of picture X,Y
HBITMAP g_hBitmap = NULL; //for displaying on window
//------------------------------------------------------------------------------
int ShowBitmapStretch(HDC hdc, HBITMAP hBmp, int RozX, int RozY, int RozXSkal, int RozYSkal, int PozX, int PozY)
{
if (hBmp == NULL) return -1;
HDC hdc_mem = CreateCompatibleDC(hdc); //utworzenie kontekstu pamięciowego
if (NULL == hdc_mem) return -2;
//Trzeba połączyć BMP z hdc_mem, tzn. umieścić bitmapę w naszym kontekście pamięciowym
if (DeleteObject(SelectObject(hdc_mem, hBmp)) == NULL) return -3;
SetStretchBltMode(hdc, COLORONCOLOR); //important! for smoothness
if (StretchBlt(hdc, PozX, PozY, RozXSkal, RozYSkal, hdc_mem, 0, 0, RozX, RozY, SRCCOPY) == 0) return -4;
if (DeleteDC(hdc_mem) == 0) return -5;
return 0; //OK
}
//---------------------------------------------------------------------------
void ClearBitmaps(void)
{
if (g_hBitmap) { DeleteObject(g_hBitmap); g_hBitmap = NULL; }
if (g_pGDIBitmap) { delete g_pGDIBitmap; g_pGDIBitmap = NULL; }
}
//---------------------------------------------------------------------------
void MyOpenFile(HWND hWnd, szFileName)
{
ClearBitmaps(); //Important!
g_pGDIBitmap = new Bitmap(szFileName); //load a picture from file
if (g_pGDIBitmap == 0) return;
//---Checking if picture was loaded
gRozXObrazu = g_pGDIBitmap->GetWidth();
gRozYObrazu = g_pGDIBitmap->GetHeight();
if (gRozXObrazu == 0 || gRozYObrazu == 0) return;
//---Uworzenie bitmapy do wyświatlaia; DO IT ONCE HERE!
g_pGDIBitmap->GetHBITMAP(Gdiplus::Color(), &g_hBitmap); //creates a GDI bitmap from this Bitmap object
if (g_hBitmap == 0) return;
//---We need to force the window to redraw itself
InvalidateRect(hWnd, NULL, TRUE);
UpdateWindow(hWnd);
}
//---------------------------------------------------------------------------
void MyOnPaint(HDC hdc, HWND hWnd) //in case WM_PAINT; DO IT MANY TIMES
{
if (g_hBitmap)
{
double SkalaX = 1.0, SkalaY = 1.0; //scale
if (gRozXObrazu > gRozXOkna || gRozYObrazu > gRozYOkna || //too big picture, więc zmniejsz;
(gRozXObrazu < gRozXOkna && gRozYObrazu < gRozYOkna)) //too small picture, można powiększyć
{
SkalaX = (double)gRozXOkna / (double)gRozXObrazu; //np. 0.7 dla zmniejszania; FOR DECREASE
SkalaY = (double)gRozYOkna / (double)gRozYObrazu; //np. 1.7 dla powiększania; FOR INCREASE
if (SkalaY < SkalaX) SkalaX = SkalaY; //ZAWSZE wybierz większe skalowanie, czyli mniejszą wartość i utaw w SkalaX
}
if (ShowBitmapStretch(hdc, g_hBitmap, gRozXObrazu, gRozYObrazu, (int)(gRozXObrazu*SkalaX), (int)(gRozYObrazu*SkalaX), 0, 0, msg) < 0) return;
Try using copy of Bitmap from file. FromFile function on some files returns "slow" image, but its copy will draw faster.
Bitmap *bitmap = Bitmap::FromFile("XXXX",...);
Bitmap *bitmap2 = new Bitmap(bitmap); // make copy
DrawImage(bitmap2,0,0,width,height);
I have made some researching and wasn't able to find a way to render images with GDI/GDI+ more faster than
Graphics.DrawImage/DrawImageUnscaled
and at the same time simple like it.
Till I discovered
ImageList.Draw(GFX,Point,Index)
and yeah it's really so fast and simple.

Resources