DWM in Win7/8 + GDI - gdi

Question

A problem I've noticed once on my Win7 system, thought it was a DWM bug as it was fixed after a reboot. But now I'm realizing that it's happening (as default behavior) on other people's systems, and it's the normal behavior on Surface Pro as well.
How to reproduce the problem: implement, using GDI, a basic lasso system. Define a rectangle controlled by the mouse, when the rectangle changes, invalidate the old one & the new one (or invalidate a union of both rectangles, either as a new rectangle or a complex region, it doesn't matter, the "bug" still shows anyway).
During wm_paint, you just erase the background & paint the rectangle (it has to be a rectangle outline, the problem won't be visible if it's a filled rectangle). You can do double-buffering if you wanna be sure that it's not a flickering problem (& trust me it's not).
So what you'll see, if you have a system like mine (desktop Win7 with a geforce, aero on), is a normal lasso system, with no more ghosting than the monitor's own.
On other systems (like Surface Pro, to define a fully known system), you'll see, as you extend the lasso outwards, the border of the lasso disappearing. A bit like LCD ghosting, but massively more noticable.
Now, instead of invalidating the lasso's rectangle, try invalidating the whole window. And there, no more ghosting.
What I found out is that it's not the invalidation that "fixes" it, it's GDI access. You can as well invalidate the whole rectangle, but only paint the lasso's zone, still ghosting. But if you paint the lasso's zone and draw a single little pixel on each corners of the window, no more ghosting.
There must be something in DWM, probably since version 1.1, that uses some kind of caching of the bounding box of the last GDI access, and for some weird reason, what falls within the last bounding box will get immediately on screen, while the new part will be delayed by at least 1 frame.
This is pretty bad because it's breaking very basic window invalidation that everyone uses, and I haven't found any way to fix it (other than by invalidating the whole window of course, but that'd be stupid, & besides it's a problem that affects the whole GDI, so you get poor visual results everywhere).
Again it's most likely in DWM 1.1, I don't think you can get this in Vista, but I'm not sure. I also don't know why it doesn't do that on my desktop, possibly it depends on the graphic card's driver.
So if anyone happens to know more about this...

Answer 1

An update on this. I didn't find any clean solution, but a hack that works.
First, I also discovered that this "bug" seems to affect all systems, just not the same way.
Using DwmGetCompositionTimingInfo, once can make v-synced GDI animation. That's in theory, because in practice it won't work, because of the same "bug", even on systems not visibly affected by what I describe above. The DWM will simply decide not to refresh anything when there isn't enough to fresh, and that will cause frameskips when scrolling a window using ScrollWindowEx and not enough pixels are invalidated.
So what's the solution? Fool the DWM into thinking that it has to swap buffers immediately, as it's what the whole problem is about. When doing GDI operations, one might think that the result will appear at the DWM's next vblank-synced refresh, but it's not the case. Some parts will, some will be delayed. So the trick is to force the DWM to swap, here's what I found about it:
-first, DWMFlush does not do that (nor does GDIFlush). DWMFlush is more or less a WaitForVBlank anyway
-DWMSetPresentParameters doesn't seem to allow this either, even though I didn't bother that much with that function since it's already gone in Windows 8
-the DWM seems to refresh when there are enough pixels to swap, but it also seems to be partitionned, quite possibly into wide but short rectangles (that's what it appears to be on Surface Pro - but not on my desktop). Whether it's related to apertures to the VRAM or segmentation into small textures, I have no idea, really, maybe someone knows?
So what works for me: telling the GDI to refresh vertical stripes, 1 pixel wide, about 500 pixels apart, all over the screen. If you only do it on parts of the screen, you'll still get flicker on other parts.
It also works using horizontal stripes but then you'll need a lot more of them, that's why I believe that the segmentation is done in wide, short rectangles. You can kinda see those rectangles when tricking the DWM.
Which GDI functions work? Many do, but they don't all have the same CPU usage.
First, forget GetPixel, it works but the CPU usage is obviously extremely high.
Second, the GDI seems to be intelligent enough to detect things that can be buffered just as commands, so filling a rectangle with a null brush, or drawing empty text, won't force the DWM to refresh.
What does work is BitBlt or AlphaBlend using empty vertical stripes. The CPU usage is still ok, even though it's not so far from blitting a whole screen.
It has to be done on a top-level window, not the desktop.
Creating a Direct2D render target for the DC & doing a begin/enddraw also works, but it's normal since it will force a refresh of the full rectangle, & thus the CPU cost is higher.
If anyone knows a better way to force a DWM refresh, I'd like to know. But since Windows 8 already made obsolete most of the interesting DWM functions (fortunately DwmGetCompositionTimingInfo still partially works, or it would be impossible do vsynced timers without DirectX), I'm not sure there's any better way.
Also, it doesn't have to be done on all top-level windows. You can see the effect working on a Windows desktop blue lasso when it gets near a top-level window that invalidates stripes, it stops flickering as soon as it enters a zone near it (the segmentation I'm talking of above).

Answer 2

Here's a couple of general GDI-related pointers regarding your code:
When you call InvalidateRect API it may not immediately dispatch WM_PAINT notification, so calling InvalidateRect two times in a row like you did in your TForm1.FormMouseMove method is most certainly causing this visual effect. The first redrawing is not yet processed when you call it the second time.
I'm not sure what Canvas.Rectangle exactly does "on the inside" or which APIs it calls. I'm assuming it is using DrawFocusRect, in which case you have to be aware that its drawing is XORed, so doing it twice over the same rectangle will erase it.
OK, so here's what I'd do if I were drawing that selection box:
Call InvalidateRect API only once. For that you will have to calculate the bounding rectangle that will include position of the selection box before and after the move. If I'm not mistaking, you can use UnionRect API for that, or just calculate the bounding rect yourself.
If avoiding a visual lag is important, I'd do all the drawing of the selection box in TForm1.FormMouseMove. You can get a device context by calling GetDC API on your window handle. After that do the same drawing. In this case you will not need any invalidation. (Sorry, I can't give you a procedure for Delphi. This is how I'd do it with plain WinAPIs.)
EDIT: After reviewing your C++ code I was able to reproduce the visual flicker you're describing. I also made two C++ projects, derived from your original code in hopes of solving it: Here's a simple version and here's the one with mouse dragging support. None of them solved the issue though.
I was able to reproduce the flicker on a Windows 7 desktop. After having done some tests I am convinced now that this visual artifact is caused by a display driver. In my case the desktop has ATI Radeon video card. This is how I was able to conclude that: I ran the test executable on a different (older) desktop with Windows XP OS and the flicker was not present. I ran the exact same executable in a virtual machine with Windows XP installed in it on the desktop computer that was producing the flicker. The flicker was present when the executable ran in a Windows XP in that virtual machine. Since the same video driver is responsible for rendering the actual desktop and the one in the virtual machine, there's most definitely some "funny business" going on with optimization or caching in the video driver, that is causing this artifact.
So, the question now is how to resolve it. In my second build of your project, I added the code to render the whole client area for the window to eliminate the possibility of a calculation error, but that did not help. So at this point you're helpless to resolve it via plain APIs.
Your next steps should probably be these:
Contact the driver maker for the video card that you're experiencing this issue on. See if they help. I'd show them your YouTube video of the issue and give them the C++ project to reproduce it with.
Post a question on Windows Driver Development forums, preferably for video driver developers. Unfortunately it's a dwindling community, so expect long delays.
Change the tags on your question here. Add these: C, C++, WinAPI, GDI, DWM and remove what you have there now. This way it will be visible for a Win32 dev community so you'll get more views.
Other than that, good luck! This is such a minor bug (even if one can call it that) that I doubt that you'll get any serious attention to it. Although this is very admirable, in my book, that you're trying to achieve such perfection for your software.
Also, if you need me to second you on this, I can do so.