I have been looking far and wide to find out how, if it’s possible, you can fill a particular area in a QML screen with an OpenGL context and do custom OpenGL only in that context. I’ve seen plenty of demos where the QML components like buttons, etc lay on top or below a screen-wide OpenGL context (as is typically required by games), but I’d like to be able to situate several distinct OpenGL contexts within QML and have the QML file define how large they are, where they are positioned, etc.
Now, since Qt 5 is all OpenGL under the hood, it makes me wonder if using a Canvas element with custom drawing via javascript could result in similar rendering performance as custom OpenGL? This would be a meaningful alternative but it’s not clear to me how the javascript drawing is handled via runtime compared to custom OpenGL drawing.
What is it that you want to draw? QQuickPaintedItem may be the simplest way to go about it. When you're using QOpenGLFramebufferObject as the target, the painter will use OpenGL to paint the texture. It might be easier than writing your own OpenGL code if all you're doing is 2D.
I am working on certain project where the task is to paint bitmaps (currently HBITMAP/bitblt/alphablend) into non-client areas of all visible windows (i.e. windows which do not belong to my application). It must be done very fast - bitmap is update when window is moved, resized etc. Also some kind of blur algorithm must be applied on this bitmap. It should work on Win7 and Win8 (so XP is not required).
I have managed to work properly with GDI. I obtain GetWindowDC, GetWindowRect and AlphaBlend bitmap into buffer (CreateCompatibleDC/CreateCompatibleBitmap) and then BitBlt it into GetWindowDC. This works perfectly... except... it is not as fast as I want. And if I apply blur algorithm on the bitmap, then everything is slow as hell.
So my question would be how to improve the speed? I'm thinking about hardware accelerated drawing.
a) I tried GDI-compatible Direct2D (using ID2D1DCRenderTarget+BindDC) but it is much slower than pure GDI.
b) I am thinking about Direct3D. Problem is that I probably don't know how to use it. If I call D3D10CreateDeviceAndSwapChain with swapchain's OutputWindow set to HWND of my application then it returns S_OK but when I set OutputWindow to HWND of any foreign window then the method fails. So I am not sure how to render into foreign windows.
c) how to properly apply blur on a part of image? I found many algorithms but all of them are processed on CPU. How to make it on GPU?
Thanks in advance for any idea how to solve my problem.
Have you thought about using DComp? For why using DComp may be appropriate, take a look at this: http://msdn.microsoft.com/en-us/library/windows/desktop/hh449195%28v=vs.85%29.aspx
For a brief summary of what DComp is (from MSDN):
Microsoft DirectComposition is a Windows component that enables high-performance bitmap composition with transforms, effects, and animations. Application developers can use the DirectComposition API to create visually engaging user interfaces that feature rich and fluid animated transitions from one visual to another.
DirectComposition enables rich and fluid transitions by achieving a high framerate, using graphics hardware, and operating independently of the UI thread. DirectComposition can accept bitmap content drawn by different rendering libraries, including Microsoft DirectX bitmaps, and bitmaps rendered to a window (HWND bitmaps). Also, DirectComposition supports a variety of transformations, such as 2D affine transforms and 3D perspective transforms, as well as basic effects such as clipping and opacity.
DirectComposition is designed to simplify the process of composing visuals and creating animated transitions. If your application already contains rendering code or already uses the recommended DirectX API, you only need to do a minimal amount of work to use DirectComposition effectively.
I have enabled qt+OpenGl+SimpleGl on one of the ARM platform and was able to run opengl example programs.
I also has a qt+Webkit, which is working with a graphic plugin.
I wanted to use simpleGl context for every thing, instead of using the normal graphic screen. So, when I try to run Qt+Webkit with simpleGl, I just get a blank screen.
Does QT support this? If so how can we make it?
Yes, this is correct. OpenGL draws directly to the framebuffer. The simplegl driver doesn't handle what is drawn using the raster paint engine of the QWS, so you may see only black.
Using simplegl for "everything" means you want everything to be drawn using OpenGL in your EGL full-screen window? This is possible under some assumptions. You have to write all your applications to be rendered using the Qt OpenGL paint engine (using the opengl graphics system is not supported under Qt/E). This is possible also for QtWebKit, I'm doing it now. Note that this does not mean that everything is rendered using hardware acceleration. You'll have to write your applications "the right way" to get all actually hardware accelerated. Consider that you'll have to handle the mouse pointer some other way in this case.
The other way is to just modify the simplegl driver to allow for the use of Qt applications using the raster paint engine. This is possible as well with some limitations. Qt can use blit to place its own windows over OpenGL. Look for the framebuffer driver inside the Qt source tree to know how to do this. You can then have common Qt applications and OpenGL Qt applications some way. I'm doing this as well. Not everything can be done anyway.
EDIT: I'm sure you already did, but in case, give this http://doc.qt.io/qt-4.8/qt-embeddedlinux-opengl.html much attention.
Unfortunately I don't know anything about SimpleGL, but I do know that there is a way to render a standard Qt widget in a QGLWidget. Maybe have a look at this Qt Quarterly which I think is somewhat related to your question:
http://doc.qt.nokia.com/qq/qq26-openglcanvas.html
If anyone has a framebuffer implementation, could you please share some performance benchmarks? Or at least tell me how much difference are there in your point of view.
I need to make a flash game which has almost 1K 50x50 bitmaps moving on screen(more than 10K display object in display list), and want to know if flash can support it well on a normal PC with at least 30FPS.
You can have a look at the flixel library for a rasterize library and for a bitmap versus display list comparison have a look at this blog post
You should be able to do this pretty easily if you use BitmapData and you don't need to rotate your sprites. Here is an example of 10,000 3x3 pixel sprites - I have used this same technique on 7,000 50x50 pixel spites and it runs just fine:
http://actionsnippet.com/?p=224
I'm working on a standalone Flash application (written using Flex 3/ActionScript 3) that features a text crawl, like what you might see at the bottom of your TV when watching a cable news channel; it's a long narrow box that text moves across from right to left.
I've implemented it by creating a Label element, populating it with text, and then moving it using a mx:Move object with a Linear.easeNone easing function. It works, but it has ample room for improvement. It looks a bit jerky, and tends to have a fair amount of "tearing" (the top and bottom halves of the text sometimes fall out of sync).
I tried throwing math at the problem to get the crawl's movement rate synced with the monitor's refresh rate, but that was a bust. I found out the hard way that the app's frame rate jumps around too much; the "optimized" crawl varied between looking silky smooth and like it had epilepsy.
Is there anything else folks would recommend I try to smooth this thing out? Is there some alternate design you'd recommend I try?
Edit: Some context: the crawl is part of a digital signage application (played from a standalone Flash projector -- no web browser) that does stuff elsewhere on the screen, including video playback and rendering text and images. It definitely gets choppier during video playback, but it's never as smooth as I'd like it to be.
There are two potential solutions to this problem, but both have caveats, the first because of your use of Flex and a standalone projector, the second because it is a mitigator, not a complete solution.
Hardware Acceleration
When publishing your file, you can attempt to have Flash utilize hardware acceleration to alleviate the vertical refresh issue you are running into that is causing tearing. Sadly, Flex Builder 3 is incapable of enabling this setting at the SWF (projector) level (Link to bug). This has yet to be resolved and has been pushed from 4.0 to 4.1 to 4.x... If and when it is resolved, it will likely be a compiler argument in the project settings of Flash Builder 4.
You may be able to determine if this solution works for you by outputting your projector as a standard SWF and embedding it on an HTML document with the wmode set to "direct" or "gpu". Sadly, if it does (it should), you can't use it right now anyway. If you have Flash Builder 4, certain projects are capable of making round trips between FB4 and Flash Professional CS5, though I am not sure what the criteria for that is (my current AIR project has all the project modification menu options grayed out). If you do manage to get your project into Flash, you can enable hardware acceleration in the Publish Settings of the project (File->Publish Settings->Flash tab->Hardware Acceleration option in CS5).
This method is almost a certain solution for your problem, though it has two issues, one already highlighted above, and (for people publishing for the web) that by utilizing direct or GPU rendering on a webpage, you are unable to layer any DOM elements on top of flash.
direct: This mode tries to use the fastest path to screen, or direct path if you will. In most cases it will ignore whatever the browser would want to do to have things like overlapping HTML menus or such work. A typical use case for this mode is video playback. On Windows this mode is using DirectDraw or Direct3D on Vista, on OSX and Linux we are using OpenGL. Fidelity should not be affected when you use this mode.
gpu: This is fully fledged compositing (+some extras) using some functionality of the graphics card. Think of it being similar to what OSX and Vista do for their desktop managers, the content of windows (in flash language that means movie clips) is still rendered using software, but the result is composited using hardware. When possible we also scale video natively in the card. More and more parts of our software rasterizer might move to the GPU over the next few Flash Player versions, this is just a start. On Windows this mode uses Direct3D, on OSX and Linux we are using OpenGL.
**Source*
Direct is the ideal option for this situation, as you can actually have performance degredation with "gpu" as well as visual differences from graphics card to graphics card.
Lower your framerate
The Flash player will continue to play video at its native refresh rate independent of the rest of your project as long as you keep the framerate at or above approximately 2FPS (though I suggest 5FPS minimum). You won't want to run that low for this example, but you are able to lower the framerate of the entire scene without impacting video performance. The closer your framerate is to the screen refresh rate, the more apt you are to actually create the tearing effect unless you are able to absolutely sync with the monitor's refresh rate, which you probably cannot do without the above... Hardware Acceleration.
This problem has existed in the Flash Player for as long as it has been able to move objects horizontally. What happens is that Flash updates a buffered snapshot of the running animation at the same time that the screen is refreshing. If the buffered snapshot changes partway through a screen refresh, you get a tear. This is why lowering the framerate actually reduces the amount of tearing, you are refreshing the buffer less frequently.
As #Tegeril mentioned, using Flex is one of the reasons. Flex is a pretty heavy framework and it does a lot of things behind the scenes. If you're familiar with the life cycle of a component(especially invalidating properties, invalidating the display list, etc.).
As a few minor things that might improve performance:
try to keep a simple display list. If you know the app will always be displayed at one size, then flex won't waste time traversing the display list/tree up to the top and back for measurements. Also, try to use a Canvas. I know, it's not very clean, but since it uses absolute values and doesn't check with the 'parents' much, it should be faster than other containers(like HBox,VBox, etc.)
try to display the video at it's full size(make sure the encoded video dimensions are right so there be any CPU cycles on resizing video
Ok, this was Flex stuff.
It might be very handy to read sencular's article on Asynchronous ActionScript Execution which explains how Flash Player handles updates and renders.
(source: senocular.com)
Frames both execute ActionScript and render the screen
(source: senocular.com)
ActionScript taking a long time to complete delays rendering
I imagine the jerkiness is related to this. Also, I'm guessing you might
get moments of smooth movement then sudden halts, every now and then, when
Flash Player catches it's breath(Garbage Collector cleans up)
Victor Drâmbă article on “Multithreading” in Actionscript might also
be useful.
Soo, to recap:
use Profiler or something and see if the Flex framework is slowing you down, or where the 'bottleneck' is
improve as much as you can on that side then check if it's how Flash Player handles all the actionscript('elastic' frames)
If the bottleneck comes from the Flex framework, worst case, you
can try to minimise the number of components that traverse the display list,
and use pure actionscript for the other things(as #PatrickS suggested, use TweenLite, etc.)
If it helps, try to preload data(fetch rss feed and all that) at the start, and when you've got most of the important bits that don't require 'refreshes'/loads frequently, display the app. You will use more memory, but will have more cpu cycles to spare for other tasks.
Also, if it's display objects that are the 'bottleneck' and there's plenty of them, check if you can reuse them using Object Pools.
HTH
TweenMax or even TweenLite ( http://www.greensock.com )handles this sort of job pretty well. What else is your app doing while the text is scrolling though? Is it possible that some other processes are interfering?
This may not be helpful, but have you considered putting the crawling text into the html DOM and using CSS transitions to crawl the text. Obviously there's the IE problem, but it should be supported in IE9 and you could use javascript as a fallback.
This may seem silly, but CSS transitions are getting hardware acceleration and separate processes for plugins meaning on a multicore machine you could get parallel threads.
One thing you might consider is to move your label incrementally using a Timer instead of an easing function. That way you can take advantage of the updateAfterEvent method to get smoother rendering. Here's a link to an article/video from Chet Haase (Adobe's Flex graphics dude) that explains usage along with an example app with code:
http://graphics-geek.blogspot.com/2010/04/video-event-performance-in-flex.html
Hope that helps.