Imagine a huge rectangular grid filled with tiles. The individual tiles are not very complicated, they are svg images containing a low amount of shapes.
The number of different types of tiles in not very large, I estimate in the low hundreds. However, the grid can become very large, so the number of total tiles is huge (at least tens of thousands, maybe more).
I have to be able to smoothly scroll the grid both horizontally and vertically, as well as smoothly zoom it in and out. I also have to be able to jump to a specific position.
It would also be nice if I could populate it asynchronously, first the elements which are actually visible, and then the rest. This means that a table-handling class where I first have to add rows and columns in a loop would not be the best solution, because the starting position is not necessarily the upper left corner.
Zooming is simply achieved by having all the width and height properties of the items within a tile specified as a multiple of a scaling factor. The svg shouldn't be a problem as the number of different images is not high, it should be able to be cached. In the unlikely case svg became the bottleneck, I could just use sets of different pngs in different resolutions.
I tried (or considered) the following approaches:
Using the methods of the SameGame example, creating QML objects dynamically (Component.createObject). This works if the number of objects is small, but is very slow with a large number of objects. Even if the objects are completely empty, this method takes a very long time.
Using a Repeater inside a Flickable. The Flickable contains a Grid, which is populated by a Repater. The Grid, of course, is immense.
This method is faster than creating the objects dynamically, but still inefficient as the number of tiles grows. The QML engine keeps track of every item, even those which are not visible. Zooming is also quite slow, as the properties of every item are recalculated, not just the visible ones.
Using a GridView. This looks like the perfect solution at a first glance. The GridView inherits Flickable, and it also takes care to only render contents which are within the bounds of the view. Even a test case with millions of svg images runs reasonably fast, and it scrolls and resizes smoothly. There is only one problem: The GridView is only flickable either horizontally or vertically, but not both. There has been a feature request about this since 2012, but it still seems to be ignored.
Using a QGraphicsView directly. It is capable of displaying, scrolling and zooming the needed amount of elements, but it's not QML-based. The rest of my GUI is in QML, and I've only read horror stories about combining QML and QGraphicsView. I've never seen any reasonable examples of it.
What other solutions are there? Some horrible hack of using Javascript to add and remove rows and columns of a simple GridLayout (which is only a couple rows and columns larger than the visible area) while it is moved around in a Flickable? Or just embedding an OpenGL window and drawing everything manually?
I hope this shouldn't be an impossible task. There were strategy games written more than 20 years ago for DOS and Windows 95 which could handle this amount of tiles, while additionally having textures and animations.
Yeah, Qt is very good at ignoring community suggestions for years, even if they would be extremely useful, considered important, and happen to be the most up-voted, such as zip support.
I personally wouldn't bother "fixing" GridView, but rather implement something from scratch that suits my specific requirements, in C++ so that it is fast and efficient. And it will be very easy if your tiles are uniform squares, and it sounds like you could get away with that, even if the actual images inside are not square. This will make it very easy to determined their positions programmatically, and also determine the top left corner tile, how much tiles per line and the stride for the subsequent lines. Then as the visibility rectangle moves you iterate your container and signal to create QML elements for those which enter visibility. Easy peasy.
You don't need anything fancy, just inherit QObject, register the type to QML, then go and populate it's internal "model". You definitely do not want to have all the objects in memory, even if the scene graph is smart enough to not render them, it will still be processing them, I suspect your drop in FPS is not the product of a GPU but a CPU bottleneck.
The actual grid object can emit creation and destruction signals with their data Q_SIGNAL void create(x, y, imgPath);, so you bind custom handlers on the QML side, which will give you flexibility and ease of use, such as easily specifying the "delegate" object, it will be more elegant than doing the actual creation/destruction in C++. You can use bindings on the QML side for the few items that are visible to track when they go out of screen to self-destruct, that would minimize complexity, as you won't have to track all the "living" objects.
Component {
id: objComponent
Image {
property bool isVisible: { is in grid.visibleRect ??? }
onIsVisibleChanged: if (!isVisible) destroy()
}
}
MyGrid {
id: grid
contentX: flickable.contentX
contentY: flickable.contentY
onCreate: objComponent.createObject(flickable.contentItem, {"x" : x, "y" : y, "source" : imgPath})
}
Flickable {
id: flickable
contentWidth: grid.contentWidth
contentHeight: grid.contentHeight
}
Normally, when a user has a question, important enough to offer a bounty I'd produce working code, but unfortunately I am currently too busy. The concept is pretty simple though and should not be too problematic to implement.
Related
I have already read
What is the difference between SVG and HTML5 Canvas?
&&
https://en.wikipedia.org/wiki/Canvas_element#Canvas_versus_Scalable_Vector_Graphics_.28SVG.29
So i am aware of the basic differences, but i was wondering if anyone had encountered any practical difference between the two within the context of ggvis and shiny apart from SVG inability to deal with NA's in the data
The short answer:
SVG would be easier for you, since selection and moving it around is already built in. SVG objects are DOM objects, so they have "click" handlers, etc.
DIVs are okay but clunky and have awful performance loading at large numbers.
Canvas has the best performance hands-down, but you have to implement all concepts of managed state (object selection, etc) yourself, or use a library.
The long answer:
HTML5 Canvas is simply a drawing surface for a bit-map. You set up to draw (Say with a color and line thickness), draw that thing, and then the Canvas has no knowledge of that thing: It doesn't know where it is or what it is that you've just drawn, it's just pixels. If you want to draw rectangles and have them move around or be selectable then you have to code all of that from scratch, including the code to remember that you drew them.
SVG on the other hand must maintain references to each object that it renders. Every SVG/VML element you create is a real element in the DOM. By default this allows you to keep much better track of the elements you create and makes dealing with things like mouse events easier by default, but it slows down significantly when there are a large number of objects
Those SVG DOM references mean that some of the footwork of dealing with the things you draw is done for you. And SVG is faster when rendering really large objects, but slower when rendering many objects.
A game would probably be faster in Canvas. A huge map program would probably be faster in SVG. If you do want to use Canvas, I have some tutorials on getting movable objects up and running here.
Canvas would be better for faster things and heavy bitmap manipulation (like animation), but will take more code if you want lots of interactivity.
I've run a bunch of numbers on HTML DIV-made drawing versus Canvas-made drawing. I could make a huge post about the benefits of each, but I will give some of the relevant results of my tests to consider for your specific application:
I made Canvas and HTML DIV test pages, both had movable "nodes." Canvas nodes were objects I created and kept track of in Javascript. HTML nodes were movable Divs.
I added 100,000 nodes to each of my two tests. They performed quite differently:
The HTML test tab took forever to load (timed at slightly under 5 minutes, chrome asked to kill the page the first time). Chrome's task manager says that tab is taking up 168MB. It takes up 12-13% CPU time when I am looking at it, 0% when I am not looking.
The Canvas tab loaded in one second and takes up 30MB. It also takes up 13% of CPU time all of the time, regardless of whether or not one is looking at it. (2013 edit: They've mostly fixed that)
Dragging on the HTML page is smoother, which is expected by the design, since the current setup is to redraw EVERYTHING every 30 milliseconds in the Canvas test. There are plenty of optimizations to be had for Canvas for this. (canvas invalidation being the easiest, also clipping regions, selective redrawing, etc.. just depends on how much you feel like implementing)
There is no doubt you could get Canvas to be faster at object manipulation as the divs in that simple test, and of course far faster in the load time. Drawing/loading is faster in Canvas and has far more room for optimizations, too (ie, excluding things that are off-screen is very easy).
Conclusion:
SVG is probably better for applications and apps with few items (less than 1000? Depends really)
Canvas is better for thousands of objects and careful manipulation, but a lot more code (or a library) is needed to get it off the ground.
HTML Divs are clunky and do not scale, making a circle is only possible with rounded corners, making complex shapes is possible but involves hundreds of tiny tiny pixel-wide divs. Madness ensues.
I have past content from the following link.
Please see this link for more details
HTML5 Canvas vs. SVG vs. div
I'm developing a application which has a ListView which contains items which needs complex cell layouts. The cells are in variable heights and some of the cells tends to be larger than the view port height.
But when the ListView is filled with items the scroll thumb tends to resize its self while scrolling, which makes it hard to hold onto the thumb while scrolling. This happens mainly when passing through different size of cells.
This is not a problem in Swing if I create a same kind of a cell render to be used with the JList. This problem is there in JavaFX 2 and JavaFX8 both.
When looking at the VirtualFlow which is responsible for layout of the ListView and handle scrolling, it seems that the scrollbar thumb side (lenghtbar) is calculated based on the cell count and the visible cell count, which is actually a problem when it comes to lists which has variable heights of cells.
So is this the future of the scroll bar behavior for Java FX list views? or is there any solution available for this problem? Or should I try to hide the scrollbar and provide a different user interaction to scroll?
This problem is already reported under https://javafx-jira.kenai.com/browse/RT-25059 and fixed in Java8 upto some extend. So if this fix is needed on JavaFx2 we have to backport the changes under commit http://hg.openjdk.java.net/openjfx/8/controls/rt/rev/81cc13fe6f96
To get this changes in JavaFX 2.2 you need to apply the required changes on to FX2.2 VirtualFlow.java class and load those changes before the jfxrt.jar is loaded. Another approach is if you don't like to mess up with the jfxrt classes is to have you own ListView which uses your own Skin and the patched VirtualFlow version may be with a different name. But this might require lot of customization compared to first solution.
More approaches are welcome :).
I have a Direct3D11 scene set up in SlimDX in a window. The rendering is done in a separate thread.
Is there a way to keep the renderer from stretching the image when it draws to the resized control? I've tried ModeDescription.Scaling = DisplayModeScaling.Centered and it doesn't seem to have any effect. Is there something I'm missing?
(I already am updating the render target size. The reason I ask this is that when I resize the control it stretches the image to fill the control for a split second before the render target gets updated with the new size. This has the result that as I resize it, it flickers terribly. If I could reset the render target just slightly faster it might get rid the the flicker. Keeping the image in the corner without scaling it is perfectly fine since ultimately it won't be scaling at all.)
Workaround 1: One can put the render target inside a control. When the window resizes only resize the control though a special method that first stops the rendering, then update the buffers and begins rendering again. It's a bit of hack, though. I have to wait for the render cycle to complete, then block it, then resize, then unblock.
Workaround 2: A similar workaround is to check for a resize flag in the render loop rather than interrupting it. The renderer should be able to draw directly without scaling. This is much more acceptable performancewise. However, a blocking call to the UI thread must be made to execute the actual resize.
Workaround 3: Rather than resizing the control at all, one could make it as large as the maximum size it could be, but clipped (inside the window). No resize is necessary, but a scissor rectangle must be maintained in a similar manner to the workarounds above unless you don't mind rendering a whole lot of offscreen pixels. Rendering twenty or so extra rows and columns of pixels does have the favorable effect of supplying immediate image at the edge when the window is resized back larger.
Ideally the resize should be done directly from the UI thread (no fooling around with delaying it and reentering the UI thread from the render thread). It's not the render thread's responsibility and it slows it down. Likewise, the buffer resize should be done in the render thread for maximum performance (no fooling around with waiting/blocking/resizing/unblocking). The only reason this does not work is that the render thread scales if the resize is done before the buffers are resized.
So my question still stands: Is there a way to render without scaling?
I am going to answer this in terms of the raw Win32 APIs involved, which might require a bit of finesse to translate to a managed .NET environment & SlimDX.
When you are dragging or resizing a window, windows basically hijacks your message pump and creates a new one specifically designed to do the resizing logic efficiently. This operation is more or less modal until it is completed. Many messages you would normally get are quite literally blocked until the resize or drag is completed. In the app terms you get a WM_ENTERSIZEMOVE when this behavior begins, and either WM_EXITSIZEMOVE or WM_CAPTURECHANGED when it ends. You need to check both of these messages, as alt-tabing out when doing a drag will send WM_CAPTURECHANGED and never a WM_EXITSIZEMOVE!
All this means that when you get a WM_ENTERSIZEMOVE you can set a flag and know that all WM_SIZE and WM_MOVE messages that occur afterwards are for the drag operation. Typically resizing rendertargets and tracking down and de-allocating all default pool resources is a very slow operation, and well worth defering until the drag/resize has completed. This has the side effect of making the window stretch which is exactly the same problem you are describing here, and you want to fix that.
It should be possible to add special handlers in WM_SIZE, WM_MOVE, or WM_SIZING, and WM_MOVING that forces a syncronous render and repaint via SendMessage (as opposed to PostMessage) when they occur, but you will need to make sure you only do this when inside the modal loop owned by WM_ENTERSIZEMOVE.
I would recommend either forcing the window to a fixed size (which seems rather common) or not redrawing the D3D control until the changes are done (that or simply clear it black).
There seems to be a way to do what you want, as I have seen games capable of changing size with no flicker, but I've no idea how that's done, unfortunately. Locking the window size and providing a menu was the old solution.
Is it possible to get flex renderer recycling while using browser scrollbars?
I have a flex tree control with custom item renderers for rich editing of a server-side data structure which may have any number of child nodes. If at all possible, I'd like to avoid using a flex scrollbar if the content exceeds the viewable range, instead preferring to scroll with the browser's scrollbar. I could use javascript and ExternalInterface to resize the application when the tree's size changes, but my understanding is that this would cause renderers to be created for every row in the tree, which I would like to avoid for performance reasons.
The Flex Tree component doesn't support something like that out of the box. It will create enough renderers to fill it's entire height, assuming the data provider is that large. Since you want it to be larger than the browser window, it will create more renderers than those bounds.
If you were willing to subclass or patch the existing Tree, you could probably add new properties to override the starting value and the height used for the calculation of the number of renderers needed. I'm not sure how much work this would be, but it could be as simple as overriding a single function where this calculation happens.
Obviously, when the browser scrolls, you'd need to pass new values for the start and end values of what's displayed in your Tree.
I'm developing a Flex 2 application, and I noticed that part of the library which is responsible for moving GUI windows (TitleWindows) around when you drag them with the mouse gets confused if there is a clickable (buttonMode = true) sprite beneath them. When I say confused, I mean that the window is moved around normally for a while, but then at some point "jumps" into the upper left corner of the flash app, and makes very minor movement there. Then at some other point it jumps back. It is more difficult to explain than to experience, so please go and see for yourself. Here's how to reproduce the problem:
Go to http://www.panocast.com
In the left sidebar, choose "Real Estate"
Just below the bottom right corner of the flash window, choose "high res" by clicking on the rightmost icon.
When (part of) the video loads, click on the staircase. A TitleWindow will pop up.
Try dragging it around the screen. When the mouse cursor is moved above one of the clickable areas (like the staircase), the window is misplaced.
(Sorry, but can't give you a direct link, part of the page is generated dynamically.)
(What's makes the problem even more interesting is that for me, in "low res" mode, the problem does not occur! There is very little difference between the various modes.) I would really appreciate if someone told me what was going on here and how it can be fixed.
I'm not sure if it matters, but the underlying sprite is actually not just plain sprite, rather it is a Papervision3D renderer object with some 3D elements in it. I'm telling this because it is possible that the incorrect mouse coordinates somehow come from the texture UV mapped on the clickable objects.
I've managed to replicate this on the low res mode as well, so I don't think it's related to the resolution.
This looks to be because the MouseEvent is being handled by the TitleWindow AND the Papervision3D window. Perhaps you need to force stopImmediatePropagation() on one or the other? Or maybe switch off the MouseEvent handling for the Pv3D window when the TitleWindow pops up?
That's a tough one to debug without some source; something's apparently calling either move() or setting x and y properties on that TitleWindow and scheduling it be moved.
When I first read the post, it "smelled" like maybe a rotation miscalculation somewhere (using Math.atan vs. Math.atan2 can sometimes have that kind of effect), so you're right, it could have something to do with PaperVision, assuming you're not using Math.atan or setting rotation properties yourself anywhere. Just thought I'd mention it, though it's probably not happening in your case. You never know, though. ;)
More likely the LayoutManager is moving the component in response to a property change on the component. The Flex docs explain that in addition to setting its x and y properties, and explicit calls to move(), a UIComponent's move event can also be triggered when any of the following other properties change:
minWidth
minHeight
maxWidth
maxHeight
explicitWidth
explicitHeight
PaperVision or no, maybe that info might help you isolate the source of the move. Good luck.
I got this figured out. Apparently, this is a Papervision3D problem. There is a class deep inside Papervision3D called VirtualMouse, which is supposed to generate MouseEvents programmatically. This happens, for example, when the user interacts with any of the interactive objects on stage, e.g., a Plane with an interactive material on it (as in my case).
The problem is that the x and y coordinates of the generated event represent texture UV coordinates (just as I suspected) and not real world screen coordinates. When a TitleWindow (or any Panel object) is dragged, a "mouseMove" handler (among others) is added to the SystemManager, which then uses the stageX and stageY properties of the event object to determine the new position of the window. Unfortunately for VirtualMouse's mouse events, these are invalid, since the original x,y coordinates, which are probably used to determine the global stage coordinates are, as I said, not screen coordinates.
Honestly, I'm still unsure whether the events dispatched by VirtualMouse are used anywhere within Papervision3D itself, or they are just offered for convenience, but they sure make it difficult to integrate a viewport into a Flex program. Assuming that such events aren't necessary for PV3D itself, there is a one-liner fix for my problem, which must be added right after the creation of the viewport:
viewport.interactiveSceneManager.virtualMouse.
disableEvent(MouseEvent.MOUSE_MOVE);
BTW., there was a very similar (or rather, as it turns out, the same) bug with dragging sliders, also fixed by this line.