I just tried out Qt WebGL and was thrilled to see my app running in the browser w/o making any changes (other then starting it with -platform webgl)!
I would like to use WebGL for screen sharing so that it would still usable on the device while also being able to interact with the app through the browser. Is this somehow possible with the current platform plugin or would it be possible to extend the platform plugin to support this in the future?
Qt WebGL streaming is intentionally only done for a single user per application. The reasons are mentioned here in a presentation about Qt WebGL streaming:
Why single user?
Problem with user input
Problem with querying the GPU
We can improve security
However, I found a blog post presenting a solution: start multiple parallel processes of the Qt application, one per use, and then sync the state of all these processes using Qt Remote Objects. One of the processes would be the "master application", and the others would duplicate what it shows. The application's state has to include everything that influences its rendered content, including model content and window size.
A detailed recipe for this technique can be found in this article.
There is also this section in a Qt blog post that confirms that this approach is possible:
By the way, there is an idea to complement streaming with an ability of mirroring as in some cases having the latter is more important.
Speaking about mirroring, I would like to mention our recent webinar [edit: link update by me] that we had together with Toradex. There you can see an interesting combination of WebGL streaming and Remote Objects, which allows you to implement mirroring functionality as of now already.
Within the webinar video linked above, the demonstration of mirroring / screen sharing starts here. This type of mirroring is even two-way, allowing to operate the application from multiple screens simultaneously.
Unlike in "real" screen sharing, the mouse pointer would not be shared. You might however be able to track the mouse pointer position as a state property of the master application and then paint an "artificial" mouse pointer at that position in the client applications.
Related
I'm considering moving a project from cef (CefSharp) to WebView2 WPF. My preliminary tests shows that WebView2 API have the interface I need for this port. I am afraid that I can miss something that can prevent me from switching to WebView2 and I'll figure this out at later stage of this transition. If anyone went through this process, please share if I need to be aware of something that can be a roadblock for this transition. Are there important APIs from cef that are missing in WebView2?
This issue in WebView2's Github answers your question in part. And I would like to add my take.
A summary:
WebView2 doesn't have an API for simulating user input, it's unsuitable for automation.
WebView2 can't render offscreen. No running in the console, as a service, on a server.
WebView2 runs in a separate process; CefSharp runs in the application's process.
WebView2 has a sandbox; CefSharp doesn't.
This answer is based on my personal experience and is not an exhaustive comparison.
Process Model
The Chromium process model has the main process and many auxiliary processes. It is, of course, a bald simplification, but it will do.
Both CefSharp and WebView2 follow this process model with one very significant difference. CefSharp starts Chromium in the application's process, and WebView2 starts it as a separate process.
The in-process model is common to all CEF-based browsers. Pros: the browser starts faster. Cons:
If CEF crashes, it takes the application down with it.
If there's a vulnerability in CEF or Chromium, it can also expose the application's memory.
The out-of-process model is popular among proprietary browsers. Not only WebView2 works this way, but also DotNetBrowser and EO.WebBrowser. Pros:
It's safer.
It doesn't bite off the application's RAM.
Cons:
It starts longer.
It may be slower because of inter-process communication.
Occasionally, there are issues with the focus and drag-and-drop because the browser window belongs to another process. Not applicable for the offscreen rendering, though.
CefSharp process model. WebView2 process model.
A score for WebView2.
Offscreen Rendering
There are two approaches to the rendering of embedded web content. One is to shake off the bells and whistles from an actual Chromium window and embed it into the application. We call this "windowed" or "heavyweight" mode. Another approach is to render web content in memory and draw it on an arbitrary surface. It's called "offscreen rendering."
WebView2 supports only windowed rendering. This mode has two significant drawbacks: it requires a window to function, and the browser always stays on top (aka the airspace issue).
CefSharp also has offscreen rendering. It allows the application to run in the console, overlay web content with other controls, render web content in Unity3D, etc.
A score for CefSharp.
Automation
The bread and butter of automation is simulating user input. CefSharp has an API for dispatching "real" mouse and keyboard events to the browser. The browser handles these events as user gestures, and JavaScript can't tell between them and real human input. Look at the SendMouse* and SendKey* methods in IBrowserHost.
There's no such API in WebView2, only workarounds based on Win API.
A score for CefSharp.
Sandbox
By default, all Chromium processes are sandboxed. It makes them more secure. The Chromium team explains the sandbox in great detail here.
WebView2 and other out-of-process libraries are sandboxed as well.
CefSharp doesn't support sandboxing and, it seems, never will (#697).
A score for WebView2.
See also
CefSharp supports only primary AppDomain.
CefSharp doesn't support commercial codes by default.
In addition to the other answer I'll add some additional points
License
CefSharp is open source, 3-BSD licensed. You can customise the code, fix bugs, submit pull requests, etc.
WebView2 is not currently open source. You need to wait for Microsoft to fix bugs. Currently WebView2 team releases updates every four weeks. Even if a bug is fixed quickly you can be waiting weeks before it's released.
Privacy
WebView2: Data collection privacy notice (at time of writing)
The software may collect information about you and your end users’ use of the software, and send that to Microsoft. Microsoft may use this information to provide services and improve our products and services. There are also some features in the software that may enable you to collect data from users of your applications. If you use these features to enable data collection in your applications, you must comply with applicable law, including providing appropriate notices to users of your applications. You can learn more about data collection and use in the help documentation and the privacy statement at https://aka.ms/privacy. Your use of the software operates as your consent to these practices.
Microsoft has yet to disclose exactly what is captured. Users must opt-out of data collection.
A number of users have raised concerns regarding the data telemetry. There are a number of discussions regarding the privacy concerns here and here.
Distribution Model
Both CefSharp and WebView2 require a specialised version of Chromium to be installed/copied on your computer.
Fixed Version: A fixed version of Chromium is bundled with your application. You control which version.
Evergreen: Chromium is is installed then automatically updated via windows update. On some OS's it will be pre-installed. Currently limited/no control over if the runtime is updated.
CefSharp
WebView2
Fixed
[x]
[x]
Evergreen
[x]
Chromium is rapidly changing, there's a new major version every four weeks at time of writing. Each major release adds/removes features. Security policies are changed.
What worked flawlessly on a previous version of Chromium may no longer be allowed for security reasons. Deprecated features are frequently removed.
For complex applications it's important/imperative to tested against the specific version of Chromium your application uses. This applies to all Chromium based browser.
Evergreen WebView2 Runtime automatically updated via Windows Update sounds perfect? Get the latest version without having to do everything.
What happens when a bug makes it into a Stable Release? Did you test your application against the dev/beta channels to flag upcoming issues? No? You could be inundated with user complaints their application now crashes.
A quick look though the WebView2Feedback Issue Tracker and you can see examples of this happening.
M108 release was particularly problematic with many users reporting issues.
The following are just some examples
https://github.com/MicrosoftEdge/WebView2Feedback/issues/3029
https://github.com/MicrosoftEdge/WebView2Feedback/issues/3068
https://github.com/MicrosoftEdge/WebView2Feedback/issues/3062
https://github.com/MicrosoftEdge/WebView2Feedback/issues/3032
For those using an Evergreen model you must test your application on every dev/beta release to ensure your application works as expected.
WPF
CefSharp provides two WPF implementations, the first CefSharp.Wpf renders every frame to a bitmap, this allows for a native WPF experience. You can apply transforms, layer elements over the top, etc. There are no airspace issues. There are a few downsides the main being performance is lower (there are a few things you can do to squeeze some extra frames).
The second is CefSharp.Wpf.HwndHost, it's a HwndHost based implementation, it's like embedding the WinForms version in WPF with binding support, you get much better performance, no transforms, cannot layer over the top. Airspace issues apply.
WebView2 currently only provides a HwndHost implementation, airspace issues apply. There is talk of providing a more native WPF implementation
Background Information: I'm building a small cross-desktop application using Qt5. I want to limit this application to only one specific task. Anything else should be delegated to other applications installed on the user's system. More concretely:
I want my application to be opened by other applications, such as image viewers, and be suggested in context menus (e.g. "Open with..").
I want my application to present the user with a list of applications he or she can use to continue working with the result. (e.g. image viewers' "open with" menu option).
As far as I can tell, input integration can not be implemented in a desktop agnostic way; i.e. I must install .desktop application files in XDG desktops or define appropriate keys under HKEY_CLASSES_ROOT\Applications in the windows registry before the desktop can suggest the applications to users.
Question: Since my application will be responsible for providing the user with a list of applications to open the output, does Qt offer anything I can use to limit the amount of platform-specific code I need to write and test?
I have already found many ways to get this information in a desktop-specific way (e.g. Gnome, KDE, or even processing XDG directories or relevant windows registry entries directly) but my hope here is to limit the the amount of time I need to spend on each desktop.
Thank you for your time and apologies for the poor English.
I am interested in Kotlin GUI system and I found Tornado FX.
I noticed that we can transform javaFX app as a webapp, so my question is : Can we do the same with Tornado FX and if we can, How should we do that?
Anything you can do with JavaFX, you can do with TornadoFX, as it is basically a higher level API on top of JavaFX.
JavaFX itself has no rendering capability that targets the web, but JPro has rather geniously made that possible by "rendering" the JavaFX UI as SVG directly to the browser using WebSockets.
Infact, TornadoFX has special support to make it easier to create JPro based apps.
Your starting point would be https://www.jpro.one/
The TornadoFX guide contains information about how to use JPro as well:
https://github.com/edvin/tornadofx-guide/blob/master/part2/Scopes.md#using-tornadofx-with-jpro
JPro is a commercial offering, starting av 99 EURO per month for one server with 8GB memory at the time of writing (2018-11-20). Each user would get their own instance of the application, though still running within the same JVM.
While this system works remarkably well, it is best suited for situations where there is low latency and preferably high bandwidth between the client and the server, as every event will be transferred over the WebSocket connection. It would typically be a bad choice for a public facing web page, but it is doable. In fact, http://jpro.one is actually a JavaFX app!
Disclaimer: I am not affiliated with JPro, I'm merely telling you about the (only) way to make JavaFX apps available through the browser.
I am experienced writing automation tests for web apps using Selenium.
However I now have to automate a Windows Desktop app which I'm new to.
I'm using QTP 11 (old version) and I can get QTP to login type username/password to the desktop app. However when the app loads there are icons like a Windows desktop. I tried using ObjectSpy on the Actions folder icon but it can't find the object ID and it thinks the icon is a WinObject("COMPOSITE")
Also tried using QTP Record feature but the code that it generates uses hardcoded x and y values. I don't want to use x,y values as if the Actions icon moves 3cms left or right in future the test will fail.
e.g.
Window("Loan IQ").WinObject("COMPOSITE").Click 369,33
Need help finding the object ID in a Win32 app. Thanks
First of all you should make sure that UFT is configured to test your application. In the Record and Run Settings dialog, make sure that either _any windows application__ is selected or your app is explicitly listed.
If this doesn't improve the situation you can try using image based testing (aka Insight).
WIN32 Apps can be a nightmare to automate especially with QTP 11, as it is a kinda outdated version. If you want to get stable automation I propose the following:
Upgrade to a newer version of UFT (14+)
This will most probably not help you indentify the objects but will have a lot of new technologies supported that may help you as described in the following steps
Use Image Based Recognition
Even if your screen resolution changes UFT is still able to identify pictures.IT does not use absolute vectors to compare bitmaps but a different technology which I won't go in detailed (long story short, screen resolution changes are okay)
Provide support for your Widgets
Microsoft has 2 frameworks that can be used to provide UI Automation capabilities (initially for people with accessibility needs, but now is used for RPA and GUI Testing). UFT supports the MSAA and UIA frameworks of Microsoft so if your company is ready to implement support for the UI widgets via one of these Technologies, you are on your way for a smooth Test Automation Experience. Please note: This is mostly a huge investment, so if the tool is something internal and not planned for longer term usage, go with the image based Recognition
Users of my Flex application report that sometimes the application is freezed when the browser window is minimized or they select another active tab over it. In this suspended state, the application receives no CPU share and all network connections it uses are closed. When the browser window is restored, the application is resumed. This happened with Safari 5 on Mac OS Leopard, with both Flash player 10.0 and 10.1. I searched a lot but I could not find any information about such behavior.
This behavior is not reproducible on each Mac with Safari, so my questions are:
Under which circumstances this may happen?
Is it possible entering in suspended state to be prevented and how?
Is it possible for the Flex application to be notified about going into sleep mode and wake up back?
This is a Safari thing and is by design. Newer versions of Safari suspend flash content that is not in the foreground tab (not sure when this started, version 4?) For instance, if you have multiple tabs open, each with a youtube video playing and you go back and forth between the tabs, only one of the videos will be playing at a time. To answer your specific questions:
This will happen to any flash content that is in a background tab (not sure about the minimized state.)
Not aware of a means of disabling this behavior.
You may want to dig around in the Safari documentation to see if there is some sort of JavaScript event that you can grab onto, but I don't think you are likely to have success there.
Good luck.
Are your users using Flash Player 10.1 ? As part of the performance improvements in 10.1; I believe an application in a minimized state will get throttled in order to use less system resources / battery power.
I don't think there is any way to prevent this; and no APIs exposed that relate to this.
Other people I've spoken to have had issues with using local connections between minimized apps and active apps. I'd bet there is already a bug in the bug base on that.
Wrt the following above:
.I don't think there is any way to prevent this; and no APIs exposed that relate to this.
You may want to check out if Silverline from Librato can help you control how much and which applications get what system resources (CPU, memory, Disk and Network IO) with dynamic control based on application demand. If the above issue is a feature of flash - then obviously it may not help. But if you are trying to say run multiple applications / processes and would like to control who gets how much system resources (dynamically) then you could try Silverline - it does not require any changes to OS or app. http://silverline.librato.com