Qt - QML Screen Mirroring in a dual monitor setup - qt

Question

I've developed an application, with the UI done using QML. What I was asked to do now is to make it so that when a second monitor is used, the second monitor shows everything that the program is doing. At first I thought of just telling the client to configure Windows to clone its screens. However when the applications uses some of its functionalities I need for the cloned screen to display certain indicators in the cloned screens but not on the original screen.
So my question is, How can I accomplish this. How can mirror what is happening in one screen, while maintaing enough control to draw in one and not in the other.
My only idea is to use a timer to take as screen shot at regular intervals and show that image in the second screen.
Is this doable?

Answer 1

Taking periodic screenshots, although perfectly doable, is undesired because of the impact to performance. Instead you should make use the main window's onFrameSwapped() signal, to grab images only when a new frame is generated.
Ideally, you'd want to make use of Layer or ShaderEffectSource, as suggested by #dtech, to read and re-render the frame straight from the GPU. Unfortunately, due to limitations in Qt Quick's Scene Graph, it is not possible to accomplish this across separate windows without destabilizing the source window's scene graph.
Update: Nevertheless, I've found a way to emulate the desired features of ShaderEffectSource using ShaderEffect instead. Your other options are to either copy frames through the CPU using Item's grabToImage function, or to re-implement QQuickView (your QML window) in C++ to grab images from the QML Engine when a new frame is generated.
Working Solution 1: Use ShaderEffect (Update: New Solution)
The new approach I've found consists of using ShaderEffect on a child Window and forcing it to update by calling that Window's update() method, which is inherited from QWindow. By using ShaderEffect, you guarantee the source image will be accessed only through the GPU, increasing performance by orders of magnitude over solution #2.
import QtQuick 2.15
import QtQuick.Window 2.15
Window {
id: mainWindow
title: qsTr("Main Window")
visible: true
width: 400
height: 200
color: "#0F0"
Rectangle {
id: viewport
color: "#00F"
width: parent.width/2
height: parent.height/2
anchors.horizontalCenter: parent.horizontalCenter
anchors.verticalCenter: parent.verticalCenter
// Draw QML Item into an off screen layer
layer.enabled: true
// Performance tweaks
layer.smooth: false
layer.mipmap: false
}
onFrameSwapped: {
// Update mirror window manually with each new frame
projectionWindow.update()
}
Window {
id: projectionWindow
property int screenID: 1
transientParent: mainWindow
visible: true
x: Qt.application.screens[screenID].virtualX
y: Qt.application.screens[screenID].virtualY
width: Qt.application.screens[screenID].desktopAvailableWidth
height: Qt.application.screens[screenID].desktopAvailableHeight
flags: Qt.FramelessWindowHint
color: "#000"
visibility: Window.Maximized
ShaderEffect {
id: img
// Set source to copy visuals from
property variant source: viewport
// Setting shader to the same resolution as the source may improve performance
width: viewport.width
height: viewport.height
// Performance tweak
blending: false
}
}
}
Triggering update() manually has the side effect of preventing the child from updating on its own. Calling the update method on the source window from the child Window once whenever needed works around this limitation.
It is worth noting that this solution only works with Qt's basic renderer, which is single threaded. Qt prevents the texture from being accessed from a diferent rendering threads, giving the following error message and returning a null pointer:
QQuickShaderEffectSource::textureProvider: can only be queried on the rendering thread of an exposed window
The performance gained from this method is far greater than the performance loss of using the single threaded renderer. You can force use of the single threaded renderer by setting an environment variable at the start of your application. This can be done from inside the app by adding the following code at the very start of your main function:
#if defined(Q_OS_WINDOWS)
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
putenv("QSG_RENDER_LOOP=windows");
#else
putenv("QSG_RENDER_LOOP=basic");
#endif
#elif defined(Q_OS_MACOS) || defined(Q_OS_LINUX)
setenv("QSG_RENDER_LOOP", "basic", 1);
#endif
Qt's documentation states that using the basic renderer reduces portability, but I can attest this approach works fine on Windows, macOS, and Linux and that it does not work on Haiku OS.
I've switched to this solution on the most recent versions of a project I develop. You can study the full implementation at:
https://github.com/Cuperino/QPrompt/blob/072f0a7808f3a361cb93bca8961e884ac9d8bb56/src/kirigami_ui/PrompterPage.qml#L760
https://github.com/Cuperino/QPrompt/blob/072f0a7808f3a361cb93bca8961e884ac9d8bb56/src/kirigami_ui/main.qml#L847
https://github.com/Cuperino/QPrompt/blob/072f0a7808f3a361cb93bca8961e884ac9d8bb56/src/prompter/ProjectionsManager.qml#L261
Working Solution 2: Use Item's grabToImage (original answer)
Another easy approach is to grab frames from QML, using an Item's grabToImage() function. Since Window is not an item in itself, you'd have to grab the image from one of its elements. In this example, I grab the image from an item called viewport each time a frame is swapped on mainWindow, using the onFrameSwapped() signal. Then the path to that image in memory is set as the source for the image in the second window, named projectionWindow. The second window will open on the screen set by the screenID variable; it is also set to be a frame-less window with its visibility is set to either Maximized or FullScreen, such that it is the only window seen on the second screen.
import QtQuick 2.15
import QtQuick.Window 2.15
Window {
id: mainWindow
title: qsTr("Main Window")
visible: true
width: 400
height: 200
color: "#0F0"
Rectangle {
id: viewport
color: "#00F"
width: parent.width/2
height: parent.height/2
anchors.horizontalCenter: parent.horizontalCenter
anchors.verticalCenter: parent.verticalCenter
}
onFrameSwapped: {
viewport.grabToImage(function(result) {
projectionWindow.frame = String(result.url);
});
}
Window {
id: projectionWindow
property int screenID: 1
property alias frame: img.source
transientParent: mainWindow
visible: true
x: Qt.application.screens[screenID].virtualX
y: Qt.application.screens[screenID].virtualY
width: Qt.application.screens[screenID].desktopAvailableWidth
height: Qt.application.screens[screenID].desktopAvailableHeight
flags: Qt.FramelessWindowHint
color: "#000"
visibility: Window.Maximized
Image {
id: img
anchors.fill: parent
fillMode: Image.PreserveAspectFit
// Performance tweaks
asynchronous: true
cache: false
}
}
}
I used to employed this solution on a project I develop. You can study the full implementation at:
https://github.com/Cuperino/QPrompt/blob/a873a2ec9b0619ec1bfdbde22fe90850e76969a5/src/kirigami_ui/main.qml#L828
https://github.com/Cuperino/QPrompt/blob/a873a2ec9b0619ec1bfdbde22fe90850e76969a5/src/prompter/ProjectionsManager.qml#L260
Alternate, incomplete, solution
The third, more manual, and unfinished solution consists of the following: You'd inherit from QQuickView or QQuickWindow, and then send the image to a second window on either the QQuickWindow::afterRendering() or the QQuickWindow::frameSwapped() signal. You would use a QQuickFramebufferObject or some other rendering pipeline such as DirectX, Metal or Vulkan, to render and grab the frame off screen. Using OpenGL as your renderer, means negating all the performance advantages that come from using the native rendering pipelines supported by Qt 6. You may need to implement the frame grab once per pipeline to get all performance benefits.
https://doc.qt.io/qt-6/qtquick-visualcanvas-scenegraph.html
The following talk by Giuseppe D’Angelo shows how some of this is setup in Qt 5. It doesn't show how to copy the contents to another window/screen but it could aid in this regard.
https://www.youtube.com/watch?v=V_idc9BBRuI
https://www.youtube.com/watch?v=D-7fVGIBz6k

Answer 2

You can use particular QML elements as texture sources and easily duplicate them via trivial fragment shaders.
You definitely do not want to take screenshots and draw that image back, it is wildly inefficient.
Keep in mind that it will be just a visual duplicate, it will not take user input. If you want it to be interactive in both windows, then you should simply use a single data object and connect it to two individual GUI elements.
OK, here is the code, but unfortunately, it evidently uncovers a bug in QML, as the implementation doesn't seem to work across different windows:
Window {
id: mainw
visible: true
width: 640
height: 480
title: qsTr("main window")
Row {
spacing: 5
Rectangle {
id: source
width: 100
height: 100
color: ma.containsPress ? "red" : "cyan"
Text {
text: "adoy"
anchors.centerIn: parent
}
MouseArea {
id: ma
anchors.fill: parent
}
}
ShaderEffectSource {
width: source.width
height: source.height
live: true
sourceItem: source
}
}
Window {
visible: true
width: 640
height: 480
title: qsTr("another window")
x: mainw.x + width + 10
y: mainw.y
Row {
ShaderEffectSource {
width: source.width
height: source.height
live: true
sourceItem: source
}
Rectangle {
width: 100
height: 100
color: "blue"
}
}
}
}