Qt Widget click 2D pos to 3D world coordinate - qt

Question

My Qt widget show the 3D world.
I have the 3D world's Qt3DRender::QCamera
How can I use Qcamera to convert the mouse position in widget to 3D world coordinate?
I tried to use point * .viewMatrix4x4().transposed().inverted() but it is wrong.

Answer 1

You cannot simply convert a 2D click to a 3D coordinate, only to a 3D vector which is a direction vector. You can only obtain a 3D coordinate if you have an object which is underneath the mouse.
In this case, you can use QObjectPicker to obtain the coordinate.
The steps to do so are:
Create the object picker in your scene hierarchy (it's best to create it as a child of the object you want to pick, but I'm not sure if that's necessary)
Attach the object picker as a component to the object you want to pick
Connect the object picker's signal that gets fired when you click the object to a function of yours that gives you the coordinates
You can also check out the manual Qt3D test on GitHub. It's in QML but you should be able to translate it to C++ (if that's what you're programming in).

Answer 2

I know this question is old, but I had the same problem and wanted to go from qt3d window coordinates to 3d space. I hunted around in the qt3d source code and was able to come up with the following. I think it should work as long as you are using the Qt3DWindow from Qt3DExtras and no additional Viewports.
QVector3D mouseEventToSpace(const QMouseEvent* mouseEvent, const Qt3DRender::QCamera* camera, const QSurface* surface) {
const QPointF glCorrectSurfacePosition{static_cast<float>(mouseEvent->pos().x()),
surface->size().height() - static_cast<float>(mouseEvent->pos().y())};
const QMatrix4x4 viewMatrix{camera->viewMatrix()};
const QMatrix4x4 projectionMatrix{camera->lens()->projectionMatrix()};
const int areaWidth = surface->size().width();
const int areaHeight = surface->size().height();
const auto relativeViewport = QRectF(0.0f, 0.0f, 1.0f, 1.0f);
const auto viewport =
QRect(relativeViewport.x() * areaWidth, (1.0 - relativeViewport.y() - relativeViewport.height()) * areaHeight,
relativeViewport.width() * areaWidth, relativeViewport.height() * areaHeight);
const auto nearPos = QVector3D{static_cast<float>(glCorrectSurfacePosition.x()),
static_cast<float>(glCorrectSurfacePosition.y()), 0.0f}
.unproject(viewMatrix, projectionMatrix, viewport);
return nearPos;}