I need to calculate screen position outside of Three, with with no camera present.
I know the object position, camera position and camera target
I've seen lots of instructions such as three.js Vector3 to 2D screen coordinate with rotated scene
vector.project(camera);
vector.x = Math.round( ( vector.x + 1 ) * w / 2 );
vector.y = Math.round( ( - vector.y + 1 ) * h / 2 );
I understand Vector.project camera takes into account the camera settings, FOV etc I'd assume?
Vector3 has projectOnVector(), does this do the same thing as vector3.project(camera) ?
Is there a way to calculate where the object would be on screen without being able to access the camera?
Yes, the Vector3.project takes into account the camera settings...
You need to calculate a projection matrix as you are trying to transform a position from world space to view space. This is a great little animation describing the journey that point will make: https://jsantell.com/model-view-projection/mvp.webm (lifted from this useful page: https://jsantell.com/model-view-projection/).
If you look in the three source code it will show you everything you need to do this. Vector3.project is just applying the two matrices from the camera:
return this.applyMatrix4( camera.matrixWorldInverse ).applyMatrix4( camera.projectionMatrix );
So how do you get these matrices? The project matrix is generated here.
You can ignore the view, skew and zoom, so you just need near, far, aspect and fov.
updateProjectionMatrix() {
const near = this.near;
let top = near * Math.tan( MathUtils.DEG2RAD * 0.5 * this.fov );
let height = 2 * top;
let width = this.aspect * height;
let left = - 0.5 * width;
this.projectionMatrix.makePerspective( left, left + width, top, top - height, near, this.far );
}
If you need makePerspective it is here
The matrixWorldInverse is just that... take your world matrix and inverse it. Three.js does it here.
This gives you your view matrix. So, view matrix multiplied with the projection gives you your screen space position... just like in a shader i.e:
gl_Position = projectionMatrix * modelViewMatrix * vec4(pos, 1.0);
I'm assuming your point is in world space so you can ignore the model part as this just takes you from model to world.
Related
I need to be able to unproject a screen pixel into object space using Vulkan, but somewhere my math is going wrong.
Here is the shader as it stands today for reference:
void main()
{
//the depth of this pixel is between 0 and 1
vec4 obj_space = vec4( float(gl_FragCoord.x)/ubo.screen_width, float(gl_FragCoord.y)/ubo.screen_height, gl_FragCoord.z, 1.0f);
//this puts us in normalized device coordinates [-1,1 ] range
obj_space.xy = ( obj_space.xy * 2.0f ) -1.0f;
//this two lines will put is in object space coordinates
//mvp_inverse is derived from this in the c++ side:
//glm::inverse(app.three_d_camera->get_projection_matrix() * app.three_d_camera->view_matrix * model);
obj_space = ubo.mvp_inverse * obj_space;
obj_space.xyz /= obj_space.w;
//the resulting position here is wrong
out_color = obj_space;
}
when I output the position in color, the colors are off. I know I can simply pass in the object space position from the vertex shader to the fragment shader, but I'd like to understand why my math is not working, it will help me understand Vulkan and maybe learn a little math myself.
Thanks!
I'm not entirely sure what your problem is, but lets go over potential problems.
Remember, vulkan clip space is:
positive y = down,
positive x = right,
positive z = out,
centered at the middle of the screen.
Additionally, despite OpenGL's GLSL docs saying it is centered at the bottom left corner, in vulkan gl_FragCoord is centered at the top left corner.
in this step:
obj_space.xy = ( obj_space.xy * 2.0f ) -1.0f;
obj_space is now:
left x : -1.0
right x : 1.0
top y = -1.0
bottom y = 1.0
out z = 1.0
back z = 0
I'm almost entirely sure you don't mean your object space to have Y be negative at the top. The reasoning for y increasing starting from top to bottom is for images and textures, which on the CPU are ordered the same way, and now are ordered like that in vulkan.
Some other notes:
You claim your inverse is derivied from glm::inverse here:
glm::inverse(app.three_d_camera->get_projection_matrix() * app.three_d_camera->view_matrix * model);
But GLM uses OpenGL notation for matrix dimensions and handedness, and unless you force it to the correct coordinate system, it is going to assume right handed positive Y up, z negative out. You'll need to include the following #defines before it works correctly (or physically change your calculations to accommodate this).
#define GLM_FORCE_DEPTH_ZERO_TO_ONE
#define GLM_FORCE_LEFT_HANDED
Additionally you'll need to modify your matrices to account for the negative Y direction. Here is an example of how I've handled this in the past (modifying the perspective matrix directly):
ubo.model = glm::translate(glm::mat4(1.0f), glm::vec3(pos_x,pos_y,pos_z));
ubo.model *= glm::rotate(glm::mat4(1.0f), time * glm::radians(0.0f), glm::vec3(0.0f, 0.0f, 1.0f));
ubo.view = glm::lookAt(glm::vec3(0.0f, 0.0f, -10.0f), glm::vec3(0.0f, 0.0f, 0.0f), glm::vec3(0.0f, 1.0f, 0.0f));
ubo.proj = glm::perspective(glm::radians(45.0f), swapChainExtent.width / (float) swapChainExtent.height, 0.1f, 100.0f);
ubo.proj[1][1] *= -1; // makes the y axis projected to the same as vulkans
The setup
Draw XY-coordinate axes on a piece of paper. Write a word on it along X-axis, so that the word's centerpoint is at origo (half on positive side of X/Y, the other half on negative side of X/Y).
Now, if you flip the paper upside down you'll notice that the word is mirrored in relation to both X- and Y-axis. If you look from behind the paper, it's mirrored in relation to Y-axis. If you look at it from behind and upside down, it's mirrored in relation to X-axis.
Ok, I have points in 2D-plane (vertices) that are created in similar way at the origo and I need to apply exactly the same rule for them. To make things interesting:
The 2D plane is actually 3D, each point (vertex) being (x, y, 0). Initially the vertices are positioned to the origo and their normal is Pn(0,0,1). => Correctly seen when looked at from point Pn towards origo.
The vertex-plane has it's own rotation matrix [Rp] and position P(x,y,z) in the 3D-world. The rotation is applied before positioning.
The 3D world is "right handed". The viewer would be looking towards origo from some distance along positive Z-axis but the world is also oriented by rotation matrix [Rw]. [Rw] * (0,0,1) would point directly to the viewer's eye.
From those I need to calculate when the vertex-plane should be mirrored and by which axis. The mirroring itself can be done before applying [Rp] and P by:
Vertices vertices = Get2DPlanePoints();
int MirrorX = 1; // -1 to mirror, 1 NOT to mirror
int MirrorY = 1; // -1 to mirror, 1 NOT to mirror
Matrix WorldRotation = GetWorldRotationMatrix();
MirrorX = GetMirrorXFactor(WorldRotation);
MirrorY = GetMirrorYFactor(WorldRotation);
foreach(Vertex v in vertices)
{
v.X = v.X * MirrorX * MirrorY;
v.Y = V.Y * MirrorY;
}
// Apply rotation...
// Add position...
The question
So I need GetMirrorXFactor() & ..YFactor() -functions that return -1 if the viewer's eyepoint is at greater "X/Y"-angle than +-90 degrees in relation to the vertex-plane's normal after the rotation and world orientation. I have already solved this, but I'm looking for more "elegant" mathematics. I know that rotation matrices somehow contain info about how much is rotated by which axis and I believe that can be utilized here.
My Solution for MirrorX:
// Matrix multiplications. Vectors are vertical matrices here.
Pnr = [Rp] * Pn // Rotated vertices's normal
Pur = [Rp] * (0,1,0) // Rotated vertices's "up-vector"
Wnr = [Rw] * (0,0,1) // Rotated eye-vector with world's orientation
// = vector pointing directly at the viewer's eye
// Use rotated up-vector as a normal some new plane and project viewer's
// eye on it. dot = dot product between vectors.
Wnrx = Wnr - (Wnr dot Pur) * Pur // "X-projected" eye.
// Calculate angle between eye's X-component and plane's rotated normal.
// ||V|| = V's norm.
angle = arccos( (Wnrx dot Pnr) / ( ||Wnrx|| * ||Pnr|| ) )
if (angle > PI / 2)
MirrorX = -1; // DO mirror
else
MirrorX = 1; // DON'T mirror
Solution for mirrorY can be done in similar way using viewer's up and vertex-plane's right -vectors.
Better solution?
if (([Rp]*(1,0,0)) dot ([Rw]*(1,0,0))) < 0
MirrorX = -1; // DO mirror
else
MirrorX = 1; // DON'T mirror
if (([Rp]*(0,1,0)) dot ([Rw]*(0,1,0))) < 0
MirrorY = -1; // DO mirror
else
MirrorY = 1; // DON'T mirror
Explaining in more detail is difficult without diagrams, but if you have trouble with this solution we can work through some cases.
I am writing a volume render program that constantly adjusts some plane geometry so it always faces the camera. The plane geometry rotates whenever the camera rotates in order to appear as if it doesn't move--relative to everything else in the scene. (I use the camera's viewing direction as a normal vector to these plane geometries.)
Currently I am manually storing a custom rotation vector ('rotations') and applying its affects as follows in the render function:
gl2.glRotated(rotations.y, 1.0, 0.0, 0.0);
gl2.glRotated(rotations.x, 0.0, 1.0, 0.0);
Then later on I get the viewing direction by rotating the initial view direction (0,0,-1) around the x and y axes with the values from rotation. This is done in the following manner. The final viewing direction is stored in 'view':
public Vec3f getViewingAngle(){
//first rotate the viewing POINT
//then find the vector from there to the center
Vec3f view=new Vec3f(0,0,-1);
float newZ=0;
float ratio=(float) (Math.PI/180);
float vA=(float) (-1f*rotations.y*(ratio));
float hA=(float) (-1f*rotations.x)*ratio;
//rotate about the x axis first
float newY=(float) (view.y*Math.cos(vA)-view.z*Math.sin(vA));
newZ=(float) (view.y*Math.sin(vA)+view.z*Math.cos(vA));
view=new Vec3f(view.x,newY,newZ);
//rotate about Y axis
float newX=(float) (view.z*Math.sin(hA)+view.x*Math.cos(hA));
newZ=(float) (view.z*Math.cos(hA)-view.x*Math.sin(hA));
view=new Vec3f(newX,view.y,newZ);
view=new Vec3f(view.x*-1f,view.y*-1f,view.z*-1f);
//return the finalized normal viewing direction
view=Vec3f.normalized(view);
return view;
}
Now I am moving this program to a larger project wherein the camera rotation is handled by a 3rd party graphics library. I have no rotations vector. Is there some way I can get my view direction vector from:
GLfloat matrix[16];
glGetFloatv (GL_MODELVIEW_MATRIX, matrix);
I am looking at this for reference http://3dengine.org/Modelview_matrix but I still don't get how to come up with the view direction. Can someone explain to me if it is possible and how it works?
You'll want to look at this picture # http://db-in.com/images/local_vectors.jpg
The Direction-of-Flight ( DOF) is the 3rd row.
GLfloat matrix[16];
glGetFloatv( GL_MODELVIEW_MATRIX, matrix );
float DOF[3];
DOF[0] = matrix[ 2 ]; // x
DOF[1] = matrix[ 6 ]; // y
DOF[2] = matrix[ 10 ]; // z
Reference:
http://blog.db-in.com/cameras-on-opengl-es-2-x/
Instead of trying to follow the modelview matrix, to adjust your volume rasterizer's fragment impostor, you should just adjust the modelview matrix to your needs. OpenGL is not a scene graph, it's a drawing system and you can, and should change things however they suit you best.
Of course if you must embedd the volume rasterization into a larger scene, it may be neccessary to extract certain info from the modelview matrix. The upper left 3×3 submatrix contains the composite rotation of models and view. The 3rd column contains the view rotated Z vector.
This is probably a pretty simple thing but my knowledge of direct x is just not up to par with what I'm trying to achieve.
For the moment I am trying to create a vehicle that moves around on terrain. I am attempting to make the vehicle recognize the terrain by creating a square (4 D3DXVECTOR3 points) around the vehicle who's points each detect the height of the terrain and adjust the vehicle accordingly.
The vehicle is a simple object derived from Microsoft sample code. It has a world matrix, coordinates, rotations etc.
What I am trying to achieve is to make these points move along with the vehicle, turning when it does so they can detect the difference in height. This requires me to update the points each time the vehicle moves but I cannot for the life of me figure out how to get them to rotate properly.
So In summary I am looking for a simple way to rotate a vector about an origin (my vehicles coordinates).
These points are situated near the vehicle wheels so if it worked they would stay there regardless of the vehicles y -axis rotation.
Heres What Ive tryed:
D3DXVECTOR3 vec;
D3DXVec3TransformCoord(&vectorToHoldTransformation,&SquareTopLeftPoint,&matRotationY);
SquareTopLeftPoint = vec;
This resulted in the point spinning madly out of control and leaving the map.
xRot = VehicleCoordinateX + cos(RotationY) * (SquareTopleftX - VehicleCoordinateX) - sin(RotationY) * (SquareTopleftZ - VehicleCoordinateZ);
yRot = VehicleCoordinateZ + sin(RotationY) * (SquareTopleftX - VehicleCoodinateX) + cos(RotationY) * (SquareToplefteZ - VehicleCoordinateZ);
BoxPoint refers to the vector I am attempting to rotate.
Vehicle is of course the origin of rotation
RotationY is the amount it has rotated.
This is the code for 1 of 4 vectors in this square but I assume once I get 1 write the rest are just copy-paste.
No matter what I try the point either does not move or spirals out of control under leaving the map all-together.
Here is a snippet of my object class
class Something
{
public:
float x, y, z;
float speed;
float rx, ry, rz;
float sx, sy, sz;
float width;
float length;
float frameTime;
D3DXVECTOR3 initVecDir;
D3DXVECTOR3 currentVecDir;
D3DXMATRIX matAllRotations;
D3DXMATRIX matRotateX;
D3DXMATRIX matRotateY;
D3DXMATRIX matRotateZ;
D3DXMATRIX matTranslate;
D3DXMATRIX matWorld;
D3DXMATRIX matView;
D3DXMATRIX matProjection;
D3DXMATRIX matWorldViewProjection;
//these points represent a box that is used for collision with terrain.
D3DXVECTOR3 frontLeftBoxPoint;
D3DXVECTOR3 frontRightBoxPoint;
D3DXVECTOR3 backLeftBoxPoint;
D3DXVECTOR3 backRightBoxPoint;
}
I was thinking it might be possible to do this using D3DXVec3TransformCoord
D3DXMatrixTranslation(&matTranslate, origin.x,0,origin.z);
D3DXMatrixRotationY(&matRotateY, ry);
D3DXMatrixTranslation(&matTranslate2,width,0,-length);
matAllRotations = matTranslate * matRotateY * matTranslate2;
D3DXVECTOR3 newCoords;
D3DXVECTOR3 oldCoords = D3DXVECTOR3(x,y,z);
D3DXVec3TransformCoord(&newCoords, &oldCoords, &matAllRotations);
Turns out that what I need to do was
Translate by -origin.
rotate
Translate by origin.
What I was doing was
Move to origin
Rotate
Translate by length/width
Thought it was the same.
D3DXMATRIX matTranslate2;
D3DXMatrixTranslation(&matTranslate,-origin.x,0,-origin.z);
D3DXMatrixRotationY(&matRotateY,ry);
D3DXMatrixTranslation(&matTranslate2,origin.x,0,origin.z);
//D3DXMatrixRotationAxis(&matRotateAxis,&origin,ry);
D3DXMATRIX matAll = matTranslate * matRotateY * matTranslate2;
D3DXVECTOR4 newCoords;
D3DXVECTOR4 oldCoords = D3DXVECTOR4(x,y,z,1);
D3DXVec4Transform(&newCoords,&oldCoords,&matAll);
//D3DXVec4TransformCoord(&newCoords, &oldCoords, &matAll);
return newCoords;
Without knowing more about your code I can't say what it does exactly, however one 'easy' way to think about this problem if you know the angle of the heading of your vehicle in world coordinates is to represent your points in a manner such that the center of the vehicle is at the origin, use a simple rotation matrix to rotate it around the vehicle according to the heading, and then add your vehicle's center to the resulting coordinates.
x = vehicle_center_x + cos(heading) * corner_x - sin(heading) * corner_y
y = vehicle_center_y - sin(heading) * corner_x + cos(heading) * corner_y
Keep in mind that corner_x and corner_y are expressed in coordinates relative to the vehicle -- NOT relative to the world.
I have a 2D Point (x,y) and I want to project it to a Vector, so that I can perform a ray-trace to check if the user clicked on a 3D Object, I have written all the other code, Except when I got back to my function to get the Vector from the xy cords of the mouse, I was not accounting for Field-Of-View, and I don't want to guess what the factor would be, as 'voodoo' fixes are not a good idea for a library. any math-magicians wanna help? :-).
Heres my current code, that needs FOV of the camera applied:
sf::Vector3<float> Camera::Get3DVector(int Posx, int Posy, sf::Vector2<int> ScreenSize){
//not using a "wide lens", and will maintain the aspect ratio of the viewport
int window_x = Posx - ScreenSize.x/2;
int window_y = (ScreenSize.y - Posy) - ScreenSize.y/2;
float Ray_x = float(window_x)/float(ScreenSize.x/2);
float Ray_y = float(window_y)/float(ScreenSize.y/2);
sf::Vector3<float> Vector(Ray_x,Ray_y, -_zNear);
// to global cords
return MultiplyByMatrix((Vector/LengthOfVector(Vector)), _XMatrix, _YMatrix, _ZMatrix);
}
You're not too fart off, one thing is to make sure your mouse is in -1 to 1 space (not 0 to 1)
Then you create 2 vectors:
Vector3 orig = Vector3(mouse.X,mouse.Y,0.0f);
Vector3 far = Vector3(mouse.X,mouse.Y,1.0f);
You also need to use the inverse of your perspective tranform (or viewprojection if you want world space)
Matrix ivp = Matrix::Invert(Projection)
Then you do:
Vector3 rayorigin = Vector3::TransformCoordinate(orig,ivp);
Vector3 rayfar = Vector3::TransformCoordinate(far,ivp);
If you want a ray, you also need direction, which is simply:
Vector3 raydir = Normalize(rayfar-rayorigin);