Implementing Ray Picking - math

I have a renderer using directx and openGL, and a 3d scene. The viewport and the window are of the same dimensions.
How do I implement picking given mouse coordinates x and y in a platform independent way?

If you can, do the picking on the CPU by calculating a ray from the eye through the mouse pointer and intersect it with your models.
If this isn't an option I would go with some type of ID rendering. Assign each object you want to pick a unique color, render the objects with these colors and finally read out the color from the framebuffer under the mouse pointer.
EDIT: If the question is how to construct the ray from the mouse coordinates you need the following: a projection matrix P and the camera transform C. If the coordinates of the mouse pointer is (x, y) and the size of the viewport is (width, height) one position in clip space along the ray is:
mouse_clip = [
float(x) * 2 / float(width) - 1,
1 - float(y) * 2 / float(height),
0,
1]
(Notice that I flipped the y-axis since often the origin of the mouse coordinates are in the upper left corner)
The following is also true:
mouse_clip = P * C * mouse_worldspace
Which gives:
mouse_worldspace = inverse(C) * inverse(P) * mouse_clip
We now have:
p = C.position(); //origin of camera in worldspace
n = normalize(mouse_worldspace - p); //unit vector from p through mouse pos in worldspace

Here's the viewing frustum:
First you need to determine where on the nearplane the mouse click happened:
rescale the window coordinates (0..640,0..480) to [-1,1], with (-1,-1) at the bottom-left corner and (1,1) at the top-right.
'undo' the projection by multiplying the scaled coordinates by what I call the 'unview' matrix: unview = (P * M).inverse() = M.inverse() * P.inverse(), where M is the ModelView matrix and P is the projection matrix.
Then determine where the camera is in worldspace, and draw a ray starting at the camera and passing through the point you found on the nearplane.
The camera is at M.inverse().col(4), i.e. the final column of the inverse ModelView matrix.
Final pseudocode:
normalised_x = 2 * mouse_x / win_width - 1
normalised_y = 1 - 2 * mouse_y / win_height
// note the y pos is inverted, so +y is at the top of the screen
unviewMat = (projectionMat * modelViewMat).inverse()
near_point = unviewMat * Vec(normalised_x, normalised_y, 0, 1)
camera_pos = ray_origin = modelViewMat.inverse().col(4)
ray_dir = near_point - camera_pos

Well, pretty simple, the theory behind this is always the same
1) Unproject two times your 2D coordinate onto the 3D space. (each API has its own function, but you can implement your own if you want). One at Min Z, one at Max Z.
2) With these two values calculate the vector that goes from Min Z and point to Max Z.
3) With the vector and a point calculate the ray that goes from Min Z to MaxZ
4) Now you have a ray, with this you can do a ray-triangle/ray-plane/ray-something intersection and get your result...

I have little DirectX experience, but I'm sure it's similar to OpenGL. What you want is the gluUnproject call.
Assuming you have a valid Z buffer you can query the contents of the Z buffer at a mouse position with:
// obtain the viewport, modelview matrix and projection matrix
// you may keep the viewport and projection matrices throughout the program if you don't change them
GLint viewport[4];
GLdouble modelview[16];
GLdouble projection[16];
glGetIntegerv(GL_VIEWPORT, viewport);
glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
glGetDoublev(GL_PROJECTION_MATRIX, projection);
// obtain the Z position (not world coordinates but in range 0 - 1)
GLfloat z_cursor;
glReadPixels(x_cursor, y_cursor, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &z_cursor);
// obtain the world coordinates
GLdouble x, y, z;
gluUnProject(x_cursor, y_cursor, z_cursor, modelview, projection, viewport, &x, &y, &z);
if you don't want to use glu you can also implement the gluUnProject you could also implement it yourself, it's functionality is relatively simple and is described at opengl.org

Ok, this topic is old but it was the best I found on the topic, and it helped me a bit, so I'll post here for those who are are following ;-)
This is the way I got it to work without having to compute the inverse of Projection matrix:
void Application::leftButtonPress(u32 x, u32 y){
GL::Viewport vp = GL::getViewport(); // just a call to glGet GL_VIEWPORT
vec3f p = vec3f::from(
((float)(vp.width - x) / (float)vp.width),
((float)y / (float)vp.height),
1.);
// alternatively vec3f p = vec3f::from(
// ((float)x / (float)vp.width),
// ((float)(vp.height - y) / (float)vp.height),
// 1.);
p *= vec3f::from(APP_FRUSTUM_WIDTH, APP_FRUSTUM_HEIGHT, 1.);
p += vec3f::from(APP_FRUSTUM_LEFT, APP_FRUSTUM_BOTTOM, 0.);
// now p elements are in (-1, 1)
vec3f near = p * vec3f::from(APP_FRUSTUM_NEAR);
vec3f far = p * vec3f::from(APP_FRUSTUM_FAR);
// ray in world coordinates
Ray ray = { _camera->getPos(), -(_camera->getBasis() * (far - near).normalize()) };
_ray->set(ray.origin, ray.dir, 10000.); // this is a debugging vertex array to see the Ray on screen
Node* node = _scene->collide(ray, Transform());
cout << "node is : " << node << endl;
}
This assumes a perspective projection, but the question never arises for the orthographic one in the first place.

I've got the same situation with ordinary ray picking, but something is wrong. I've performed the unproject operation the proper way, but it just doesn't work. I think, I've made some mistake, but can't figure out where. My matix multiplication , inverse and vector by matix multiplications all seen to work fine, I've tested them.
In my code I'm reacting on WM_LBUTTONDOWN. So lParam returns [Y][X] coordinates as 2 words in a dword. I extract them, then convert to normalized space, I've checked this part also works fine. When I click the lower left corner - I'm getting close values to -1 -1 and good values for all 3 other corners. I'm then using linepoins.vtx array for debug and It's not even close to reality.
unsigned int x_coord=lParam&0x0000ffff; //X RAW COORD
unsigned int y_coord=client_area.bottom-(lParam>>16); //Y RAW COORD
double xn=((double)x_coord/client_area.right)*2-1; //X [-1 +1]
double yn=1-((double)y_coord/client_area.bottom)*2;//Y [-1 +1]
_declspec(align(16))gl_vec4 pt_eye(xn,yn,0.0,1.0);
gl_mat4 view_matrix_inversed;
gl_mat4 projection_matrix_inversed;
cam.matrixProjection.inverse(&projection_matrix_inversed);
cam.matrixView.inverse(&view_matrix_inversed);
gl_mat4::vec4_multiply_by_matrix4(&pt_eye,&projection_matrix_inversed);
gl_mat4::vec4_multiply_by_matrix4(&pt_eye,&view_matrix_inversed);
line_points.vtx[line_points.count*4]=pt_eye.x-cam.pos.x;
line_points.vtx[line_points.count*4+1]=pt_eye.y-cam.pos.y;
line_points.vtx[line_points.count*4+2]=pt_eye.z-cam.pos.z;
line_points.vtx[line_points.count*4+3]=1.0;

Related

Find angle between two points

I am trying to make an image move towards my mouse pointer. Basically, I get the angle between the points, and move along the x axis by the cosine of the angle, and move along the y axis the sine of the angle.
However, I don't have a good way of calculating the angle. I get the difference in x and the difference in y, and use arctangent of Δy/Δx. The resulting angle in quadrant 1 is correct, but the other three quadrants are wrong. Quadrant 2 ranges from -1 to -90 degrees. Quadrant 3 is always equal to quadrant 1, and quadrant 4 always equals quadrant 4. Is there an equation that I can use to find the angle between the two points from 1-360 degrees?
Note: I cannot use atan2(), and I do not know what a vector is.
// This working code is for Windows HDC mouse coordinates gives the angle back that is used in Windows. It assumes point 1 is your origin point
// Tested and working on Visual Studio 2017 using two mouse coordinates in HDC.
//
// Code to call our function.
float angler = get_angle_2points(Point1X, Point1Y, Point2X, Point2Y);
// Takes two window coordinates (points), turns them into vectors using the origin and calculates the angle around the x-axis between them.
// This function can be used for any HDC window. I.e., two mouse points.
float get_angle_2points(int p1x, int p1y, int p2x,int p2y)
{
// Make point1 the origin, and make point2 relative to the origin so we do point1 - point1, and point2-point1,
// Since we don’t need point1 for the equation to work, the equation works correctly with the origin 0,0.
int deltaY = p2y - p1y;
int deltaX = p2x - p1x; // Vector 2 is now relative to origin, the angle is the same, we have just transformed it to use the origin.
float angleInDegrees = atan2(deltaY, deltaX) * 180 / 3.141;
angleInDegrees *= -1; // Y axis is inverted in computer windows, Y goes down, so invert the angle.
//Angle returned as:
// 90
// 135 45
//
// 180 Origin 0
//
//
// -135 -45
//
// -90
// The returned angle can now be used in the C++ window function used in text angle alignment. I.e., plf->lfEscapement = angle*10;
return angleInDegrees;
}
The answers regarding atan2 are correct. For reference, here is atan2 in Scratch block form:
If you're unable to use atan2() directly, you could implement its internal calculations on your own:
atan2(y,x) = atan(y/x) if x>0
atan(y/x) + π if x<0 and y>0
atan(y/x) - π if x<0 and y<0
This is the code I use, and it seems to work perfectly fine.
atan(x/y) + (180*(y<0))
where X is the difference between the Xs of the points (x2 - x1), and Y is the difference between the Ys (y2 - y1).
atan((x2-x1)/(y1-y2)) + (180*((y1-y2)<0))

How to calculate the angles of the projection in 3d for an object to step at given point?

I need to calculate the angles to through the ball in that direction for a given speed and the point where it should land after thrown.
The horizontal angle is easy(We know both start and step points).How to calculate the vertical angle of projection.There is gravy applying on object.
Time of travel will be usual as bowling time(time between ball release and occurring step) as per video.
Is there a way directly in unity3d?
Watch this video for 8 seconds for clear understating of this question.
According to the Wikipedia page Trajectory of a projectile, the "Angle of reach" (The angle you want to know) is calculated as follows:
θ = 1/2 * arcsin(gd/v²)
In this formula, g is the gravitational constant 9.81, d is the distance you want the projectile to travel, and v is the velocity at which the object is thrown.
Code to calculate this could look something like this:
float ThrowAngle(Vector3 destination, float velocity)
{
const float g = 9.81f;
float distance = Vector3.Distance(transform.position, destination);
//assuming you want degrees, otherwise just drop the Rad2Deg.
return Mathf.Rad2Deg * (0.5f * Asin((g*distance)/Mathf.Pow(velocity, 2f)));
}
This will give you the angle assuming no air resistance etc. exist in your game.
If your destination and your "throwing point" are not at the same height, you may want to set both to y=0 first, otherwise, errors may occur.
EDIT:
Considering that your launch point is higher up than the destination, this formula from the same page should work:
θ = arctan(v² (+/-) √(v^4-g(gx² + 2yv²))/gx)
Here, x is the range, or distance, and y is the altitude (relative to the launch point).
Code:
float ThrowAngle(Vector3 start, Vector3 destination, float v)
{
const float g = 9.81f;
float xzd = Mathf.Sqrt(Mathf.Pow(destination.x - start.x, 2) + Mathf.Pow(destination.z - start.z, 2));
float yd = destination.y - start.y;
//assuming you want degrees, otherwise just drop the Rad2Deg. Split into two lines for better readability.
float sqrt = (Mathf.Pow(v,4) - g * (g*Mathf.Pow(xzd,2) + 2*yd*Mathf.Pow(v,2))/g*xzd);
//you could also implement a solution which uses both values in some way, but I left that out for simplicity.
return Mathf.Atan(Mathf.Pow(v, 2) + sqrt);
}

Drawing a rotating sphere by using a pixel shader in Direct3D

I would like to draw a textured circle in Direct3D which looks like a real 3D sphere. For this purpose, I took a texture of a billard ball and tried to write a pixel shader in HLSL, which maps it onto a simple pre-transformed quad in such a way that it looks like a 3-dimensional sphere (apart from the lighting, of course).
This is what I've got so far:
struct PS_INPUT
{
float2 Texture : TEXCOORD0;
};
struct PS_OUTPUT
{
float4 Color : COLOR0;
};
sampler2D Tex0;
// main function
PS_OUTPUT ps_main( PS_INPUT In )
{
// default color for points outside the sphere (alpha=0, i.e. invisible)
PS_OUTPUT Out;
Out.Color = float4(0, 0, 0, 0);
float pi = acos(-1);
// map texel coordinates to [-1, 1]
float x = 2.0 * (In.Texture.x - 0.5);
float y = 2.0 * (In.Texture.y - 0.5);
float r = sqrt(x * x + y * y);
// if the texel is not inside the sphere
if(r > 1.0f)
return Out;
// 3D position on the front half of the sphere
float p[3] = {x, y, sqrt(1 - x*x + y*y)};
// calculate UV mapping
float u = 0.5 + atan2(p[2], p[0]) / (2.0*pi);
float v = 0.5 - asin(p[1]) / pi;
// do some simple antialiasing
float alpha = saturate((1-r) * 32); // scale by half quad width
Out.Color = tex2D(Tex0, float2(u, v));
Out.Color.a = alpha;
return Out;
}
The texture coordinates of my quad range from 0 to 1, so I first map them to [-1, 1]. After that I followed the formula in this article to calculate the correct texture coordinates for the current point.
At first, the outcome looked ok, but I'd like to be able to rotate this illusion of a sphere arbitrarily. So I gradually increased u in the hope of rotating the sphere around the vertical axis. This is the result:
As you can see, the imprint of the ball looks unnaturally deformed when it reaches the edge. Can anyone see any reason for this? And additionally, how could I implement rotations around an arbitrary axis?
Thanks in advance!
I finally found the mistake by myself: The calculation of the z value which corresponds to the current point (x, y) on the front half of the sphere was wrong. It must of course be:
That's all, it works as exspected now. Furthermore, I figured out how to rotate the sphere. You just have to rotate the point p before calculating u and v by multiplying it with a 3D rotation matrix like this one for example.
The result looks like the following:
If anyone has any advice as to how I could smooth the texture a litte bit, please leave a comment.

OpenGL Math - Projecting Screen space to World space coords

Time for a little bit of math for the end of the day..
I need to project 4 points of the window size:
<0,0> <1024,768>
Into a world space coordinates so it will form a quadrilateral shape that will later be used for terrain culling - without GluUnproject
For test only, I use mouse coordinates - and try to project them onto the world coords
RESOLVED
Here's how to do it exactly, step by step.
Obtain your mouse coordinates within the client area
Get your Projection matrix and View matrix if no Model matrix required.
Multiply Projection * View
Inverse the results of multiplication
Construct a vector4 consisting of
x = mouseposition.x within a range of window x
transform to values between -1 and 1
y = mouseposition.y within a range of window y
transform to values between -1 and 1
remember to invert mouseposition.y if needed
z = the depth value ( this can be obtained with glReadPixel)
you can manually go from -1 to 1 ( zNear, zFar )
w = 1.0
Multiply the vector by inversed matrix created before
Divide result vector by it's w component after matrix multiplication ( perspective division )
POINT mousePos;
GetCursorPos(&mousePos);
ScreenToClient( this->GetWindowHWND(), &mousePos );
CMatrix4x4 matProjection = m_pCamera->getViewMatrix() * m_pCamera->getProjectionMatrix() ;
CMatrix4x4 matInverse = matProjection.inverse();
float in[4];
float winZ = 1.0;
in[0]=(2.0f*((float)(mousePos.x-0)/(this->GetResolution().x-0)))-1.0f,
in[1]=1.0f-(2.0f*((float)(mousePos.y-0)/(this->GetResolution().y-0)));
in[2]=2.0* winZ -1.0;
in[3]=1.0;
CVector4 vIn = CVector4(in[0],in[1],in[2],in[3]);
pos = vIn * matInverse;
pos.w = 1.0 / pos.w;
pos.x *= pos.w;
pos.y *= pos.w;
pos.z *= pos.w;
sprintf(strTitle,"%f %f %f / %f,%f,%f ",m_pCamera->m_vPosition.x,m_pCamera->m_vPosition.y,m_pCamera->m_vPosition.z,pos.x,pos.y,pos.z);
SetWindowText(this->GetWindowHWND(),strTitle);
I had to make some adjustments to the answers provided here. But here's the code I ended up with (Note I'm using GLM, that could affect multiplication order). nearResult is the projected point on the near plane and farResult is the projected point on the far plane. I want to perform a ray cast to see what my mouse is hovering over so I convert them to a direction vector which will then originate from my camera's position.
vec3 getRayFromScreenSpace(const vec2 & pos)
{
mat4 invMat= inverse(m_glData.getPerspective()*m_glData.getView());
vec4 near = vec4((pos.x - Constants::m_halfScreenWidth) / Constants::m_halfScreenWidth, -1*(pos.y - Constants::m_halfScreenHeight) / Constants::m_halfScreenHeight, -1, 1.0);
vec4 far = vec4((pos.x - Constants::m_halfScreenWidth) / Constants::m_halfScreenWidth, -1*(pos.y - Constants::m_halfScreenHeight) / Constants::m_halfScreenHeight, 1, 1.0);
vec4 nearResult = invMat*near;
vec4 farResult = invMat*far;
nearResult /= nearResult.w;
farResult /= farResult.w;
vec3 dir = vec3(farResult - nearResult );
return normalize(dir);
}
Multiply all your matrices. Then invert the result. Point after projection are always in the -1,1. So the four corner screen points are -1,-1; -1,1; 1,-1;1,1. But you still need to choose th z value. If you are in OpenGL, z is between -1 and 1. For directx, the range is 0 to 1. Finally take your points and transform them with the matrix
If you have access to the glu libraries, use gluUnProject(winX, winY, winZ, model, projection, viewport, &objX, &objY, &objZ);
winX and winY will be the corners of your screen in pixels. winZ is a number in [0,1] which will specify where between zNear and zFar (clipping planes) the points should fall. objX-Z will hold the results. The middle variables are the relevant matrices. They can be queried if needed.

Calculate Angle from Two Points and a Direction Vector

I have two vectors in a game. One vector is the player, one vector is an object. I also have a vector that specifies the direction the player if facing. The direction vector has no z part. It is a point that has a magnitude of 1 placed somewhere around the origin.
I want to calculate the angle between the direction the soldier is currently facing and the object, so I can correctly pan some audio (stereo only).
The diagram below describes my problem. I want to calculate the angle between the two dashed lines. One dashed line connects the player and the object, and the other is a line representing the direction the player is facing from the point the player is at.
At the moment, I am doing this (assume player, object and direction are all vectors with 3 points, x, y and z):
Vector3d v1 = direction;
Vector3d v2 = object - player;
v1.normalise();
v2.normalise();
float angle = acos(dotProduct(v1, v2));
But it seems to give me incorrect results. Any advice?
Test of code:
Vector3d soldier = Vector3d(1.f, 1.f, 0.f);
Vector3d object = Vector3d(1.f, -1.f, 0.f);
Vector3d dir = Vector3d(1.f, 0.f, 0.f);
Vector3d v1 = dir;
Vector3d v2 = object - soldier;
long steps = 360;
for (long step = 0; step < steps; step++) {
float rad = (float)step * (M_PI / 180.f);
v1.x = cosf(rad);
v1.y = sinf(rad);
v1.normalise();
float dx = dotProduct(v2, v1);
float dy = dotProduct(v2, soldier);
float vangle = atan2(dx, dy);
}
You shoud always use atan2 when computing angular deltas, and then normalize.
The reason is that for example acos is a function with domain -1...1; even normalizing if the input absolute value (because of approximations) gets bigger than 1 the function will fail even if it's clear that in such a case you would have liked an angle of 0 or PI instead. Also acos cannot measure the full range -PI..PI and you'd need to use explicitly sign tests to find the correct quadrant.
Instead atan2 only singularity is at (0, 0) (where of course it doesn't make sense to compute an angle) and its codomain is the full circle -PI...PI.
Here is an example in C++
// Absolute angle 1
double a1 = atan2(object.y - player.y, object.x - player.x);
// Absolute angle 2
double a2 = atan2(direction.y, direction.x);
// Relative angle
double rel_angle = a1 - a2;
// Normalize to -PI .. +PI
rel_angle -= floor((rel_angle + PI)/(2*PI)) * (2*PI) - PI;
In the case of a general 3d orientation you need two orthogonal directions, e.g. the vector of where the nose is pointing to and the vector to where your right ear is.
In that case the formulas are just slightly more complex, but simpler if you have the dot product handy:
// I'm assuming that '*' is defined as the dot product
// between two vectors: x1*x2 + y1*y2 + z1*z2
double dx = (object - player) * nose_direction;
double dy = (object - player) * right_ear_direction;
double angle = atan2(dx, dy); // Already in -PI ... PI range
In 3D space, you also need to compute the axis:
Vector3d axis = normalise(crossProduct(normalise(v1), normalise(v2)));

Resources