I'm having trouble converting pixel coordinates from the mouse to a 3D frustum. I'm using code similar to glProject() on OGL ES 1.1 . I have tried using glUnProject() but I couldn't get the vector to work and I know there is an easier way to do this.
I was hoping to be able to compare the 3D and 2D coordinates and figure it out but I have not succeeded. So here's what I know:
I am using a 3D Vertex from the polygon that is being picked:
-1.0,1.0,0.0
Then I convert it to pixels coordinates with glProject():
140.0, 259.0, 0.0
I then use the mouse pixel coordinates:
140.0, 220.0, 0.0
This is the part I can't figure out:
-1.0, -1.0, 0.0
I got the #3 coordinates from the coordinates of #4 but what I want to do is the opposite which is convert pixels to 3D.
All I really need to know is how far the mouse has been dragged in 3D coordinates from another 3D point.
Okay I got an approximate algorithm working. The way I got it working was by using:
glFrustum(ax,bx,ay,by,az,bz);
I used the ratio of ax and az multiplied by the length of z to find the length of the tx and ty sides of the triangle. Then I used the pixel coordinates to find x and y. I'm not sure if gluUnProject() is any better but the one I used is a lot smaller and easier to understand.
Related
My problem is quite simple yet I struggle to solve it correctly.
I have a camera looking towards the ground and I know all the parameters of the shot. So, using some maths I was able to compute the 4 points defining the field of view of the camera (the coordinates on the ground of each image's corners).
Now, from the coordinates (x, y) of a pixel of the image, I would like to know its real coordinates projected on the ground.
I thought that homography was the way to go, but I read here and there that "homography maps a plane seen from a camera to the same plane seen from another" which is a slightly different problem.
What should I use, please?
Edit: Here is an example.
Given this image:
I know everything about the camera that took the picture (height, angles of view, orientation), so I could calculate the coordinates of the four corners forming its field of view on the ground, for example (in centimeters, relative to the camera position, clockwise from top-left): (-300, 500), (300, 500), (100, 50), (-100, 50).
Knowing that the coordinates on the image of the blade of grass are (1750, 480), how can I know its actual coordinates on the ground?
By "knowing everything" about the camera, do you mean you have the camera FOV, rotation and translation with respect to the ground plane? Then it's trivial, right?
Write the camera matrix K = [[f, 0, w/2],[0, f, h/2],[0, 0, 1]]. Let R and t be respectively the 3x3 rotation matrix and 3x1 translation from camera to ground. A point on the ray going through a given pixel p=[u, v, 1] has camera coordinates r = inv(K) * p. Express it in world coordinates as R * r + t, intersect with the ground plane and you are done.
I have four points in a 3D space, example:
(0,0,1)
(1,0,1)
(1,0,2)
(0,0,2)
Then I have a 2D position on that square plane:
x = 0.5
y = 0.5
I need to find out the 3D space point of that position in the plane. In this example it's easy: (0.5,0,1.5), because Y is zero. But imagine that Y was not zero (and not all the same), that the plane is leaning in some direction. How would I calculate the point in that case?
I imagine this should be a pretty easy thing to solve, but I can't figure it out. Please answer in programming terms and not in straight math terms, if possible.
Update with image: The gray plane (made out of two triangles) are the real one actually existing. I create a non-existing plane on top of this, the ABCD corners are exactly the same, however it doesn't slope. What I need to do is project a pixel (blue one in example) from the non-existing plane to the existing plane. It will be in the exact same location, except that it has gained a Y value from the sloping plane.
(couldn't actually make the image appear because i need 10 reputation to show it, wtf?)
What I've been able to work out so far on my own is which one of the two triangles to use in the gray plane and the normal of triangle. I basically just need to figure out how I can project the pixel.
Figured it out mostly thanks to http://gamedeveloperjourney.blogspot.com/2009/04/point-plane-collision-detection.html
Made me realize I had to verify the normal a bit closer, turns out my plane's grid was being rendered a little different than the actual coordinates for the verticles. No wonder this was so hard to get right! The pixel was projected correctly but rendered incorrectly.
To view my 3D environment, I use the "true" 3D isometric projection (flat square on XZ plane, Y is "always" 0). I used the explanation on wikipedia: http://en.wikipedia.org/wiki/Isometric_projection to come to how to do this transformation:
The projection matrix is an orthographic projection matrix between some minimum and maximum coordinate.
The view matrix is two rotations: one around the Y-axis (n * 45 degrees) and one around the X-axis (arctan(sin(45 degrees))).
The result looks ok, so I think I have done it correctly.
But now I want to be able to pick a coordinate with the mouse. I have successfully implemented this by rendering coordinates to an invisible framebuffer and then getting the pixel under the mouse cursor to get the coordinate. Although this works fine, I would really like to see a mathematical sollution because I will need it to calculate bounding boxes, frustums of the area on the screen and stuff like that.
My instincts tell me to:
- go from screen-coordinates to 2D projection coordinates (or how do you say this, I mean transforming screen coordinates to a coordinate between -1 and +1 for both axisses, with y inverted)
- untransform the coordinate with the inverse of the view-matrix.
- yeah... untransform this coordinate with the inverse of the projection matrix, but as my instincts tell, this won't work as everything will have the same Z-coordinate.
This, while every information is perfectly available on the isometric view (I know that the Y value is always 0). So I should be able to convert the isometric 2D x,y coordinate to a calculated 3d (x, 0, z) coordinate without using scans or something like that.
My math isn't bad, but this is something I can't seem to grasp.
Edit: IMO. every different (x, 0, z) coordinate corresponds to a different (x2, y2) coordinate in isometric view. So I should be able to simply calculate a way from (x2, y2) to (x, 0, z). But how?
Anyone?
there is something called project and unproject to transform screen to world and vice versa....
You seem to miss some core concepts here (it’s been a while since I did this stuff, so minor errors included):
There are 3 kinds of coordinates involved here (there are more, these are the relevant ones): Scene, Projection and Window
Scene (3D) are the coordinates in your world
Projection (3D) are those coordinates after being transformed by camera position and projection
Window (2D) are the coordinates in your window. They are generated from projection by scaling x and y appropriately and discarding z (z is still used for “who’s in front?” calculations)
You can not transform from window to scene with a matrix, as every point in window does correspond to a whole line in scene. If you want (x, 0, z) coordinates, you can generate this line and intersect it with the y-plane.
If you want to do this by hand, generate two points in projection with the same (x,y) and different (arbitrary) z coordinates and transform them to scene by multiplying with the inverse of your projection transformation. Now intersect the line through those two points with your y-plane and you’re done.
Note that there should be a “static” solution (a single formula) to this problem – if you solve this all on paper, you should get to it.
It's been a while since my math in university, and now I've come to need it like I never thought i would.
So, this is what I want to achieve:
Having a set of 3D points (geographical points, latitude and longitude, altitude doesn't matter), I want to display them on a screen, considering the direction I want to take into account.
This is going to be used along with a camera and a compass , so when I point the camera to the North, I want to display on my computer the points that the camera should "see". It's a kind of Augmented Reality.
Basically what (i think) i need is a way of transforming the 3D points viewed from above (like viewing the points on google maps) into a set of 3d Points viewed from a side.
The conversion of Latitude and longitude to 3-D cartesian (x,y,z) coordinates can be accomplished with the following (Java) code snippet. Hopefully it's easily converted to your language of choice. lat and lng are initially the latitude and longitude in degrees:
lat*=Math.PI/180.0;
lng*=Math.PI/180.0;
z=Math.sin(-lat);
x=Math.cos(lat)*Math.sin(-lng);
y=Math.cos(lat)*Math.cos(-lng);
The vector (x,y,z) will always lie on a sphere of radius 1 (i.e. the Earth's radius has been scaled to 1).
From there, a 3D perspective projection is required to convert the (x,y,z) into (X,Y) screen coordinates, given a camera position and angle. See, for example, http://en.wikipedia.org/wiki/3D_projection
It really depends on the degree of precision you require. If you're working on a high-precision, close-in view of points anywhere on the globe you will need to take the ellipsoidal shape of the earth into account. This is usually done using an algorithm similar to the one descibed here, on page 38 under 'Conversion between Geographical and Cartesian Coordinates':
http://www.icsm.gov.au/gda/gdatm/gdav2.3.pdf
If you don't need high precision the techniques mentioned above work just fine.
could anyone explain me exactly what these params mean ?
I've tried and the results where very weird so i guess i am missunderstanding some of the params for the perspective projection
* {a}_{x,y,z} - the point in 3D space that is to be projected.
* {c}_{x,y,z} - the location of the camera.
* {\theta}_{x,y,z} - The rotation of the camera. When {c}_{x,y,z}=<0,0,0>, and {\theta}_{x,y,z}=<0,0,0>, the 3D vector <1,2,0> is projected to the 2D vector <1,2>.
* {e}_{x,y,z} - the viewer's position relative to the display surface. [1]
Well, you'll want some 3D vector arithmetic to move your origin, and probably some quaternion-based rotation functions to rotate the vectors to match your direction. There are any number of good tutorials on using quaternions to rotate 3D vectors (since they're used a lot for rendering and such), and the 3D vector stuff is pretty simple if you can remember how vectors are represented.
well, just a pice ov advice, you can plot this points into a 3d space (you can do easily this using openGL).
You have to transforrm the lat/long into another system for example polar or cartesian.
So starting from lat/longyou put the origin of your space into the center of the heart, than you have to transform your data in cartesian coord:
z= R * sin(long)
x= R * cos(long) * sin(lat)
y= R * cos(long) * cos(lat)
R is the radius of the world, you can put it at 1 if you need only to cath the direction between yoour point of view anthe points you need "to see"
than put the Virtual camera in a point of the space you've created, and link data from your real camera (simply a vector) to the data of the virtual one.
The next stemp to gain what you want to do is to try to plot timages for your camera overlapped with your "virtual space", definitevly you should have a real camera that is a control to move the virtual one in a virtual space.
I am doing a behind the curtains 3d simulation while rendering the world in my 2d isometric engine. I've never done an isometric engine before, and my matrix math is rusty in general so I am having problems.
I have a projection matrix, which in its simplest form is this:
0.7 0.35 0
0 -0.87 0
-0.71 0.35 1
A couple of signs are flipped because my engines coordinate system is 0,0 in the top left, with +X to the right/east and +Z to the south.
Now, the inverse of that is:
1.4080 0.5670 0.0000
0.0000 -1.1490 0.0000
1.0000 0.8050 1.0000
Now, these matrices mostly work.
For instance
WC: 500,0,500 = Screen: -1.44, 350, 500 (X and Y are correct)
WC: 0,0,500 = Screen: -355, 175, 500 (X and Y are correct again)
But, now if you need to go the other way, you no longer have that handy Z value, so
Screen: -1.44, 350, 0 = WC: -2, -402.97, 0 (So, garbage.)
And lots more - as soon as I no longer have that Z value, I can't retrieve the world coords from the screen coords.
What's the workaround here?
EDIT
I should point out that the point of the unproject is to get a ray for mouse picking..
It seems like it's just my misperception of what I was doing that was screwing me up here.
As you discovered, your conversion back into 3D space requires some kind of Z coordinate to make any sense at all.
I would suggest that you do the reverse transformation twice. Once with a Z coordinate near the screen (closest to the observer), and once with a Z coordinate at the back of your 3D scene. These two 3D points would give you a 3D line, which would occupy all of the positions "behind" that 2D point.
you can't. you are projecting onto the screen which loses information.
If you think about it, several 3d coordinates get projected onto the same point on the screen, and just knowing that screen coordinate isn't enough to retrieve the original coordinate.
[edit]
looking at your screen coordinates, you give them all z-value 0. which means the last columns of your projection matrix should have all zeros, making that matrix non-invertible.
Every pixel on the screen represents a line from the eye of the beholder into the imaginary 3D world behind the screen. You have to intersect this line with whatever objects may lurk in that world in order to get 3D coordinates.