I have extracted a 3D surface from an MRI acquisition and the coordinates of the points describing this surface are (I believe) with respect to the reference system of the first image of the series (I mean that the origin corresponds to the Image Position(Patient) and the axes orientation to the Image Orientation(Patient)).
I have another set of images with a different Image Position(Patient) and a different Image Orientation(Patient); I want to rotate and translate the surface extracted from the first set in order to have it match exactly the second set of images.
I'm having trouble with finding the correct 4x4 matrix that would do the job, once I get it, I know how to apply it to my surface.
Any kind of help would be greatly appreciated, thank you.
Simon
This page explains how to form a transformation matrix from the geometry information in the DICOM headers. These transformation matrices are used to transform from the volume coordinate system (pixel-x, pixel-y, slice number) to the patient/world coordinate system (in millimeters).
The basic idea to transform from volume 1 to volume 2 is to tranform from volume 1 to patient coordinates and from patient coordinates to volume 2 coordinate system. Multiplying both matrices yields the matrix to transform directly from volume 1 to volume 2.
Caution: Obviously, there can be no guarantee that every coordinate in v1 matches a coordinate in v2, i.e. the stacks may have different size and/or position.
So you have:
M1 - the matrix to transform from volume 1 to the world coordinate system and
M2 - the matrix to transform from volume 2 to the world coordinate system
Then
M1 * (M2^(-1)) is the matrix to transform a position vector from volume 1 to volume 2 (input and output is pixel-x, pixel-y, slice number)
and
M2 * (M1^(-1)) is the matrix to transform a position vector from volume 1 to volume 2 (input and output is pixel-x, pixel-y, slice number)
Related
Basically, I want to transform a point from the image of camera 2 (x_pixel_c2, y_pixel_c2) to a pixel point (x_pixel_c2, y_pixel_c3) in camera 3.
For a simple setup (no distortion parameters, no rectification), I would usually:
assign a certain distance to the point in C2
compute the coordinates in 3D space of C2 from the intrinsics
compute the World coordinates from the extrinsic matrix of C2
compute the 3D coordinates in C3 from its extrinsic matrix
project in pixel space of C2
For the Kitty dataset, I do not use this approach because of the distortion parameters (especially). In Kitty, we have the projection matrices for the 4 rectified cameras, which from what I understand, are the transformation matrices relating the 3D coordinates in Camera 0 to Camera X. However, from the tests I've made, this does not work:
pt_3d_cam0 = np.dot(inverse(P_rect_02), pt_cam_2)
pt_cam_3 = np.dot(P_rect_03, pt_3d_cam0)
I'm not sure the projection matrices move the referential back to camera 0 as it is mentioned. For instance, the points are closer to the expected coordinates if I add a translation in X equal to the amount of the baseline.
I'm not sure if I'm missing something. If anyone has encountered a similar problem with Kitty, it would be appreciated if you could help.
Thank you!
I have a 3D oriented point (XYZ + Yaw/Pitch/Roll). Lets say this point is based on "user frame 1" (UF1)(the frame is defined in relation to the world frame (WF)).
I also have another "user frame", lets call it "user frame 2" (UF2) that is also defined in reference to the world frame.
How would I take my 3d oriented point (6 coordinates; 3 location + 3 rotation) from one frame to another ?
For example; Since all user frame are related to the world frame, How would I calculate (transform) my 3d oriented point from UF1 to WF ? Or vice-versa or from any combination ? (UF1->WF, WF->UF2, etc.)
The resulting point must have the 6 coordinates (3 location + 3 rotation) in relation to the destination frame.
PS: I'm mainly working in C#, so if possible code sample or pseudo-code to accomplish this would be appreciated.
What you are looking for is the transition matrix from one coordinate system to an other.
It is a 4x4 matrix fully determined by the 3 parameters of translation and the 6 angles of ratotion between the 2 coordinate system.
The 9 coefficients depend on the rotation angles (Euler angles or quaternion depending on what you are using). Be aware that if you are using Euler angles, the order is important : a vector of rotation (rx, ry, rz) does not describe the same rotation if you turn around X axis first then Y then Z or if you turn first around Z, then Y then X for example.
I suggest you read this, it will help you compute the matrix:
http://brainvoyager.com/bv/doc/UsersGuide/CoordsAndTransforms/SpatialTransformationMatrices.html
Note that all this is using homogenous coordinate.
From the Velodyne point, how to get pixel coordinate for each camera?
Using pykitti
point_cam0 = data.calib.T_cam0_velo.dot(point_velo)
We can get the projection on the image which is equation 7 of the Kitti Dataset paper:
y = Prect(i) Rrect(0) Tvelocam x
But from there, how to get the actual pixel coordinates on each image?
Update: PyKitti version 0.2.1 exposes projection matrices for all cameras.
I recently faced the same problem. For me, the problem was that pykitty didn't expose Prect and Rrect matrices for all cameras.
For Pykitti > 0.2.1, use Prect and Rrect from calibration data.
For previous versions, you have two options:
Enter the matrices by hand (data is in the .xml calibration file for each sequence).
Use this fork of pykitti: https://github.com/Mi-lo/pykitti/
Then, you can use equation 7 to project a velodyne point into an image. Note that:
You will need 3D points as a 4xN array in homogeneous coordinates. Points returned by pykitti are a Nx4 numpy array, with the reflectance in the 4th column. You can prepare the points with the prepare_velo_points function below, which keeps only points with reflectance > 0, then replaces reflectance values with 1 to get homogeneous coordinates.
The velodyne is 360°. Equation 7 will give you a result even for points that are behind the camera (they will get projected as if they were in front, but vertically mirrored). To avoid this, you should project only points that are in front of the camera. For this, you can use the function project_velo_points_in_img below. It returns 2d points in homogeneous coordinates so you should discard the 3rd row.
Here are the functions I used:
def prepare_velo_points(pts3d_raw):
'''Replaces the reflectance value by 1, and tranposes the array, so
points can be directly multiplied by the camera projection matrix'''
pts3d = pts3d_raw
# Reflectance > 0
pts3d = pts3d[pts3d[:, 3] > 0 ,:]
pts3d[:,3] = 1
return pts3d.transpose()
def project_velo_points_in_img(pts3d, T_cam_velo, Rrect, Prect):
'''Project 3D points into 2D image. Expects pts3d as a 4xN
numpy array. Returns the 2D projection of the points that
are in front of the camera only an the corresponding 3D points.'''
# 3D points in camera reference frame.
pts3d_cam = Rrect.dot(T_cam_velo.dot(pts3d))
# Before projecting, keep only points with z>0
# (points that are in fronto of the camera).
idx = (pts3d_cam[2,:]>=0)
pts2d_cam = Prect.dot(pts3d_cam[:,idx])
return pts3d[:, idx], pts2d_cam/pts2d_cam[2,:]
Hope this helps!
As far as I know, Direct3D works with an LH coordinate system right?
So how would I get position and x/y/z axis (local orientation axis) out of a LH 4x4 (world) matrix?
Thanks.
In case you don't know: LH stands for left-handed
If the 4x4 matrix is what I think it is (a homogeneous rigid body transformation matrix, same as an element of SE(3)) then it should be fairly easy to get what you want. Any rigid body transformation can be represented by a 4x4 matrix of the form
g_ab = [ R, p;
0, 1]
in block matrix notation. The ab subscript denotes that the transformation will take the coordinates of a point represented in frame b and will tell you what the coordinates are as represented in frame a. R here is a 3x3 rotation matrix and p is a vector that, when the rotation matrix is unity (no rotation) tells you the coordinates of the origin of b in frame a. Usually, however, a rotation is present, so you have to do as below.
The position of the coordinate system described by the matrix will be given by applying the transformation to the point (0,0,0). This will well you what world coordinates the point is located at. The trick is that, when dealing with SE(3), you have to add a 1 at the end of points and a 0 at the end of vectors, which makes them vectors of length 4 instead of length 3, and hence operable on by the matrix! So, to transform point (0,0,0) in your local coordinate frame to the world frame, you'd right multiply your matrix (let's call it g_SA) by the vector (0,0,0,1). To get the world coordinates of a vector (x,y,z) you multiply the matrix by (x,y,z,0). You can think of that as being because vectors are differences of points, so the 1 in the last element goes the away. So, for example, to find the representation of your local x-axis in the world coordinates, you multiply g_SA*(1,0,0,0). To find the y-axis you do g_SA*(0,1,0,0), and so on.
The best place I've seen this discussed (and where I learned it from) is A Mathematical Introduction to Robotic Manipulation by Murray, Li and Sastry and the chapter you are interested in is 2.3.1.
If I have two vector coordinates representing positions on the surface of the earth where the center of the earth is (0,0,0) and the Up vector is (0,0,1);
What is the best way to calculate a 3d plane that is running along the two vectors (the direction v2 - v1) but back by a set number of meters (just imagine a virtual clip plane that is behind the two vectors and running parallel to them).
well, you do the cross product of v1 and v2 to get the normal of your plane (don't forget to normalize if you want to), then the 4th element of your plane would just be 0 (because it crosses 0,0,0).
and then you want to project the plane in a certain direction based on the UP vector, not the plane's normal?
in that case I think you would just get the dot product of the normal and the up vector, then multiply the inverse of the dot (1/dot) by the number of units you want to project along the up vector and set that as your 4th element?
to clarify, that creates a plane where the two vectors, and the center of the earth are points on the plane, the plane can then be projected up or down by a certain amount in the UP direction.
Planes are usually described by a normal vector N, and as all points x,y,z fitting the equation Ax + By + Cz + D = 0. (A,B,C) is the normal vector. It doesn't even need to be normalized (unit length) if you choose D. Sounds like you want a plane to which v1 and v2 are parallel (and v2-v1 too). For that, make N perpendicular to v1 and v2 by setting it to the cross product of v2 x v1. Then pick a point in (x,y,z) coordinates that you know the plane should pass through. Plug N and (x,y,z) into the equation and compute D.
1- Find the Normal vector N = V1 X V2
2-Select a point that you want your plane tuch P0 ==>R0
3- All the other points(P==>R) in the plane follow N(R-R0)=0
See the link