Related
I'm trying to align multiple line objects along a human body circumference depending on the orientation of the triangles from the mesh. I would like to put the lines parallel to the mesh. I correctly assign the position for the lines along the circumference, but I also need to add the rotation of the lines such that to be parallel with the body.
The body is a mesh formed by multiple triangles and every line is "linked" with a triangle.
All I have is:
3 points for the closest triangle from the mesh for every line
The normal of the triangle
The positions for the instantiated lines (2 points, start and end)
I need to calculate the angle for every X, Y, Z axes for the line such that the normal of the triangle is perpendicular with the line mesh. I don't know how to get the desired angle. I really appreciate if someone would like to help me.
input:
FVector TrianglePoints[3];
FVector Triangle_Normal; //Calculated as (B-A)^(C-A), where A,B,C are the points of the triangle
FVector linePosition; //I also have the start line and the endLine position if that helps
ouput:
//FRotator rotation(x,y,z), such that the triangle normal and the line object to be perpendicular.
An overview of the circumference line construction. Now the rotation is calculated using the Start position and End position for each line. When we cross some irregular parts of the mesh we want to rotate the lines correctly. Now the rotation is fixed, depending just on the line start and end position.
If I have understood correctly your goal, here is some related vector geometry:
A,B,C are the vertices of the triangle:
A = [xA, yA, zA],
B = [xB, yB, zB]
C = [xC, yC, zC]
K,L are the endpoints of the line-segment:
K = [xK, yK, zK]
L = [xL, yL, zL]
vectors are interpreted as row-vectors
by . I denote matrix multiplication
by x I denote cross product of 3D vectors
by t() I denote the transpose of a matrix
by | | I denote the norm (magnitude) of a vector
Goal: find the rotation matrix and rotation transformation of segment KL
around its midpoint, so that after rotation KL is parallel to the plane ABC
also, the rotation is the "minimal" angle rotation by witch we need to
rotate KL in order to make it parallel to ABC
AB = B - A
AC = C - A
KL = L - K
n = AB x AC
n = n / |n|
u = KL x n
u = u / |u|
v = n x u
cos = ( KL . t(v) ) / |KL|
sin = ( KL . t(n) ) / |KL|
U = [[ u[0], u[1], u[2] ],
[ v[0], v[1], v[2] ],
[ n[0], n[1], n[2] ],
R = [[1, 0, 0],
[0, cos, sin],
[0, -sin, cos]]
ROT = t(U).R.U
then, one can rotate the segment KL around its midpoint
M = (K + L)/2
Y = M + ROT (X - M)
Here is a python script version
A = np.array([0,0,0])
B = np.array([3,0,0])
C = np.array([2,3,0])
K = np.array([ -1,0,1])
L = np.array([ 2,2,2])
KL = L-K
U = np.empty((3,3), dtype=float)
U[2,:] = np.cross(B-A, C-A)
U[2,:] = U[2,:] / np.linalg.norm(U[2,:])
U[0,:] = np.cross(KL, U[2,:])
U[0,:] = U[0,:] / np.linalg.norm(U[0,:])
U[1,:] = np.cross(U[2,:], U[0,:])
norm_KL = np.linalg.norm(KL)
cos_ = KL.dot(U[1,:]) / norm_KL
sin_ = KL.dot(U[2,:]) / norm_KL
R = np.array([[1, 0, 0],
[0, cos_, sin_],
[0,-sin_, cos_]])
ROT = (U.T).dot(R.dot(U))
M = (K+L) / 2
K_rot = M + ROT.dot( K - M )
L_rot = M + ROT.dot( L - M )
print(L_rot)
print(K_rot)
print(L_rot-K_rot)
print((L_rot-K_rot).dot(U[2,:]))
A more inspired solution was to use a procedural mesh, generated at runtime, that have all the requirements that I need:
Continuously along multiple vertices
Easy to apply a UV map for texture tiling
Can be updated at runtime
Isn't hard to compute/work with it
In a game, an area is defined by triangles that never overlap, and characters are defined by circles.
How can I know whether the full character's collision circle is contained within these triangles?
Example image:
Here, the red parts are outside triangles, so the circle isn't contained within them. Is there an algorithm that can detect this?
I've only came up with "non-perfect" solutions, like sampling points at the border of the circle, then testing if each is inside a triangle.
So basically, the triangles form a domain with polygonal boundary and you want to check if a disk, defined by a center point and a radius is contained inside the domain. So if you start with the triangles, you have to find a way to extract the polygonal boundary of your domain and represent it as a 2D array (matrix) of shape n rows and two columns so that every row is the two coordinates of a vertex point of the polygonal boundary line and the points are ordered so that they are consecutive order along the boundary in a counterclockwise position, i.e. when you walk in a direction from point of index i to the next point i+1 the domain stays on your left. For example, here is the representation of a polygonal boundary of a domain like yours:
a = 4/math.sqrt(3)
Pgon = np.array([[0,0],
[a,0],
[2*a,-1],
[2*a+4,0],
[2*a+4,4],
[2*a,4],
[2*a,2],
[a,1],
[a,4],
[0,0]])
Observe that the first and the last points are the same.
In such a scenario, maybe you can try the following algorithm:
import numpy as np
import math
def angle_and_dist(p1, p2, o):
p12 = p2 - p1
op1 = p1 - o
op2 = p2 - o
norm_p12 = math.sqrt(p12[0]**2 + p12[1]**2)
norm_op1 = math.sqrt(op1[0]**2 + op1[1]**2)
norm_op2 = math.sqrt(op2[0]**2 + op2[1]**2)
p12_perp = np.array([ - p12[1], p12[0] ])
h = - op1.dot(p12_perp)
theta12 = op1.dot(op2) / (norm_op1*norm_op2)
theta12 = math.acos( theta12 )
if h < 0:
theta12 = - theta12
if op1.dot(p12) > 0:
return theta12, norm_op1
elif op2.dot(p12) < 0:
return theta12, norm_op2
else:
return theta12, h/norm_p12
def is_in_polygon(p, disk):
o, r = disk
n_p = len(p)-1
index_o = 0
h_min = 400
for i in range(n_p):
theta, h = angle_and_dist(p[i,:], p[i+1,:], o)
index_o = index_o + theta
if 0 <= h and h < h_min:
h_min = h
if theta <= math.pi/100:
return 'center of disc is not inside polygon'
elif theta > math.pi/100:
if h_min > r:
return 'disc is inside polygon'
else:
return 'center of disc is inside polygon but disc is not'
a = 4/math.sqrt(3)
Pgon = np.array([[0,0],
[a,0],
[2*a,-1],
[2*a+4,0],
[2*a+4,4],
[2*a,4],
[2*a,2],
[a,1],
[a,4],
[0,0]])
# A test example:
#disc = (np.array([3*a/4, 2]), a/4-0.001)
disc = (np.array([3*a/4, 2]), math.sqrt(3)*a/8 - 0.0001)
print(is_in_polygon(Pgon, disc))
i know the title seems bad - please feel free to specify. I can not come up with a better title.
All points are stated in polar coordinates relative to the circle center. Two points (v1, v2, s, e) lay on a circle edge. As shown in the image, s & e are the pink points, s = blocked_arc_start_angle, e = blocked_arc_end_angle.
Problem:
How do i check which direction on the circle is not blocked? With direction i mean clockwise (CW, math negative) or counter clockwise (CCW, math positive).
all angles are normalized and have a range of [-PI, PI]
I tried a lot of if-else case checks but due to the range of atan2 i have problems. Is there a easy simple way?
Any ideas?
regards!
I am still looking for a better solution!!! nobady likes many if-else calls...*
Ugly Answer:
I thought about all cases which can occur, you can easily split by imagining the rage of the angle space:
1st case - obstacle/blocked arc/interval is not overlapping:
-Pi PI
|-------------S::::::::E-------------|
2nd case - obstacle/blocked arc/interval is overlapping:
-Pi PI
|::::E--------------------------S::::|
and so on...
important: check in before hand if a point lays inside the blocked interval:
# no point is inside of a blocked arc
if m.point_inside_arc(v1.location, arc) or m.point_inside_arc(v2.location, arc):
return False
Here is some pseudo code. Hopefully this helps some one.
if arc.start_angle < arc.end_angle:
if angle_v1 < angle_v2 < arc.start_angle or arc.end_angle < angle_v1 < angle_v2:
# |----V1---V2----S::::::::E----(V1---V2)----|
math_negative = False
elif angle_v2 < angle_v1 < arc.start_angle or arc.end_angle < angle_v2 < angle_v1:
# |----V2---V1----S::::::::E----(V2---V1)----|
math_positive = False
elif angle_v1 < arc.start_angle and arc.end_angle < angle_v2:
# |----V1---------S::::::::E---------V2----|
math_positive = False
elif angle_v2 < arc.start_angle and arc.end_angle < angle_v1:
# |----V2---------S::::::::E---------V1----|
math_negative = False
else:
# |::::E-------------------------S::::|
if angle_v1 < angle_v2:
math_negative = False
else:
math_positive = False
Let c to be the center of the circle. Then if you calculate:
dot = DotProduct( p1 - c, p2 - c )
All points v for which both conditions are true:
DotProduct( v - c, p1 - c ) > dot
DotProduct( v - c, p2 - c ) > dot
will be in the blocked area
In order to know the direction you can use also dot product. As dot product is the cos of the angle between vectors (supposing both vectors are unitary), you can calculate for each point p1 and p2 the angle alpha:
cos( alpha ) = DotProduct( vector( 1, 0 ), ( p1 - c ).normalized );
After getting alpha, you must use the value of component y of ( p1 - c ).
If it is negative, then add 180 degrees to alpha.
Reduce the two angles modulo 2π. Then if e < s, the circle is uncovered from 0 to s and from e to 2π; otherwise, the circle is uncovered from s to e.
I want to calculate a point on a given line that is perpendicular from a given point.
I have a line segment AB and have a point C outside line segment. I want to calculate a point D on AB such that CD is perpendicular to AB.
I have to find point D.
It quite similar to this, but I want to consider to Z coordinate also as it does not show up correctly in 3D space.
Proof:
Point D is on a line CD perpendicular to AB, and of course D belongs to AB.
Write down the Dot product of the two vectors CD.AB = 0, and express the fact D belongs to AB as D=A+t(B-A).
We end up with 3 equations:
Dx=Ax+t(Bx-Ax)
Dy=Ay+t(By-Ay)
(Dx-Cx)(Bx-Ax)+(Dy-Cy)(By-Ay)=0
Subtitute the first two equations in the third one gives:
(Ax+t(Bx-Ax)-Cx)(Bx-Ax)+(Ay+t(By-Ay)-Cy)(By-Ay)=0
Distributing to solve for t gives:
(Ax-Cx)(Bx-Ax)+t(Bx-Ax)(Bx-Ax)+(Ay-Cy)(By-Ay)+t(By-Ay)(By-Ay)=0
which gives:
t= -[(Ax-Cx)(Bx-Ax)+(Ay-Cy)(By-Ay)]/[(Bx-Ax)^2+(By-Ay)^2]
getting rid of the negative signs:
t=[(Cx-Ax)(Bx-Ax)+(Cy-Ay)(By-Ay)]/[(Bx-Ax)^2+(By-Ay)^2]
Once you have t, you can figure out the coordinates for D from the first two equations.
Dx=Ax+t(Bx-Ax)
Dy=Ay+t(By-Ay)
function getSpPoint(A,B,C){
var x1=A.x, y1=A.y, x2=B.x, y2=B.y, x3=C.x, y3=C.y;
var px = x2-x1, py = y2-y1, dAB = px*px + py*py;
var u = ((x3 - x1) * px + (y3 - y1) * py) / dAB;
var x = x1 + u * px, y = y1 + u * py;
return {x:x, y:y}; //this is D
}
There is a simple closed form solution for this (requiring no loops or approximations) using the vector dot product.
Imagine your points as vectors where point A is at the origin (0,0) and all other points are referenced from it (you can easily transform your points to this reference frame by subtracting point A from every point).
In this reference frame point D is simply the vector projection of point C on the vector B which is expressed as:
// Per wikipedia this is more efficient than the standard (A . Bhat) * Bhat
Vector projection = Vector.DotProduct(A, B) / Vector.DotProduct(B, B) * B
The result vector can be transformed back to the original coordinate system by adding point A to it.
A point on line AB can be parametrized by:
M(x)=A+x*(B-A), for x real.
You want D=M(x) such that DC and AB are orthogonal:
dot(B-A,C-M(x))=0.
That is: dot(B-A,C-A-x*(B-A))=0, or dot(B-A,C-A)=x*dot(B-A,B-A), giving:
x=dot(B-A,C-A)/dot(B-A,B-A) which is defined unless A=B.
What you are trying to do is called vector projection
Here i have converted answered code from "cuixiping" to matlab code.
function Pr=getSpPoint(Line,Point)
% getSpPoint(): find Perpendicular on a line segment from a given point
x1=Line(1,1);
y1=Line(1,2);
x2=Line(2,1);
y2=Line(2,1);
x3=Point(1,1);
y3=Point(1,2);
px = x2-x1;
py = y2-y1;
dAB = px*px + py*py;
u = ((x3 - x1) * px + (y3 - y1) * py) / dAB;
x = x1 + u * px;
y = y1 + u * py;
Pr=[x,y];
end
I didn't see this answer offered, but Ron Warholic had a great suggestion with the Vector Projection. ACD is merely a right triangle.
Create the vector AC i.e (Cx - Ax, Cy - Ay)
Create the Vector AB i.e (Bx - Ax, By - Ay)
Dot product of AC and AB is equal to the cosine of the angle between the vectors. i.e cos(theta) = ACx*ABx + ACy*ABy.
Length of a vector is sqrt(x*x + y*y)
Length of AD = cos(theta)*length(AC)
Normalize AB i.e (ABx/length(AB), ABy/length(AB))
D = A + NAB*length(AD)
For anyone who might need this in C# I'll save you some time:
double Ax = ;
double Ay = ;
double Az = ;
double Bx = ;
double By = ;
double Bz = ;
double Cx = ;
double Cy = ;
double Cz = ;
double t = ((Cx - Ax) * (Bx - Ax) + (Cy - Ay) * (By - Ay)) / (Math.Pow(Bx - Ax, 2) + Math.Pow(By - Ay, 2));
double Dx = Ax + t*(Bx - Ax);
double Dy = Ay + t*(By - Ay);
Here is another python implementation without using a for loop. It works for any number of points and any number of line segments. Given p_array as a set of points, and x_array , y_array as continues line segments or a polyline.
This uses the equation Y = mX + n and considering that the m factor for a perpendicular line segment is -1/m.
import numpy as np
def ortoSegmentPoint(self, p_array, x_array, y_array):
"""
:param p_array: np.array([[ 718898.941 9677612.901 ], [ 718888.8227 9677718.305 ], [ 719033.0528 9677770.692 ]])
:param y_array: np.array([9677656.39934991 9677720.27550726 9677754.79])
:param x_array: np.array([718895.88881594 718938.61392781 718961.46])
:return: [POINT, LINE] indexes where point is orthogonal to line segment
"""
# PENDIENTE "m" de la recta, y = mx + n
m_array = np.divide(y_array[1:] - y_array[:-1], x_array[1:] - x_array[:-1])
# PENDIENTE INVERTIDA, 1/m
inv_m_array = np.divide(1, m_array)
# VALOR "n", y = mx + n
n_array = y_array[:-1] - x_array[:-1] * m_array
# VALOR "n_orto" PARA LA RECTA PERPENDICULAR
n_orto_array = np.array(p_array[:, 1]).reshape(len(p_array), 1) + inv_m_array * np.array(p_array[:, 0]).reshape(len(p_array), 1)
# PUNTOS DONDE SE INTERSECTAN DE FORMA PERPENDICULAR
x_intersec_array = np.divide(n_orto_array - n_array, m_array + inv_m_array)
y_intersec_array = m_array * x_intersec_array + n_array
# LISTAR COORDENADAS EN PARES
x_coord = np.array([x_array[:-1], x_array[1:]]).T
y_coord = np.array([y_array[:-1], y_array[1:]]).T
# FILAS: NUMERO DE PUNTOS, COLUMNAS: NUMERO DE TRAMOS
maskX = np.where(np.logical_and(x_intersec_array < np.max(x_coord, axis=1), x_intersec_array > np.min(x_coord, axis=1)), True, False)
maskY = np.where(np.logical_and(y_intersec_array < np.max(y_coord, axis=1), y_intersec_array > np.min(y_coord, axis=1)), True, False)
mask = maskY * maskX
return np.argwhere(mask == True)
As Ron Warholic and Nicolas Repiquet answered, this can be solved using vector projection. For completeness I'll add a python/numpy implementation of this here in case it saves anyone else some time:
import numpy as np
# Define some test data that you can solve for directly.
first_point = np.array([4, 4])
second_point = np.array([8, 4])
target_point = np.array([6, 6])
# Expected answer
expected_point = np.array([6, 4])
# Create vector for first point on line to perpendicular point.
point_vector = target_point - first_point
# Create vector for first point and second point on line.
line_vector = second_point - first_point
# Create the projection vector that will define the position of the resultant point with respect to the first point.
projection_vector = (np.dot(point_vector, line_vector) / np.dot(line_vector, line_vector)) * line_vector
# Alternative method proposed in another answer if for whatever reason you prefer to use this.
_projection_vector = (np.dot(point_vector, line_vector) / np.linalg.norm(line_vector)**2) * line_vector
# Add the projection vector to the first point
projected_point = first_point + projection_vector
# Test
(projected_point == expected_point).all()
Since you're not stating which language you're using, I'll give you a generic answer:
Just have a loop passing through all the points in your AB segment, "draw a segment" to C from them, get the distance from C to D and from A to D, and apply pithagoras theorem. If AD^2 + CD^2 = AC^2, then you've found your point.
Also, you can optimize your code by starting the loop by the shortest side (considering AD and BD sides), since you'll find that point earlier.
Here is a python implementation based on Corey Ogburn's answer from this thread.
It projects the point q onto the line segment defined by p1 and p2 resulting in the point r.
It will return null if r falls outside of line segment:
def is_point_on_line(p1, p2, q):
if (p1[0] == p2[0]) and (p1[1] == p2[1]):
p1[0] -= 0.00001
U = ((q[0] - p1[0]) * (p2[0] - p1[0])) + ((q[1] - p1[1]) * (p2[1] - p1[1]))
Udenom = math.pow(p2[0] - p1[0], 2) + math.pow(p2[1] - p1[1], 2)
U /= Udenom
r = [0, 0]
r[0] = p1[0] + (U * (p2[0] - p1[0]))
r[1] = p1[1] + (U * (p2[1] - p1[1]))
minx = min(p1[0], p2[0])
maxx = max(p1[0], p2[0])
miny = min(p1[1], p2[1])
maxy = max(p1[1], p2[1])
is_valid = (minx <= r[0] <= maxx) and (miny <= r[1] <= maxy)
if is_valid:
return r
else:
return None
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.