I'm using OpenCV to fit a line from a set of points using cvFitLine()
cvFitLine() returns a normalized vector that is co-linear to the line and a point on the line.
See details here
Using this information how can I get the equation of a line so that I can draw the line?
If cvFitLine() returns normalized vector (vx,vy) and point (x0,y0), then the equation of the line is
(x,y) = (x0,y0) + t*(vx,vy)
where t runs from −∞ to +∞.
This is what you asked for, but probably isn't immediately helpful in drawing the line. You would want to clip it either to the screen boundaries, or perhaps the bounding box of the the original set of points. To clip a line to a rectangle, just solve for values of t where the line crosses the boundary of the rectangle.
Just draw a big line instead of solving for the boundaries. eg:
cv.Line(img, (x0-m*vx[0], y0-m*vy[0]), (x0+m*vx[0], y0+m*vy[0]), (0,0,0))
will do it for example.. for m large enough :)
This just spells out #brainjam's answer in python for any passers by.
The formula for a line using a unit vector (vx, vy) and some point on the line (x0, y0) is:
(x, y) = (x0, y0) + t*(vx, vy)
The return from cv2.fitLine() is:
np.array([vx, vy, x0, y0])
In the example case, I have a line spanning the height of my image, so I want to find the t0 and t1 that intersect with y=0 and y=img.shape[0] (the top/bottom boundaries).
# get the fitLine for your set of points in the array, `line`
fit_line = cv2.fitLine(line, cv2.DIST_L2, 0, 0.01, 0.01)
# compute t0 for y=0 and t1 for y=img.shape[0]: (y-y0)/vy
t0 = (0-fit_line[3])/fit_line[1]
t1 = (img.shape[0]-fit_line[3])/fit_line[1]
# plug into the line formula to find the two endpoints, p0 and p1
# to plot, we need pixel locations so convert to int
p0 = (fit_line[2:4] + (t0 * fit_line[0:2])).astype(np.uint32)
p1 = (fit_line[2:4] + (t1 * fit_line[0:2])).astype(np.uint32)
# draw the line. For my version of opencv, it wants tuples so we
# flatten the arrays and convert
# args: cv2.line(image, p0, p1, color, thickness)
cv2.line(img, tuple(p0.ravel()), tuple(p1.ravel()), (0, 255, 0), 10)
I used a strategy similar to Karpathy up there but used an extra function. As you can see, I'm using cvClipLine to trim the line to the size of the image, which is unnecessary but does add a little niceness.
Also the multiplier here is defined as theMult = max(img->height,img->width) so we dont get numbers that might one day overflow or something.
void drawLine(IplImage * img, float line[4], int thickness,CvScalar color)
{
double theMult = max(img->height,img->width);
// calculate start point
CvPoint startPoint;
startPoint.x = line[2]- theMult*line[0];// x0
startPoint.y = line[3] - theMult*line[1];// y0
// calculate end point
CvPoint endPoint;
endPoint.x = line[2]+ theMult*line[0];//x[1]
endPoint.y = line[3] + theMult*line[1];//y[1]
// draw overlay of bottom lines on image
cvClipLine(cvGetSize(img), &startPoint, &endPoint);
cvLine(img, startPoint, endPoint, color, thickness, 8, 0);
}
Adding to #brainjam answer:
To clip to the bounding box of original set of points:
// std::vector<Point2i> points = ...
//lineParams: [vx,vy, x0,y0]: (normalized vector, point on our contour)
Vec4f lineParams; fitLine(points, lineParams, CV_DIST_L2, 0, 0.01, 0.01);
// derive the bounding xs of points
decltype(points)::iterator minXP, maxXP;
std::tie(minXP, maxXP) = std::minmax_element(points.begin(), points.end(), [](const Point2i& p1, const Point2i& p2){ return p1.x < p2.x; });
// derive y coords of fitted line
float m = lineParams[1] / lineParams[0];
int y1 = ((minXP->x - lineParams[2]) * m) + lineParams[3];
int y2 = ((maxXP->x - lineParams[2]) * m) + lineParams[3];
line(clearTarget, Point(minXP->x, y1), Point(maxXP->x, y2), Scalar(255, 255, 255), 2);
To clip to the entire image boundaries substitute minXP->x to 0 and maxXP->x to image.cols - 1, which was originally answered in https://stackoverflow.com/a/14192660/2380455
we use a " Vec4f fitedLine;" for fitted Line
in fitLine we have 4 parameters
if we consider Line relation az bellow:
Y - Y0 = M (X - X0)
we have
Y0 = FitedLine[3];
X0 = FitedLine[2];
m = FitedLine[1]/FitedLine[0];
so we have a Line equation we can find other points on it.
Related
Basically, a function that fulfills this signature:
function getLineIntersection(vec2 p0, vec2 direction, vec2 p2, vec2 p3) {
// return a vec2
}
I have looked around at existing solutions, and they all seem to deal with how to find the intersection between two line segments, or between two infinite lines. Is there a solution for this problem where the line has an initial position, an angle, and needs to determine if it intersects with a line segment? Basically, something like this:
There should be one line segment that starts in a location and has a unit direction, and another line segment that is just a line connected by two points. Is this possible, and if so, is there a good way of calculating the intersection point, if it exists?
If you've a endless line which is defined by a point P and a normalized direction R and a second endless line, which is defined by a point Q and a direction S, then the intersection point of the endless lines X is:
alpha ... angle between Q-P and R
beta ... angle between R and S
gamma = 180° - alpha - beta
h = | Q - P | * sin(alpha)
u = h / sin(beta)
t = | Q - P | * sin(gamma) / sin(beta)
t = dot(Q-P, (S.y, -S.x)) / dot(R, (S.y, -S.x)) = determinant(mat2(Q-P, S)) / determinant(mat2(R, S))
u = dot(Q-P, (R.y, -R.x)) / dot(R, (S.y, -S.x)) = determinant(mat2(Q-P, R)) / determinant(mat2(R, S))
X = P + R * t = Q + S * u
If you want to detect if the intersection is on the lien, you need to compare the distance of the intersection point with the length of the line.
The intersection point (X) is on the line segment if t is in [0.0, 1.0] for X = p2 + (p3 - p2) * t
vec2 getLineIntersection(vec2 p0, vec2 direction, vec2 p2, vec2 p3)
{
vec2 P = p2;
vec2 R = p3 - p2;
vec2 Q = p0;
vec2 S = direction;
vec2 N = vec2(S.y, -S.x);
float t = dot(Q-P, N) / dot(R, N);
if (t >= 0.0 && t <= 1.0)
return P + R * t;
return vec2(-1.0);
}
Start with the intersection of two infinite lines, expressed in parametric form (e.g., A + tp, where A is the "start point", p is the direction vector and t is a scalar parameter). Solve a system of two equations to get the two parameters of the intersection point.
Now if one of your lines is really a segment AB, and B = A + p (i.e., the direction vector goes from A to B), then if the parameter t is between 0 and 1, the intersection lies on the segment.
In a game, an area is defined by triangles that never overlap, and characters are defined by circles.
How can I know whether the full character's collision circle is contained within these triangles?
Example image:
Here, the red parts are outside triangles, so the circle isn't contained within them. Is there an algorithm that can detect this?
I've only came up with "non-perfect" solutions, like sampling points at the border of the circle, then testing if each is inside a triangle.
So basically, the triangles form a domain with polygonal boundary and you want to check if a disk, defined by a center point and a radius is contained inside the domain. So if you start with the triangles, you have to find a way to extract the polygonal boundary of your domain and represent it as a 2D array (matrix) of shape n rows and two columns so that every row is the two coordinates of a vertex point of the polygonal boundary line and the points are ordered so that they are consecutive order along the boundary in a counterclockwise position, i.e. when you walk in a direction from point of index i to the next point i+1 the domain stays on your left. For example, here is the representation of a polygonal boundary of a domain like yours:
a = 4/math.sqrt(3)
Pgon = np.array([[0,0],
[a,0],
[2*a,-1],
[2*a+4,0],
[2*a+4,4],
[2*a,4],
[2*a,2],
[a,1],
[a,4],
[0,0]])
Observe that the first and the last points are the same.
In such a scenario, maybe you can try the following algorithm:
import numpy as np
import math
def angle_and_dist(p1, p2, o):
p12 = p2 - p1
op1 = p1 - o
op2 = p2 - o
norm_p12 = math.sqrt(p12[0]**2 + p12[1]**2)
norm_op1 = math.sqrt(op1[0]**2 + op1[1]**2)
norm_op2 = math.sqrt(op2[0]**2 + op2[1]**2)
p12_perp = np.array([ - p12[1], p12[0] ])
h = - op1.dot(p12_perp)
theta12 = op1.dot(op2) / (norm_op1*norm_op2)
theta12 = math.acos( theta12 )
if h < 0:
theta12 = - theta12
if op1.dot(p12) > 0:
return theta12, norm_op1
elif op2.dot(p12) < 0:
return theta12, norm_op2
else:
return theta12, h/norm_p12
def is_in_polygon(p, disk):
o, r = disk
n_p = len(p)-1
index_o = 0
h_min = 400
for i in range(n_p):
theta, h = angle_and_dist(p[i,:], p[i+1,:], o)
index_o = index_o + theta
if 0 <= h and h < h_min:
h_min = h
if theta <= math.pi/100:
return 'center of disc is not inside polygon'
elif theta > math.pi/100:
if h_min > r:
return 'disc is inside polygon'
else:
return 'center of disc is inside polygon but disc is not'
a = 4/math.sqrt(3)
Pgon = np.array([[0,0],
[a,0],
[2*a,-1],
[2*a+4,0],
[2*a+4,4],
[2*a,4],
[2*a,2],
[a,1],
[a,4],
[0,0]])
# A test example:
#disc = (np.array([3*a/4, 2]), a/4-0.001)
disc = (np.array([3*a/4, 2]), math.sqrt(3)*a/8 - 0.0001)
print(is_in_polygon(Pgon, disc))
I have the following problem, I am trying to find the following distances (F1 and F2):
This is what I have as of now:
def FindArrow(self, X1, Y1, X2, Y2, X3, Y3):
self.X1 = float(X1)
self.Y1 = float(Y1)
self.X2 = float(X2)
self.Y2 = float(Y2)
self.X3 = float(X3)
self.Y3 = float(Y3)
#center coords of the circle
self.Xc = None
self.Yc = None
#radius
self.R = None
#F1 and F2
self.FAB = None
self.FBC = None
#check if the coordinates are collinear
invalide = self.X1 * (self.Y2 - self.Y3) + self.X2 * (self.Y3 - self.Y1) + self.X3 * (self.Y1 - self.Y2)
if (invalide == 0):
return
#get the coords of the circle's center
s = (0.5 * ((self.X2 - self.X3)*(self.X1 - self.X3) - (self.Y2 - self.Y3) * (self.Y3 - self.Y1))) / invalide
self.Xc = 0.5 * (self.X1 + self.X2) + s * (self.Y2 - self.Y1)
self.Yc = 0.5 * (self.Y1 + self.Y2) + s * (self.X1 - self.X2)
#get the radius
self.R = math.sqrt(math.pow(self.Xc - self.X1, 2) + math.pow(self.Yc - self.Y1, 2))
Until here everything seems to work, now what would be the next steps to get F1 and F2 ?
EDIT:
Both of the answers (Tomer and inerjay) seem to work for the first image, but I have tried the following image and it does not work anymore :
If you have the circle's radius, you can use Pythagoras' Theorem to get the distance from the circle's center to the segment P2-P3 (look at the triangle with its vertices at: P2, the circle's center, and the middle of P2-P3). Then subtract that from the radius to get F2. Same method to get F1.
Edit: If you call x the distance between the center and the segment P2-P3, then F2 is either R-x (as in the first image), or R+x (as in the second image). It depends on how you define F1 and F2.
If you define F1 and F2 so that they always go outward from the triangle P1-P2-P3 (this fits both your examples), then F2 = R-x if the circle's center is on the same side of P2-P3 as P1, and F2 = R+x if P1 and the circle's center are on opposite sides of P2-P3.
If I understand correctly, the segments F1,F2 point towards the cirlce's center. Mark the circle's center O, and build the triangle whose vertices are (P1,P2,O). You know all the lengths of the edges, so you can calculate the triangle's area S (see here). And since you know the area, you also know the length of the triangle's height, which is the continuation of F1 to O:
S = 0.5*dist(P1,P2)*height
Then F1 = radius - height, and same goes for F2.
There's a line A-B and C at the center between A and B. It forms a circle as in the figure. If we assume A-B line as a diameter of the circle and then C is it's center. My problem is I have no idea how to draw another three lines (in blue) each 45 degree away from AC or AB. No, this is not a homework, it's part of my complex geometry in a rendering.
alt text http://www.freeimagehosting.net/uploads/befcd84d8c.png
There are a few ways to solve this problem, one of which is to find the angle of the line and then add 45 degrees to this a few times. Here's an example, it's in Python, but translating the math should be easy (and I've tried to write the Python in a simplistic way).
Here's the output for a few lines:
The main function is calc_points, the rest is just to give it A and B that intersect the circle, and make the plots.
from math import atan2, sin, cos, sqrt, pi
from matplotlib import pyplot
def calc_points(A, B, C):
dx = C[0]-A[0]
dy = C[1]-A[1]
line_angle = atan2(dy, dx)
radius = sqrt(dy*dy + dx*dx)
new_points = []
# now go around the circle and find the points
for i in range(3):
angle = line_angle + (i+1)*45*(pi/180) # new angle in radians
x = radius*cos(angle) + C[0]
y = radius*sin(angle) + C[1]
new_points.append([x, y])
return new_points
# test this with some reasonable values
pyplot.figure()
for i, a in enumerate((-20, 20, 190)):
radius = 5
C = [2, 2]
# find an A and B on the circle and plot them
angle = a*(pi/180)
A = [radius*cos(pi+angle)+C[0], radius*sin(pi+angle)+C[1]]
B = [radius*cos(angle)+C[0], radius*sin(angle)+C[1]]
pyplot.subplot(1,3,i+1)
pyplot.plot([A[0], C[0]], [A[1], C[1]], 'r')
pyplot.plot([B[0], C[0]], [B[1], C[1]], 'r')
# now run these through the calc_points function and the new lines
new_points = calc_points(A, B, C)
for np in new_points:
pyplot.plot([np[0], C[0]], [np[1], C[1]], 'b')
pyplot.xlim(-8, 8)
pyplot.ylim(-8, 8)
for x, X in (("A", A), ("B", B), ("C", C)):
pyplot.text(X[0], X[1], x)
pyplot.show()
If you want to find coordinates of blue lines, may be you will find helpful some information about tranformations (rotations):
http://en.wikipedia.org/wiki/Rotation_matrix
You need to rotate for example vector AC and then you can find coordinate of end point of blue line.
start with this and add a button with code:
private void btnCircleLined_Click(object sender, System.EventArgs e)
{
Graphics graph = Graphics.FromImage(DrawArea);
int x = 100, y = 100, diameter = 50;
myPen.Color = Color.Green;
myPen.Width = 10;
graph.DrawEllipse(myPen, x, y, diameter, diameter);
myPen.Color = Color.Red;
double radian = 45 * Math.PI / 180;
int xOffSet = (int)(Math.Cos(radian) * diameter / 2);
int yOffSet = (int)(Math.Sin(radian) * diameter / 2);
graph.DrawLine(myPen, x, y + yOffSet + myPen.Width + diameter / 2, x + xOffSet + myPen.Width + diameter / 2, y);
graph.DrawLine(myPen, x, y, x + xOffSet + myPen.Width + diameter / 2, y + yOffSet + myPen.Width + diameter / 2);
graph.Dispose();
this.Invalidate();
}
edit: could not see your picture so I misinterpeted your question, but this should get you started.
Translate A with C at the origin (i.e. A-C), rotate CW 45°, then translate back. Repeat three more times.
If I were doing this I'd use polar co-ordinates (apologies for including the link if you are already well aware what they are) as an easy way of figuring out the co-ordinates of the points on the circumference that you need. Then draw lines to there from the centre of the circle.
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.