This code is part of a more complex library I'm making, the whole thing would require larger context so I'm hoping the simplified version is enough to work with. It's part of a function for positioning 3D surfaces in a Minecraft type voxel engine I'm making in Godot (GDScript), notably generating the UV coordinates so at any surface or texture scale the image matches and repeatedly starts from the top-left corner of the largest unit. The surface scale and image resolution are independent values but each must be a division or power of two, eg: 0.0625, 0.125, 0.25, 0.5, 1, 2, 4, 8, 16, etc.
# res = surface resolution setting, tex = texture resolution setting
# pos is the center of the plane in 3D space, size is the 2D scale of the plane across the direction it spans
var scale_x = 1 / tex
var scale_y = 1 / tex
var ofs_x = 0.5 - (res / 2) if res < 0.5 else res / 2
var ofs_y = 0.5 - (res / 2) if res < 0.5 else res / 2
if dir == +x or dir == -x:
ofs = Vector2((pos.y - size.x - ofs_x) * scale_x, (pos.z - size.y - ofs_y) * scale_y)
elif dir == +y or dir == -y:
ofs = Vector2((pos.x - size.x - ofs_x) * scale_x, (pos.z - size.y - ofs_y) * scale_y)
elif dir == +z or dir == -z:
ofs = Vector2((pos.x - size.x - ofs_x) * scale_x, (pos.y - size.y - ofs_y) * scale_y)
When the sync is working correctly it looks like this in my test:
Problem is I couldn't get a predictable result. After over a day of struggling to understand why the texture wouldn't align correctly sometimes and trying values manually to find a pattern, I made a rather baffling discovery: When the geometry resolution drops below 0.5, the pattern changes and even reverses direction. If at values above 0.5 the offset is a simple res / 2 for values below 0.5 it becomes 0.5 - (res / 2). I tested it many times and could clearly confirm the following combinations work for each of those resolution values:
res: 0.0625 = ofs: 0.46875
res: 0.125 = ofs: 0.4375
res: 0.25 = ofs: 0.375
res: 0.5 = ofs: 0.25
res: 1 = ofs: 0.5
res: 2 = ofs: 1
res: 4 = ofs: 2
Why isn't it a fixed pattern in either direction? The weird offset variables I coded above seem to be producing the desired results, but I still don't understand how or why they're doing so, unless I do so this feels like a hack and I'm left wondering if I should be fixing something else. Can anyone explain the bizarre pattern please, and if there's a more efficient way to code my solution?
I have a following transformation when doing some kind of zoom/upscaling of a point. My goal is to calculate an offset based on this scale.
My problem is that when going from a big scale to a smaller scale I'd of course have the offset to be the same. Eg if I scale from 3 to 4 and back from 4 to 3, the offset on scale of 3 should always be the same.
But with my formula, it is not. And I cannot get my head around what I'm doing wrong:
px = 200
offset = 0
scale: 1, and goes always +-1
calculation based on forumla: newOffset = oldOffset +- px / scale;
scale = 2 => offset = 0 + 200 / 2 = 100
scale = 3 => offset = 100 + 200 / 3 = 166,67
scale = 4 => offset = 166,67 + 200 / 4 = 216,67
How can I revert the scaling?
scale = 3 => offset = 216,67 - 200 / 3 = 150 # //it should evaluate to 166,67
The offsets are defined by a recursion relation:
offset(0) = 0
offset(i) = offset(i-1) + px/(i+1)
Or, if we were to write out the first few terms,
offset0 = 0
offset1 = offset0 + px/2 = 100
offset2 = offset1 + px/3 = offset0 + px/2 + px/3 = 166.67
offset3 = offset2 + px/4 = offset0 + px/2 + px/3 + px/4 = 216.67
So the offsets are equal to a constant, offset0, plus the first N terms of the harmonic series (the sum of terms 1/n for n = 2,3,...) scaled by px.
There is no closed form algebraic expression for the first N terms of the harmonic series, so either store the numbers and look them up as needed, or recompute the value when you "rescale".
I know that to draw a regular polygon from a center point, you use something along the lines of:
for (int i = 0; i < n; i++) {
p.addPoint((int) (100 + 50 * Math.cos(i * 2 * Math.PI / n)),
(int) (100 + 50 * Math.sin(i * 2 * Math.PI / n))
);
}
However, is there anyway to change this code (without adding rotations ) to make sure that the polygon is always drawn so that the topmost or bottommost edge is parallel to a 180 degree line? For example, normally, the code above for a pentagon or a square (where n = 5 and 4 respectively) would produce something like:
When what I'm looking for is:
Is there any mathematical way to make this happen?
You have to add Pi/2-Pi/n
k[n_] := Pi/2 - Pi/n;
f[n_] := Line[
Table[50 {Cos[(2 i ) Pi/n + k[n]] ,Sin[(2 i) Pi/n + k[n]]}, {i,0,n}]];
GraphicsGrid#Partition[Graphics /# Table[f[i], {i, 3, 8}], 3]
Edit
Answering your comment, I'll explain how I arrived at the formula. Look at the following image:
As you may see, we want the middle point of a side aligned with Pi/2. So ... what is α? It's obvious
2 α = 2 Pi/n (one side) -> α = Pi/n
Edit 2
If you want the bottom side aligned with the x axis, add 3 Pi/2- Pi/n instead ...
Add Math.PI / n to the angles.
This is very programming related but a somewhat non-programming question. I am performing image scaling in a web based application and I need to maintain my image relative to a fixed location even though it scales anchored by its top, left corner. Hope the graphic make this possible.
The idea is that C is a fixed location that I want to maintain as my scaling origin rather than B which which is the current css behavior. C may or may not be within the actual image. So as the image scale, B needs to move relative to C. Example: if the image was scaled 50%, then B would move 1/2 the distance to C. If the image grew to 200% of its size, then B would move twice the distance away from C.
Ultimately looking for a formula for x & y for B given the location of C and a scaling factor for the image. Not sure the size of the image needs to be part of this but I have it if needed.
Thanks for any help!
Things I know:
I know the width and height of the
image rectangle.
I know the offset of B from A.
I know the offset of C from A.
I know the scale factor in percent of the image.
Effectively, you want to treat C as the origin, and just "move" B by the scaling amount. By treating it as a vector from C to B, and scaling it by the amount in question, you can do this fairly easily.
newBx = Cx - (Cx - Bx) * scale;
newBy = Cy - (Cy - By) * scale;
For example, with a scale of 0.5 (50%), this becomes:
newBx = 100 - (100 - 50) * 0.5
= 100 - 25
= 75 // 1/2 the distance to C
newBy = 100 - (100 - 25) * 0.5
= 100 - 37.5
= 62.5 // 1/2 the distance to C
With a scale of 2 (200%):
newBx = 100 - (100 - 50) * 2
= 100 - 100
= 0 // 2x the distance to C
newBy = 100 - (100 - 25) * 2
= 100 - 150
= -50 // 2x the distance to C
First you need to calculate the distance from B to C, then you just change that to scale, and that is where the new B is relative to C:
newB = C - (C - B) * scale
As you want the coordinates, it's the same function for x and y:
newBx = Cx - (Cx - Bx) * scale
newBy = Cy - (Cy - By) * scale
(The scale value used is not percentage but a size multiplier. An increase in size by 50% gives a scale of 1.5.)
So you want point C in the image which is currently at (C_x, C_y) to remain at the same position after scaling the image by a factor of s?
New position of C, say, C_new = (s*C_x,s*C_y).
And you want to move the image so that C_new = C.
Which means you'll have to shift B = (B_x,B_y) by (s*C_x-C_x,s*C_y-C_y), or the new origin of the image, say B_new is:
B_new = (B_x + s*C_x-C_x, B_y + s*C_y-C_y)
So now you can display the scaled image at B_new --- and C should remain fixed.
If I understand the problem:
X(b) = X(c) - Width*(1/3)
Y(b) = Y(c) - Height*(3/4)
The formula seems simple enough, but your sample image can't get any larger than 133x200 (scale = 133%) before it overruns Y=0 (which I assume is your northern limit).
If you want to stop it from moving past Y=0 or X=0, and push-out further to the south and east once it reaches either limit, one approach might be:
IIF(Height > 133, Y(b) = 0, Y(b) = Y(c) - Height*(3/4))
IIF(Width > 450, X(b) = 0, X(b) = X(c) - Width*(1/3))
I think scale should be converted to height and width, instead of using scale as a variable in these formulas, since your original image could be any size (assuming they're not always going to be 100x150 per your sample)
dave
here's a C# snippet that is tested to work:
void Main()
{
Application.Run(new form1());
}
public class form1 : Form
{
static Point C = new Point(100,100);
static Point origLocB = new Point(50,25);
static Size origSizeB = new Size(150,100);
Panel Rec = new Panel()
{
Left = origLocB.X,
Top = origLocB.Y,
Width = origSizeB.Width,
Height = origSizeB.Height,
BorderStyle = BorderStyle.FixedSingle,
};
NumericUpDown nud = new NumericUpDown()
{
Value = 1M,
Increment = .01M,
DecimalPlaces = 2,
Dock = DockStyle.Bottom,
};
public form1()
{
nud.ValueChanged += NumericUpDown_ValueChanged;
Controls.Add(nud);
Controls.Add(Rec);
}
public void NumericUpDown_ValueChanged(object sender, EventArgs e)
{
Rec.Location = new Point(((int)((origLocB.X - C.X) * nud.Value + C.X)),
((int)((origLocB.Y - C.Y) * nud.Value + C.Y)));
Rec.Size = new Size((int)(origSizeB.Width*nud.Value),
(int)(origSizeB.Height*nud.Value));
}
}
it really just echo's #Reed's Answer
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.