Main Content

Developing an Algorithm for Undistorting an Image

This example develops a mathematical model using the Symbolic Math Toolbox™ to undistort an image and features a local function in the live script.

Background

Any real world point P(X,Y,Z) can be defined with respect to some 3-D world origin.

Relative to a camera lens, this 3-D point can be defined as p0, which is obtained by rotating and translating P.

p0=(x0,y0,z0) = RP+t

The 3-D point p0 is then projected into the camera's image plane as a 2D point, (x1,y1).

x1=x0z0 , y1=y0z0

When a camera captures an image, it does not precisely capture the real points, but rather a slightly distorted version of the real points which can be denoted (x2,y2). The distorted points can be described using the following function:

x2=x1(1+k1r2+k2r4)+2p1x1y1+p2(r2+2x12)

y2=y1(1+k1r2+k2r4)+2p2x1y1+p1(r2+2y12)

where:

k1, k2 = radial distortion coefficients of the lens

p1, p2 = tangential distortion coefficients of the lens

r=x12+y12

Distortion

An example of lens distortion is shown below; original distorted image (left) and undistorted image (right).

Note the curvature of the lines toward the edges of the first image. For applications such as image reconstruction and tracking, it is important to know the real world location of points. When we have a distorted image, we know the distorted pixel locations (x2,y2). It's our goal to determine the undistorted pixel locations (x1,y1) given (x2,y2) and the distortion coefficients of the particular lens.

While otherwise straightforward, the nonlinear nature of the lens distortion makes the problem challenging.

Define Distortion Model

We begin by defining our distortion model:

% Parameters
syms k_1 k_2 p_1 p_2 real
syms r x y
distortionX = subs(x * (1 + k_1 * r^2 + k_2 * r^4) + 2 * p_1 * x * y + p_2 * (r^2 + 2 * x^2), r, sqrt(x^2 + y^2))
distortionX = p23x2+y2+xk2x2+y22+k1x2+y2+1+2p1xy
distortionY = subs(y * (1 + k_1 * r^2 + k_2 * r^4) + 2 * p_2 * x * y + p_1 * (r^2 + 2 * y^2), r, sqrt(x^2 + y^2))
distortionY = p1x2+3y2+yk2x2+y22+k1x2+y2+1+2p2xy

Radial Distortion k1=0

We plot a grid of pixel locations assuming our lens has a radial distortion coefficient k1=0. Note that distortion is smallest near the center of the image and largest near the edges.

% Set Parameters
parameters = [k_1 k_2 p_1 p_2];
parameterValues = [0 0 0 0];
plotLensDistortion(distortionX,distortionY,parameters,parameterValues)
spacing = 0.2000
distortionX = x
distortionY = y

Figure contains an axes object. The axes object contains 242 objects of type line, quiver.

Radial Distortion k1=0.15

Explore the sensitivity to changes in k1.

% Set Parameters
parameters = [k_1 k_2 p_1 p_2];
parameterValues = [0.15 0 0 0];
plotLensDistortion(distortionX,distortionY,parameters,parameterValues)
spacing = 0.2000
distortionX = 

x3x220+3y220+1

distortionY = 

y3x220+3y220+1

Figure contains an axes object. The axes object contains 242 objects of type line, quiver.

Calculate Inverse Distortion Model

Given a camera's lens distortion coefficients and a set of distorted pixel locations (x2,y2), we want to be able to calculate the undistorted pixel locations (x1,y1). We will look at the specific case where all distortion coefficients are zero except for k1 which equals 0.2.

We begin by defining the distortion coefficients.

syms X Y positive
eq1 = X == distortionX
eq1 = X=p23x2+y2+xk2x2+y22+k1x2+y2+1+2p1xy
eq2 = Y == distortionY
eq2 = Y=p1x2+3y2+yk2x2+y22+k1x2+y2+1+2p2xy

We define the distortion equations for given distortion coefficients, and solve for the undistorted pixel locations (x1,y1).

parameters = [k_1 k_2 p_1 p_2];
parameterValues = [0.2 0 0 0];
eq1 = expand(subs(eq1, parameters, parameterValues))
eq1 = 

X=x35+xy25+x

eq2 = expand(subs(eq2, parameters, parameterValues))
eq2 = 

Y=x2y5+y35+y

Result = solve([eq1, eq2], [x,y], 'MaxDegree', 3,'Real',true)
Result = struct with fields:
    x: (X*(((5*Y^3)/(2*(X^2 + Y^2)) + ((25*Y^6)/(4*(X^2 + Y^2)^2) + (125*Y^6)/(27*(X^2 + Y^2)^3))^(1/2))^(1/3) - (5*Y^2)/(3*(X^2 + Y^2)*((5*Y^3)/(2*X^2 + 2*Y^2) + ((25*Y^6)/(4*(X^2 + Y^2)^2) + (125*Y^6)/(27*(X^2 + Y^2)^3))^(1/2))^(1/3))))/Y
    y: ((5*Y^3)/(2*(X^2 + Y^2)) + ((25*Y^6)/(4*(X^2 + Y^2)^2) + (125*Y^6)/(27*(X^2 + Y^2)^3))^(1/2))^(1/3) - (5*Y^2)/(3*(X^2 + Y^2)*((5*Y^3)/(2*X^2 + 2*Y^2) + ((25*Y^6)/(4*(X^2 + Y^2)^2) + (125*Y^6)/(27*(X^2 + Y^2)^3))^(1/2))^(1/3))

Since element 1 is the only real solution, we will extract that expression into its own variable.

[Result.x Result.y]
ans = 

(Xσ1Yσ1)where  σ1=σ2-5Y23X2+Y2σ2  σ2=5Y32X2+Y2+25Y64X2+Y22+125Y627X2+Y231/3

Now we have analytical expressions for the pixel locations X and Y which we can use to undistort our images.

Function for Drawing the Lens Distortion

function plotLensDistortion(distortionX,distortionY,parameters,parameterValues)
% distortionX is the expression describing the distorted x coordinate
% distortionY is the expression describing the distorted y coordinate
% k1 and k2 are the radial distortion coefficients 
% p1 and p2 are the tangential distortion coefficients 

syms x y

% This is the grid spacing over the image
spacing = 0.2

% Inspect and parametrically substitute in the values for k_1 k_2 p_1 p_2
distortionX = subs(distortionX,parameters,parameterValues)
distortionY = subs(distortionY,parameters,parameterValues)

% Loop over the grid
for x_i = -1:spacing:1
    for y_j = -1:spacing:1
        
        % Compute the distorted location 
        xout = subs(distortionX, {x,y}, {x_i,y_j});
        yout = subs(distortionY, {x,y}, {x_i,y_j});
        
        % Plot the original point
        plot(x_i,y_j, 'o', 'Color', [1.0, 0.0, 0.0])
        hold on
       
        % Draw the distortion direction with Quiver
        p1 = [x_i,y_j];                     % First Point
        p2 = [xout,yout];                   % Second Point
        dp = p2-p1;                         % Difference
        quiver(p1(1),p1(2),dp(1),dp(2),'AutoScale','off','MaxHeadSize',1,'Color',[0 0 1])
        
    end
end
hold off
grid on

end