Você está na página 1de 31

Camera Calibration

What is Camera Calibration?


• Primarily, finding the quantities internal to
the camera that affect the imaging process
§ Position of image center in the image
• It is typically not at (width/2, height/2) of image
§ Focal length
§ Different scaling factors for row pixels
and column pixels
§ Skew factor
§ Lens distortion (pin-cushion effect)
Motivation
• Good calibration is important when we need
to
§ Reconstruct a world model: Virtual L.A. project
§ Interact with the world
• Robot, hand-eye coordination

Image plane Evaluation of position of a square for 2 focal


lengths (red and blue projection geometry)

We see a square of
known size
Scaling of Rows and Columns
in Image
• Camera pixels are not necessarily square
• Camera output may be analog (NTSC)
• Image may be obtained by digitizing card
§ A/D converter samples NTSC signal
Monitor
Camera NTSC signal Digitizing display

CCD/
CMOS
Compound Lens Imaging
• Inexpensive single lens systems distort image at
its periphery
• Compound lenses may be used to reduce
chromatic effects and pin-cushion effects
Camera
Image plane
Nodal Point
Principal planes
f

Image plane for equivalent


pinhole camera is f C
not camera image plane Center of Projection
Variety of Techniques
• VERY large literature on the subject
• Work of Roger Tsai influential
• Linear algebra method described here
§ Can be used as initialization for iterative non
linear methods.
• Some interesting methods use vanishing
points
Camera and Calibration Target
Calibration Procedure
• Calibration target: 2 planes at right angle with
checkerboard patterns (Tsai grid)
§ We know positions of pattern corners only with
respect to a coordinate system of the target
§ We position camera in front of target and find
images of corners
§ We obtain equations that describe imaging and
contain internal parameters of camera
• As a side benefit, we find position and orientation of
camera with respect to target (camera pose)
Image Processing of
Image of Target
• Canny edge detection
• Straight line fitting to detected linked edges
• Intersecting the lines to obtain the image
corners
• Matching image corners and 3D target
checkerboard corners
§ By counting if whole target is visible in image
• We get pairs (image point)--(world point)
( xi , yi ) → ( X i , Yi , Z i )
Central Projection
If world and image points are represented by homogeneous
vectors, central projection is a linear transformation:
xs
xi = f u   f 0
 xs 
0 0  
zs v  =  0   ys 
xi = u / w, yi = v / w
   f 0 0
 zs 
ys w  0 0 1 0  
yi = f 1 Image plane Scene point
zs y Image point
(xs , ys , zs )

center of (xi , yi , f)
x
projection
z
C
f
Transformation From Lengths to Pixels
Transformation uses: y Image point
ypix
(xi , yi , f)
• image center (x0, y0)
y0 Image center x
• scaling factors kx and ky
x0 xpix
xs xs + zs x0
xi = f x pix = k x xi + x0 = f k x
zs zs
ys y s + z s y0
yi = f y pix = k y yi + y0 = f k y
zs zs
 xs 
 u '  α x 0 x0 0   α x = f kx x pix = u ' / w'
 v'  =  0 α y0 0  ys 
   with then
y
 zs  α y = f ky y pix = v' / w'
w'  0 0 1 0  
1
Internal Camera Parameters
 xs 
 u '  α x s x0 0    α x = f kx x pix = u ' / w'
 v'  =  0  y
α y y0 0   withs
    zs  α = − f k y pix = v' / w'
w'  0 0 1 0   y y
1
α x s x0 0 α x s x0  1 0 0 0
0 α y0 0 =  0 α y y0  0 1 0 0 = K [I 3 | 03 ]
 y

 0 0 1 0  0 0 1  0 0 1 0


• αx and αy “focal lengths” in pixels
• x0 and y0 coordinates of image center in pixels

•Added parameter s is skew parameter


• K is called calibration matrix. Five degrees of freedom.
•K is a 3x3 upper triangular matrix
From Camera Coordinates
y to World Coordinates
C x
Image plane

Image point Z
z
(xi , yi , f)

Y
(xs , ys , zs )
M O
(Xs , Ys , Z s ) X
From Camera Coordinates
to World Coordinates 2
x
X
i k z M
j (Xs , Ys , Z s ) in world cordinate system
C
I (xs , ys , zs ) in image cordinate system
CO = T translation vector J
O K Z
CM = CO + OM
xS i + y S j + zS k = Tx i + Ty j + Tz k + X S I + YS J + Z S K
xS = Tx + X S I.i + YS J.i + Z S K.i
 xS  Tx   I.i J.iK.i   X S 
 y  = T  +  I.j J.j K.j   YS 
 S   y 
 z S  Tz  I.k J.k K.k   Z S 
Homogeneous Coordinates

 xS  Tx   I.i J.i K.i   X S 


 y  = T  +  I.j J.j K.j   Y 
 S  y   S 
 z S  Tz  I.k J.k K.k   Z S 

 xS   I.i J.i Tx   X S 
K.i  xS  X S 
 y   I.j y  T  YS 
 S =  J.j Ty   YS 
K.j  S = R
 z S  I.k J.k K.k Tz   Z S   z S  0T3 1   Z S 
        
1  0 0 0 1  1  1  1 
From Camera Coordinates
to World Coordinates 3
x
X
i k z M
j (Xs , Ys , Z s )
C
I (xs , ys , zs )

J
O K Z
CM = OM - OC
xS i + yS j + z S k = ( X S - X C ) I + (YS - YC ) J + ( Z S - Z C ) K
xS = ( X S - X C ) I.i + (YS - YC ) J.i + ( Z S - Z C ) K.i
~ ~
x cam = R (X - C) (T = -R C)
~
C is vector OC expressed in world coordinate system
Homogeneous Coordinates 2
~
• Here we use - R C instead of T

 xS  XS 
y   ~  
 S = R - R C  YS 

 z S  0T3 1  ZS 
   
1  1 
Linear Transformation from
World Coordinates to Pixels
• Combine camera projection and coordinate
transformation matrices into a single matrix P
 xs 
u' y 
 v '  = K [I | 0 3 ] s 
  3
 zs  XS 
w'   u'  ~  
1   R - R C  YS 
⇒  v'  = K [I 3 | 0 3 ] T 
 xS  XS  0 3 1  Z S 
 w'  
y   ~    1 
 S = R - R C  YS 
 XS 
 z S  0T3 1  ZS  u' Y 
    ⇒  v'  = P  S  x = PX
1  1  ZS 
w'  
1 
Properties of Matrix P
• Further simplification of P:
R ~
- R C
x = K [I 3 | 0 3 ] T X
0 3 1 
~
[I 3
R
| 0 3 ] T
- R C
 = R -[R
~
C = R I] [
3 | -
~
C ]
0 3 1 

[ ~
x = K R I3 | -C X ]
[
~
P = K R I3 | - C ]
• P has 11 degrees of freedom: ~
• 5 from triangular calibration matrix K, 3 from R and 3 from C
• P is a fairly general 3 x 4 matrix
•left 3x3 submatrix KR is non-singular
Calibration
• 1. Estimate matrix P using scene points and
their images
• 2. Estimate the interior parameters and the
exterior parameters
[ ~
P = K R I3 | - C ]

§ Left 3x3 submatrix of P is product of upper-


triangular matrix and orthogonal matrix
Finding Camera Translation
• Find homogeneous coordinates of C in the
scene
• C is the null vector of matrix P
§ P C = 0:  X c  0
1 0 0 X c    
P = K R I3[ ~
| -C ] 0 1 0
 Yc    Yc  0
=
 Z c  0
0 0 1 Z c     
 1  0
• Find null vector C of P using SVD
§ C is the unit singular vector of P corresponding to the
smallest singular value (the last column of V, where P
= U D VT is the SVD of P)
Finding Camera Orientation and
Internal Parameters
• Left 3x3 submatrix M of P is of form M=K R
§ K is an upper triangular matrix
§ R is an orthogonal matrix
• Any non-singular square matrix M can be
decomposed into the product of an upper-
triangular matrix K and an orthogonal matrix R
using the RQ factorization
§ Similar to QR factorization but order of 2 matrices is
reversed
RQ Factorization of M
1 0 0   c' 0 s ' c' ' − s' ' 0
R x = 0 c − s  , R y =  0 1 0  , R z = s ' ' c' ' 0
0 s c  − s ' 0 c'  0 0 1
m33 m32
• Compute c = − , s= 2
( m32 + m33 )
2 2 1/ 2
(m32 + m33
2 1/ 2
)
• Multiply M by Rx. The resulting term at (3,2) is zero
because of the values selected for c and s
• Multiply the resulting matrix by Ry, after selecting c’ and
s’ so that the resulting term at position (3,1) is set to zero
• Multiply the resulting matrix by Rz, after selecting c’’ and
s’’ so that the resulting term at position (2,1) is set to zero
M R x R y R z = K ⇒ M = K R Tz R Ty R Tx = K R
Computing Matrix P
• Use corresponding image and scene points
§ 3D points Xi in world coordinate system
§ Images xi of Xi in image
• Write xi = P Xi for all i
• Similar problem to finding projectivity
matrix H (i.e. homography) in homework
Improved Computation of P
• xi = P Xi involves homogeneous
coordinates, thus xi and P Xi just have to be
proportional: x i × P Xi = 0
• Let p1T, p2T, p3T be the 3 row vectors of P
p 1T X i   v'i p T3 Xi − w'i p T2 Xi 
 T   
P X i = p 2 X i  xi × P X i =  w'i p1 Xi − u 'i p 3 Xi 
T T

p T3 X i   u'i p T2 X i − v'i p1T X i 


   
 0T4 − w'i X Ti v 'i XTi  p1  p1 
 T    
⇒  w'i XTi 0T4 − u'i X i  p 2  = 0 p2  is a 12 × 1 vector
− v'i XTi u 'i X Ti 0T4  p 3  p3 
  
Improved Computation of P,
cont’d
• Third row can be obtained from sum of u’i
times first row - v’i times second row
 0T4 − w'i X Ti v'i X Ti  p1 
  
w '
 i i X T
0T4 − u 'i X Ti  p 2  = 0
 − v'i X Ti u 'i XTi 0T4  p 3 

• So we get 2 independent equations in 11
unknowns (ignoring scale)
• With 6 point correspondences, we get
enough equations to compute matrix P
Ap=0
Solving A p = 0
• Linear system A p = 0
• When possible, have at least 5 times as
many equations as unknowns (28 points)
• Minimize || A p || with the constraint
|| p || = 1
§ P is the unit singular vector of A corresponding
to the smallest singular value (the last column
of V, where A = U D V T is the SVD of A)
• Called Direct Linear Transformation (DLT)
Improving P Solution with
Nonlinear Minimization
• Find p using DLT
• Use as initialization for nonlinear
minimization of ∑ d ( x i , PX i ) 2

i
§ Use Levenberg-Marquardt iterative minimization
Radial Distortion
• We have assumed that lines are imaged as
lines
• Not quite true for real lenses
§ Significant error for cheap optics and for short
focal lengths

Scene Image Corrected image


Radial Distortion Modeling
• In pixel cordinates the correction is written

xc − x0 = L(r ) ( x − x0 )
(x, y) yc − y0 = L ( r ) ( y − y0 )
(xc, yc)
(x0, y0) with
Distorted line
r 2 = ( x − x0 ) 2 + ( y − y0 ) 2
And ideal line:
L(r ) = 1 + κ 1 r + κ 2 r 2 + ...
(x, y)

(x’, y’)
• Minimize f (κ 1 ,κ 2 ) = ∑ ( x'i − xci ) 2 + ( y 'i − yci ) 2
i
using lines known to be straight
(x’,y’) is radial projection of (x,y) on straight line
References

• Multiple View Geometry in Computer Vision, R.


Hartley and A. Zisserman, Cambridge University
Press, 2000, pp. 138-183
• Three-Dimensional Computer Vision: A
Geometric Approach, O. Faugeras, MIT Press,
1996, pp. 33-68
• “A Versatile Camera Calibration Technique for
3D Machine Vision”, R. Y. Tsai, IEEE J. Robotics
& Automation, RA-3, No. 4, August 1987, pp.
323-344

Você também pode gostar