Escolar Documentos
Profissional Documentos
Cultura Documentos
Camera Calibration
Zhengyou Zhang
Sung Huh
CSPS 643 Individual Presentation 1
February 25, 2009
1
Outline
Introduction
Equations and Constraints
Calibration and Procedure
Experimental Results
Conclusion
2
Outline
Introduction
Equations and Constraints
Calibration and Procedure
Experimental Results
Conclusion
3
Introduction
Extract metric information from 2D
images
Much work has been done by
photogrammetry and computer vision
community
◦ Photogrammetric calibration
◦ Self-calibration
4
Photogrammetric Calibration
(Three-dimensional reference object-based calibration)
5
Self-Calibration
Do not use any calibration object
Moving camera in static scene
The rigidity of the scene provides
constraints on camera’s internal parameters
Correspondences b/w images are sufficient
to recover both internal and external
parameters
◦ Allow to reconstruct 3D structure up to a similarity
Very flexible, but not mature
◦ Cannot always obtain reliable results due to many
parameters to estimate
6
Other Techniques
Vanishing points for orthogonal directions
Calibration from pure rotation
7
New Technique from Author
Focused on a desktop vision system (DVS)
Considered flexibility, robustness, and low
cost
Only require the camera to observe a planar
pattern shown at a few (minimum 2)
different orientations
◦ Pattern can be printed and attached on planer surface
◦ Either camera or planar pattern can be moved by hand
More flexible and robust than traditional
techniques
◦ Easy setup
◦ Anyone can make calibration pattern
8
Outline
Introduction
Equations and Constraints
Calibration and Procedure
Experimental Results
Conclusion
9
Notation
2D point, m u, v
T
3D point,
T
M X , Y , Z
0 0 1
10
Notation
s: extrinsic parameters that relates the
world coord. system to the camera coord.
System
A: Camera intrinsic matrix
(u0,v0): coordinates of the principal point
α,β: scale factors in image u and v axes
γ: parameter describing the skew of the
two image
11
Homography b/w the Model Plane
and Its Image
Assume the model plane is on Z = 0
Denote ith column of the rotation matrix R by
ri X
u Y X
s v A r1 r2 r3 t A r1 r2 t Y
0
1 1
1
Relation b/w model point M and image m
sm HM H A r1 r2 t (2)
12
Constraints on Intrinsic Parameters
Let H be H = [h1 h2 h3]
h1 h2 h3 A r1 r2 t
Homography has 8 degrees of freedom &
6 extrinsic parameters
Two basic constraints on intrinsic
parameter
h1T A T A 1h 2 0 (3)
13
Geometric Interpretation
Model plane described in camera
coordinate system
x
T
3 y
r w 0
rT t z 0, w 1
3
w
Model plane intersects the plane at
infinity at a line
r1 r2
0, 0
14
Geometric Interpretation
r1 r2
x a b
0 0
x∞ is circular point and satisfy xT x 0 , or
a2 + b2 = 0
Two intersection points
r1 ir2
x a
0
This point is invariant to Euclidean
transformation
15
Geometric Interpretation
Projection of x∞ in the image plane
m A r1 ir2 h1 ih2
1 2 T 1
h1 ih 2 0
T
h ih A A
17
Calibration
Analytical solution
Nonlinear optimization technique based
on the maximum-likelihood criterion
18
Closed-Form Solution
B11 B21 B31
Define B = A-TA-1 ≡ B B22 B32
12
B13 B23 B33
(5)
1 v0 u0
2
2 2
2 1 v0 u0 v0 2
2 2
2 2 2
2 2
v0 u0 v02 0 0 v0 1
2
v0 u0 v u 2
2 2
2 2
2 2 2
B is defined by 6D vector b
b B11 B33
T
B12 B22 B13 B23 (6)
19
Closed-Form Solution
ith column of H = hi
hi hi1 hi 3
T
hi 2
20
Closed-Form Solution
Two fundamental constraints, from homography,
become
T
v12 (8)
T
b0
v11 v 22
If observed n images of model plane
Vb 0 (9)
V is 2n x 6 matrix
Solution of Vb = 0 is the eigenvector of VTV
associated w/ smallest eigenvalue
Therefore, we can estimate b
21
Closed-Form Solution
If n ≥ 3, unique solution b defined up to a
scale factor
If n = 2, impose skewless constraint γ = 0
If n = 1, can only solve two camera
intrinsic parameters, α and β, assuming u0
and v0 are known and γ = 0
22
Closed-Form Solution
Estimate B up to scale factor, B = λATA-1
B is symmetric matrix defined by b
B in terms of intrinsic parameter is known
Intrinsic parameters are then
v0 B12 B13 B11 B23 B11B22 B122
B33 B132 v0 B12 B13 B11B23 B11
B11
B11 B11 B22 B122
B12 2
u0 v0 B13 2
23
Closed-Form Solution
Calculating extrinsic parameter from
Homography H = [h1 h2 h3] = λA[r1 r2 t]
r1 A 1h1 r2 A 1h 2 r3 r1 r2 t A 1h 3
1 A 1h1 1 A 1h 2
R = [r1 r2 r3] does not, in general, satisfy
properties of a rotation matrix because of noise
in data
R can be obtained through singular value
decomposition
24
Maximum-Likelihood Estimation
Given n images of model plane with m points
on model plane
Assumption
◦ Corrupted Image points by independent and
identically distributed noise
Minimizing following function yield maximum
likelihood estimate
mij mˆ A, Ri , t i , M j
n m
2
(10)
i 1 j 1
25
Maximum-Likelihood Estimation
ˆ A, R i , t i , M j
m is the projection of point Mj in
image i
R is parameterized by a vector of three
parameters
◦ Parallel to the rotation axis and magnitude is equal to
the rotation angle
R and r are related by the Rodrigues formula
Nonlinear minimization problem solved with
Levenberg-Marquardt Algorithm
Require initial guess A, Ri , t i | i 1..n
26
Calibration Procedure
1. Print a pattern and attach to a planar surface
2. Take few images of the model plane under
different orientations
3. Detect feature points in the images
4. Estimate five intrinsic parameters and all the
extrinsic parameters using the closed-form
solution
5. Refine all parameters by obtaining maximum-
likelihood estimate
27
Outline
Introduction
Equations and Constraints
Calibration and Procedure
Experimental Results
Conclusion
28
Experimental Results
Off-the-shelf PULNiX CCD camera w/ 6mm lense
640 x 480 image resolution
5 images at close range (set A)
5 images at larger distance (set B)
Applied calibration algorithm on set A, set B and Set
A+B
29
Experimental Result
30
Experimental Result
http://research.microsoft.com/en-us/um/people/zhang/calib/
31
Outline
Introduction
Equations and Constraints
Calibration and Procedure
Experimental Results
Conclusion
32
Conclusion
Technique only requires the camera to
observe a planar pattern from different
orientation
Pattern could be anything, as long as the
metric on the plane is known
Good test result obtained from both
computer simulation and real data
Proposed technique gains considerable
flexibility
33
Appendix
Estimating Homography b/w the Model Plane and its Image
34
Appendix
Minimizing following function yield
maximum-likelihood estimation of H
mi mˆ i m1i mi mˆ i
T
1 h
T
1 Mi
where ˆi T
m with hi = ith row of H
h3 M i h 2 M i
T
35
Appendix
Assume Λ I for all i
mi
2
36
Appendix
T
x= h h
T T T
Let
1 h 2 3
Then (2) become
37
Appendix
Elements of L
◦ Constant 1
◦ Pixels
◦ World coordinates
◦ Multiplication of both
38
Possible Future Work
Improving distortion parameter caused by
lens distortion
39
Question?
40