Escolar Documentos
Profissional Documentos
Cultura Documentos
Grard Medioni
Institute of Robotics and Intelligent Systems
Computer Science Department
Viterbi School of Engineering
University of Southern California
Problem Definition
Scenario: rigidly moving objects + moving camera
Goal
Motion segmentation: motion regions / background area
Tracking of multiple objects: consistent track(s) over time
Geo-registration and Geo-tracking: Geo-referenced mosaic and tracks
Moving cameras
Image stabilization
Motion
segmentation
Tracking
Mosaic+Tracks
+Tracks
Mosaic
Moving camera
Map
Image stabilization
Geo registration
Global data
association
Motion
segmentation
Tracking
Geo-referenced
Geo-referenced
mosaic++tracks
tracks
mosaic
Difficulties
Camera motion
3D Static structures (parallax)
Multiple moving objects
Applications
Video surveillance
Video compression and indexing
Outline
Introduction
2D Motion segmentation
Tracking of multiple moving objects
Geo-registration and geo-tracking
Summary and Discussion
Assumptions
General camera motion
Distant scene
Textured background
Matching
Small inter-frame motion
Gray-scale windows
Cross correlation
2D affine transform
Robust estimation
Random Sample Consensus
(RANSAC)
Keep the model with the
largest number of inliers
Ap1 = p 2
A11
A
21
0
A12
A22
0
A13 u1 u2
A23 v1 = v2
1 1 1
Motion Segmentation
Two-frame pixel-level segmentation?
Segmentation within a temporal window
Accumulate the pixels warped from adjacent frames
K-Means to find the most representative pixel
Frame differencing and thresholding: |Ioriginal-Imodel|>I
Frame t
Frame t-w
t: reference frame
w: half size of the window
Frame t+w
10/72
Original
Images
Motion
Prob.
Maps
Initial
Detection
Results
Tracking
Results
11/72
Original
images
Motion
Prob.
Maps
Initial
Detection
Results
Tracking
Results
Outline
Introduction
2D Motion segmentation
Tracking of multiple moving objects
Geo-registration and geo-tracking
Summary and Discussion
Radar
UAV camera
Stationary camera
Related work
One-to-one assumption
Approximately
Labeling of foreground regions
Our formulation
Given
A set of noisy observations (foreground regions)
Find
A cover of foreground regions over time
Solution space
Solution space is a collection of spatio-temporal covers of
observation Y.
Joint association event
= { 1 , 2 K, K }
(a) Observations Y
Bayesian formulation
MAP estimate
* = arg max( p( | Y ))
p ( | Y ) p (Y | ) p ( )
Prior model p()
Few number of long tracks
One track should have little overlapping with other track unless necessary
p( ) = p ( L) p( K ) p(O)
Likelihood p(Y | )
Smoothness in both motion and appearance
Areas of uncovered false alarms p(F)
K | k |1
p (Y | ) = p ( F ) L( k (ti +1 ) | k (ti ))
k =1 i =1
Motion likelihood
Appearance likelihood
Motion
xtk+1 = Ak xtk + w
y = H x +v
k
t
k
t
w ~ N (0, Q)
v ~ N (0, R)
k (ti+1)
k (ti+1)
Appearance
LM ( k (ti +1 ) | k (ti )) p( k (ti +1 ) | k +1 (ti ))
Solution to MAP:
Sampling based method to avoid enumerating all possible solutions
Two types of proposal moves (temporal and spatial moves)
Symmetric temporal information
Metropolis-Hasting algorithm
Propose a new state from the previous state (i)
' ~ q( ' | (i ) )
Accept with probability
p( ')q( ( i )
| ')
min 1,
(i )
(i )
p ( )q ( ' | )
Properties
Dont have to compute the global p(), but the local ratio p()/ p()
For MAP, dont have to keep the whole chain, but the current state and the
best one
Metropolis-Hasting algorithm
1. Initialize (0) .
2. For i = 0 to N -1
- Sample u U [0,1]
- Propose ' q( ' | (i ) ).
(i )
p
(
')
q
(
| ')
(i )
- Compute A( , ')= min 1,
(i )
(i )
p
(
)
q
(
'
|
)
- If u < A( ( i ) , ')
else
( i +1) = '
(i +1) = (i )
Endfor
Two types of q( | )
Temporal moves and spatial
Birth/Death
Data-driven proposal
q( ' | ) q( ' | , D)
Spatial moves are made only after
Temporal Moves
Extension/
Reduction
Split/Merge
Switch
Spatial Moves
collected
Segmentation
/Aggregation
Diffusion
- Sample u U [0,1]
- Sample if i < N , ' qTemporal ( ' | ( i ) )
else
(i )
p
(
')
q
(
| ')
(i )
- Compute A( , ')= min 1,
(i )
(i )
p( )q( ' | )
- If u < A( ( i ) , ')
else
Endfor
( i +1) = '
(i +1) = (i )
Determining Parameters
Determine the parameters in the full posterior
Casual setting makes ground truth p(gt|Y) even much lower than the
solution.
Take advantage of the property of MCMC
Degenerate the gt to
p( gt )
p ( ')
A [C0 , C1 , C2 , C3 , C4 ] b
C0 , C1 , C2 , C3 , C4 0
max(C + C + C + C + C )
0
1
2
3
4
Simulation experiments
Settings
Original video
Tracking result
Simulation experiments
Quantitative comparison
Simulation experiments
Online implementation
Sliding window W
Initialize t with *t-1
Real Scenarios
Experiments
CLEAR 320x240
Vivid-II 320x240
Experiments
Can handle occlusion at the beginning by using symmetric
temporal information
Outline
Introduction
2D Motion segmentation
Tracking of multiple moving objects
Geo-registration and geo-tracking
Summary and Discussion
Geo-registration
Use 2D homography to
H i +1, M = ( H i ,i +1 ) H i , M H update
view) motion
Hi,i+1
Hi,M
Hi+1,M
Hupdate
37/72
Geo-registration results
Experimental results
Results are shown on two UAV data sets
Map is acquired from Google Earth
Geo-registration is performed every 50 frames
Local data association (MCMCDA) window 50 frames
Geo-registration
Without geo-refinement
With geo-refinement
Experimental results
Experimental results
System implementation
C++ implementation
Xeon Dual Core P4 3.0GHz
Preliminary time performance
Procedure
Image registration
~ 0.25
~0.25
Geo-registration
~ 6 every 50 frames
Tracking
~ 0.4
Total
~ 1 ( GPU)
43/72
Outline
Introduction
2D Motion segmentation
Tracking of multiple moving objects
Geo-registration and geo-tracking
Summary and Discussion
Highlights
Solution to practical problems in detection and tracking area
Future directions
Multi-view geometry + object recognition
Automatically determination of applicable tasks
Reference
Qian Yu and Grard Medioni, Integrated Detection and Tracking for Multiple Moving
Objects using Data-Driven MCMC Data Association, IEEE Workshop on Motion and Video
Computing (WMVC'08), 2008
Qian Yu, Grard Medioni, Isaac Cohen, "Multiple Target Tracking Using Spatio-Temporal
Monte Carlo Markov Chain Data Association" IEEE Conference on Computer Vision and
Pattern Recognition, 2007 (CVPR'07), pp.1-8
Qian Yu, Grard Medioni, "Map-Enhanced Detection and Tracking from a Moving Platform
with Local and Global Data Association," IEEE Workshop on Motion and Video Computing
(WMVC'07), 2007
Yuping Lin, Qian Yu, Gerard Medioni "Map-Enhanced UAV Image Sequence Registration"
Workshop on Applications of Computer Vision (WACV'07), 2007
Qian Yu, Isaac Cohen, Grard Medioni and Bo Wu "Boosted Markov Chain Monte Carlo
Data Association for Multiple Target Detection and Tracking," Proceedings of the 18th
international Conference on Pattern Recognition (ICPR'06), Vol. 2, pp. 675-678.
Q&A
Thank you!