Escolar Documentos
Profissional Documentos
Cultura Documentos
A Literature Survey
Presented to University of Colombo School of Computing
in partial fulllment of the requirements
for the Degree of Bachelor in Computer Science
SCS3013 Literature Survey Courses
Adviser: Prof. N. D. Kodikara
Copyright
Abstract
In this survey, we research about ways of obstacle detection technics in detail. Study will
go through the importance of obstacle detection, various applications which are included
obstacle detection kits and various technics to achieve obstacle detection. Mainly survey
will discuss about three main areas of obstacle detection. Applications which are innovate
to assist visually impaired people, Path planning using obstacle avoidance of mobile robots
and various kinds of autonomous vehicles has built in obstacle detection system. As well
as survey will review main obstacle detection technics such as monocular technics, stereo
technics, depth maps generating technics etc. In the discussion all the technics will be
compared and consider their pros and cons briey. Later part of the survey discusses
about future enhancements and challenges in obstacle detection.
Acknowledgements
I would like to thank my advisor, Prof. N. D. Kodikara , for his experienced guidance
and constant support throughout the period . I also be obligated to thanks to my family
members and friends, whose support, well wishes and encouragements to brought me
here.
ii
Contents
Abstract
Acknowledgements
ii
List of Figures
1 Introduction
1.1
Motivation
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2
Objectives
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Background
2.1
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2
Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1
False Positive . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2
False Negative
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3
3D versus 2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3
4
. . . . . . . . . . . . . .
3.1.1
Method introduction . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.2
Stereo Measurement
. . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1
Ground/Non-ground Segmentation. . . . . . . . . . . . . . . . . .
3.2.2
Background Subtraction. . . . . . . . . . . . . . . . . . . . . . . .
3.2.3
Size lter
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.4
3.2.5
3.3
3.4
. . . . . . . . . . . . . . . . . .
3.5
Fast Human Detection for Indoor Mobile Robots Using Depth Images . .
10
3.5.1
. . . . . . . . . . . . . . . . . . . . .
10
3.5.2
11
3.5.3
Candidate Classication . . . . . . . . . . . . . . . . . . . . . . .
11
3.6
11
3.7
12
3.7.1
12
3.7.2
12
3.2
iii
. . . . . . . . . . . . . . . . . . . .
4 Discussion
13
5 Conclusion
14
6 Future works
15
References
16
iv
List of Figures
3.1
Gate, obstacle, oor and bump regions that robots can move through. . .
3.2
. . . . . . . . . . . . . . . . .
3.3
. . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4
3.5
3D grid
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6
C-Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7
3.8
Depth image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9
Algorithm owchart
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
. . . . . . . . . . . . . . . . . . . . .
11
12
Introduction
This survey refers how obstacle detection is done by various application areas. Also it
is discussed about technics which are used in obstacle detection in images. Recently obstacle detection is a quit interesting topic among the researchers. Numerous applications
are innovated dierent areas which are included obstacle detection kits. So survey will
discussed mainly about three areas. Most of the vision systems of mobile robots are
contain an obstacle detection system. Recently robots are hugely involving industrial
production and military activities. Navigating through obstacle lled environments is a
big challenge to mobile robots. So there are so many technics to apply to mobile robots
to make easier the navigating task. As well as there are so many applications which are
using obstacle detection system to assist visually impaired people. Those people suering
from lack of information of their surroundings. So assisting applications detect various
obstacles and analyze and present with details to the user. Now days there are lots of
unmanned vehicles are produced for dierent kind of purpose. Such as military, farming
and whether forecasting elds are using many aircrafts and vehicles which are come up
with obstacle detection systems.
1.1
Motivation
Vision base obstacle detection truly motivated by nature. Most animals gain dierent
kinds of vision systems in nature. As humans we got eyes which providing amazing
images.
Flying insects have optimal ows which are giving them absurd navigation and obstacle
detection capabilities.
Actually obstacle detection system is an important sensor in robots and autonomous
vehicle. Because it is a very crucial gadget to have when it in the unknown world. It
may deliver very important information about its surrounding to succeed the navigation.
So vision base obstacle detection directly commit to the safe operation of mobile robots
and autonomous vehicle
1.2
Objectives
Main aspiration of this survey is discuss about pros and cons existing algorithms and
analyze their eciency. Research question of this literature survey is the most suitable
real time technic is for navigating and path planning purposes to mobile robots.
2
2.1
Background
Applications
Vision obstacle detection is a very interesting topic in recent past. Because there are so
many applications have vision system these days. Mainly mobile robots have obstacle
detection system[1] [2] [7]. When mobile robot enter to an unknown environment his
vision system is very helpful to do their operations and survive in that environment. As
well as obstacle avoidance is help to planning their path and reach to the destination[1].
Especially military services use these kinds of robots for dangerous operations[2]. As well
as vision system use for create applications to help visually impaired people[3] [4] [7].They
are suering from lack of information of their surroundings. Researchers have invented
applications that can detect obstacles and give them a message where the obstacle is.
Staircases are really challenging to go through for a visually impaired people. So there
are applications for detecting staircase with details[4]. There are so many unmanned
autonomous vehicles are using obstacle detection systems.
control that vehicle has to have a good vision to navigate through the obstacles. In
military services use unmanned aircrafts for spy enemy activities. Those helicopters have
immediate landing emergency in case of dierent situations. Then helicopters have to
nd minimum obstacle area to landing[10]. As well as there are applications that creates
for farming purpose. There is an autonomous vehicle that has a vision system to obstacle
avoidance. It can safely navigate through the fruit trees[11] .
2.2
Errors
When we work with train data there can be some errors obviously. In vision obstacle
detection has mainly two kinds of errors.
2.3
3D versus 2D
Many robots have 2D obstacle detection system. Solving 2D obstacle detection problem
is much easier than solving 3D obstacle detection. 2D detection system always deals with
intensity map at each pixel in the particular picture. It check whether obstacles to be
identied and not identied. Very rst image indicates changes in the scene. But in 3D
obstacle detection system have to picture the whole scene.
3.1
It is very important to know about its environment for beep robot while it is seeking
path to reach a destination [1].This method will show how robot identies the obstacles
and how plan its path through below mentioned obstacle regions.(Figure 1)
Figure 3.1: Gate, obstacle, oor and bump regions that robots can move through.
Specialty of this method is robot can move through above obstacle area and it can
measure the obstacle. First robot capture images its surrounding and then generates a
3D grid using those images. After 3D grid is converted in to a 2D grid.
Method called conguration space (c space) used for represent the robot. It considers
the robot as a point without any size. As well as all the obstacles are call conguration
obstacles(c-obstacle) in the environment and they are enlarged based on robot size. Figure
6 shows an example of c- space.
After get all the 3D measurements it will create a 3D grid map as shown in gure
4.Then it converts to a 2D map.2D map can be classied into several region. Figure 6
shows all classied areas in 2D map.
Those unmeasured area will be reduced by continues learning process. Now onwards
2D grid map recognize as a graph which is including nods and graph. Robot can plan its
path by using graph traversing technics.
3.2
Then extract information of the foreground grouped into the blobs. Then those blobs
will be ltered based on size model, appearance model and shape model. Several nontarget blobs can be eliminated by this process and system can focus on possible targets.
Most important thing is this method use series of lters and each model that is learn on
the y from the previous model.
3.3
This technics mainly use for detecting obstacles with their depths[3]. This approach was
used for an application which helps for the visually impaired people[1]. They really want
detailed information about their surroundings. We use disparity image or depth map to
storing the depth of each pixel in the image. Each pixel of the depth map corresponding
to a pixel in the image.
Near objects can be seen lighter color and far objects are
Information about depth maps are stored using 64 gray levels. But there are more
levels have to be added in dierent circumstances such as outdoor scenes or high quality
pictures. Depth of a pixel is calculated by following formula.
z=
f b
d
z is the depth, f is the focal lengths in pixels, B is the base line in meters and d is the
disparity. Proposed algorithm is detected two levels of objects. One meter depth objects
catch as a near closeness and two meter threshold for rst detection.
Proposed algorithm briey
8
3.4
When a visually disable person walking through an area staircase is a major obstacle for
him to go through[4]. So there are application which detect various staircases to assist
them.[1]
Main object of the technic is improving the accuracy of detection. To achieve this
target we have to collect set of positive image set which are containing 4 to 5 steps and
photos should be taken on various conditions. As well as we have to collect set of negative
samples also which not contain any staircase. After that training is done until calcied
into 18 layers. Note that training is performed in fewer steps than a normal application.
Because always good to have a higher detection rate in staircase detection applications.
When detection staircase most of time detect similar regions like staircase and they call
candidate detection. For overcome this false detection, we use cascade classiers create
from training session and the 40 * 40 sub windows with dierent sizes for scanning. Haar
detector creates multiple hits in the staircase region and false detection has single isolated
detection.
Note that Stereo camera system produces very poor results for staircase detection
although it gives very informative results with less false detection rate. That's why we
have to come up with such algorithm.All the stages of the algorithm is shown in Figure
9.
3.5
One of the biggest challenges for mobile robots is detect moving humans[5]. Because
Humans can be appear in dierent postures like walking, sitting. Even some are seen in
partially. So detecting those various kinds of humans is very challenging task.
For this approach we use Microsoft depth camera to detect humans. So depth camera
is gives depth images. This algorithm is scanning the depth images and identies the
human in it. Mainly algorithm has three phases.
Figure 3.10: Identifying the corresponding regions of distinct object in depth images
Purpose of this step is identifying the corresponding regions of distinct object in depth
images.(Figure 10) But resolution of depth images get poor with the distance. So that
is not necessary to segment those images at larger distance. Accurate way is sub sample
the depth image and then does the segmentation.
10
In this phase algorithm take Figure 10 as input and omit unnecessary lines which are
highly unlikely humans and merge into the remaining regions.(Figure 10)
3.6
Firstly we have to take a photo of an object by using two cameras this object will specify
a particular place for itself in the pictures taken by cameras[6]. We named these points as
equivalent points. If someone takes picture from dierent distance those places no longer
equivalent.
Then we have to convert the image into binary mode by changing value of pixels into
0 and 1. After that we separate equivalent region from others and blacken the rest of
image. We should identify the object which we interested about and draw a box around
them. Since we take those two images using two cameras there are regions that are exist
in rst picture and not exist in second one. We have to cut those regions out. Since we
are using two cameras in two positions to take the photo one might be bigger than other.
So we have to stretch the images and make them standardize to comparable. Researcher
has found three regions that object can exist. In this approach we can tell object whether
exist or not and the region that obstacle exist.
11
3.7
Visual detection algorithms are mainly base on color and geometric information[7]. Color
base only approaches are copiously gives false negatives and false positives. If there is
a paper of another color on the oor robot will detect it as an obstacle. So it is a false
positive. As well as if there is obstacle which is same color as oor robot will not detect
it. It is false negative. Example instances have been mentioned in below Figure 12.
Even geometric only approaches can be given many false negative results. This approach has two kinds of obstacle detectors both color base and geometric base. There are
dierent approaches to detect them. Both algorithms will produce binary images that
are consisting of white areas which represent the obstacles.
12
Discussion
13
Conclusion
As I mention earlier obstacle detection is most important topic in many elds. Among
those elds robotic technology, unmanned aircrafts in military services and Applications
that are helps to blind people are very important. I must say this is a much brought
area in vision and recent past many researchers are involved to this elds to explore new
methods and approaches.
Although have many technics and algorithms to obstacle detection there are some
suitable approaches depends on the application which are use them.
In mobile robot area most of the time use 3D and 2D grid maps [1] and depth images[5].
I think most ecient way of path planning and obstacle avoidance technic is using the
grid maps. Because in grid maps robot train for his current environment and he knows
the all the obstacles. But only disadvantage is when we add a new obstacle in to the
environment robot have to generate all the grid maps again. In depth images robot real
time detect the obstacle and path his plan. But this approach much slower than grid
maps.
There are two approaches mainly discussed in applications about blind people. First
one is using depth images[3]. I think this is a good approach because user can get all
the information that which shape obstacle is and how many meters to the obstacles.
Problems are the accuracy of the algorithm depends on image brightness and pixel size.
Advantage is user can see the obstacle in two depths. When user knows the obstacle
exist in two meters away he has a time to change his path. Other approach is using
Equivalent Points. But this method detects only the obstacle. Depth can't be measured.
So considering these two approaches you can realize using the depth map will be a better
approach.
14
Future works
This literature survey has described how obstacle detection done by using a camera.
Besides of that there are so many obstacle detection sensors in the market. But they are
very expensive at the moment. That's why many application running with stereo camera.
For an example LADAR (Laser Detection and Ranging) sensors are using for unmanned aircrafts for emergency landing. They are scanning the whole area before landing
the aircraft[12]. LADAR system using for determine the distance to an object. Benets
of the LADAR system can get the image of the target while it is calculating the distance
for the target. That functionality can use for obstacle detection purposes.
15
References
[1]
Atsushi Yamashita, Masaaki Kitaoka, and Toru Kaneko, 2011. Motion Planning of
Biped Robot Equipped with Stereo Camera Using Grid Map. IEEE Transactions on
Robotics and Automation, Vol.5 No.5, 639-647.
[2]
Stancil, Brian A., Hyams, Jerey, Shelley, Jordan; Babu, Kartik; Badino, Hernn.;
Bansal, Aayush; Huber, Daniel; Batavia, Parag, 2013. CANINE: a robotic mine
dog. Intelligent Robots and Computer Vision XXX: Algorithms and Techniques.
Proceedings of the SPIE, Volume 8662, article id. 86620L, 12.
[3]
Paulo Costaa, Hugo Fernandesb, Paulo Martinsc, Joo Barrosod, Leontios J. Hadjileontiadise, (2012). Obstacle detection using stereo imaging to assist the navigation
of visually impaired people. In 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion. Douro Region, Portugal, July 19-22, 2012. Portugal: Procedia Computer Science. 83-94.
[4]
Young Hoon Lee ,Tung-Sing Leung, Grard Medioni, (2012). Real-time staircase
detection from a wearable stereo system. In Pattern Recognition (ICPR), 2012 21st
International Conference. Tsukuba, 11-15 Nov. 2012. 1-4.
[5]
Benjamin Choi, C etin Mericli, Joydeep Biswas, and Manuela Veloso, 2013. Fast
Human Detection for Indoor Mobile Robots Using Depth Images. IEEE International
Conference on Robotics and Automation (ICRA)
[6]
[7]
Luis J. Manso, Pablo Bustos, Pilar Bachiller and Jos e Moreno 2010. Multi-cue
Visual Obstacle Detection for Mobile Robots. journal of physical agents, Vol 4, 1 pg
[8]
Tomoyuki Mori and Sebastian Scherer, "First Results in Detecting and Avoiding
Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles,"
International Conference on Robotics and Automation, May, 2013
[9]
17