SWIPE - Image Segmentation Based Gesture Control

Nandini Muskan, Poulomi Saha, Tanay Kumar Bera

Department of Computer Science & Engineering,
Neotia Institute of Technology, Management &
Science, Kolkata, India

Sudip Kumar Adhikari

Department of Computer Science & Engineering,
Neotia Institute of Technology, Management & Science,
Kolkata, India
Email: sudipadhikari@ieee.org
*Corresponding author

Abstract This project is an image segmentation based

application which is developed using Java language. The main
aim of this application developed Swipe is to enable users to
operate their system using gesture control. It is a desktop based
application. It is basically a utility tool that provides basic key
operations without physically accessing any input hardware. In
this application a webcam/camera connected to the system is
required in order to get the video feed. The video feed is used by
the user to select the handle (object to be used for gesture
control). The handle is detected uniquely and properly using
image preprocessing and image segmentation techniques. The
viewport is divided into a grid where each block in the grid is
associated with different actions to be performed. The weighted
mean position of the handle is calculated to determine the block
of the grid in which the handle lies. As soon as the handle lies in
one of the blocks of the grid, the corresponding action associated
with that block is triggered. This application can be used in
various ways .It can be used to control different media players,
while at the same time it can also be used for exploring files and
folders. One can also use this application while browsing the web.
The basic operations that could be performed using this
application are scrolling the screen up & down, left & right,
selecting a file/folder, going back to previous folder/page,
selecting menu(right click), escaping menu.

biological vision studies and models the physiological

processes behind visual perception in humans and other
animals. Computer vision, on the other hand, studies and
describes the processes implemented in software and hardware
behind artificial vision systems. Interdisciplinary exchange
between biological and computer vision has proven fruitful for
both fields.
One example can be of Color identification. Even an
human infant can be taught to distinguish object based on its
color or shape. Even the trick goes by some animals who can
be trained to identify specific targets. This is one among few
other fundamental points that sparked the idea behind Swipe.

KeywordsImage Segmentation, Webcam, Gesture Control

I.

INTRODUCTION

In the past decades, there had been huge developments in

the field of Computer vision. Computer vision is a field that
includes methods for acquiring, processing, analyzing, and
understanding images and, in general, high-dimensional data
from the real world in order to produce numerical or symbolic
information, e.g., in the forms of decisions. A theme in the
development of this field has been to duplicate the abilities of
human vision by electronically perceiving and understanding
an image. This image understanding can be seen as the
disentangling of symbolic information from image data using
models constructed with the aid of geometry, physics,
statistics, and learning theory. Computer vision has also been
described as the enterprise of automating and integrating a
wide range of processes and representations for vision
perception.
Some strands of computer vision research are closely
related to the study of biological vision indeed, just as many
strands of AI research are closely tied with research into
human consciousness, and the use of stored knowledge to
interpret, integrate and utilize visual information. The field of
________________________________________________________
978-1-4799-7458-0/15/$31.00 2015 IEEE

An effort has been made to simulate few pre determined

actions based on gesture controls. The proposed application is
a real time image processing tool that will be continuously
looking out for specific action(s) through a digital camera.
Based on the same, commands will be generated by the
software that can be mapped to perform various tasks.
Although the application has been made as an assistant to
other applications, it can be used universally to perform any
tasks that can be controlled by arrow keys, escape, backspace,
context menu and carriage return.
Initially a detailed study of available techniques had been
carried out on separating a specific object from the
background. The procedures have been implemented and well
tested in average computational environment. Out of them one
method is chosen based on its overall complexity and ease of
implementation that was felt promising to yield appropriate
result in a real time domain without eating up heavy
computational resources.

II.

LITERATURE SURVEY

Image Acquisition-Before any video or image processing can

commence an image must be captured by a camera and
converted into a manageable entity. This is the process known
as image acquisition. Image acquisition in image processing
can be broadly defined as the action of retrieving an image
from some source, usually a hardware-based source, so it can
be passed through whatever processes need to occur afterward
Image Pre-processing-Pre-processing is a common name
for operations with images at the lowest level of abstraction both input and output are images. The aim of pre-processing is

an improvement of the image data that suppresses unwanted

distortions or enhances some image features important for
further processing. Image pre-processing methods use the
considerable redundancy in images. Neighboring pixels
corresponding to one object in real images have essentially the
same or similar brightness value. Thus, distorted pixel can
often be restored as an average value of neighboring pixels.

as adjacent rectangles, erected over discrete intervals (bins), with

an area equal to the frequency of the observations in the interval.
The height of a rectangle is also equal to the frequency density of
the interval, i.e., the frequency divided by the width of the
interval. The total area of the histogram is equal to the number of
data.

Digital Image Processing-Digital image processing is the

use of computer algorithms to perform image processing on
digital images. As a subcategory or field of digital signal
processing, digital image processing has many advantages
over analog image processing. It allows a much wider range of
algorithms to be applied to the input data and can avoid
problems such as the build-up of noise and signal distortion
during processing

Edge-Detection-An edge or boundary on an image is defined

by local pixel intensity gradient. Edge-based segmentation
represents a large group of methods based on information about
edges in the image. Edge-based segmentations rely on edges
found in an image by edge detecting operators -- these edges
mark image locations of discontinuities in gray level, color,
texture, etc.

Image Segmentation-Image segmentation is the process of

partitioning a digital image into multiple segments (sets of
pixels, also known as superpixels). The goal of segmentation
is to simplify and/or change the representation of an image
into something that is more meaningful and easier to analyze.
Image segmentation is typically used to locate objects and
boundaries (lines, curves, etc.) in images. More precisely,
image segmentation is the process of assigning a label to every
pixel in an image such that pixels with the same label share
certain characteristics.
)
DIFFERENT TECHNIQUES
Thresholding-Thresholding
is a common region
segmentation method. In this technique a threshold is selected
and an image is divided into groups of pixels having values
less than the threshold and groups of pixels with values greater
or equal to the threshold. . The threshold image g(x, y) is
defined as
G(x, y) = {1 if(x, y) > T
0 if(x, y <= T}
The result of thresholding is a binary image, where pixels with
intensity value of 1 correspond to objects, while pixels with value
0 correspond to the background.
Region growing-Region growing method looks for groups of
pixels with similar intensities. It starts with a pixel or a group of
pixels (called seeds) that belong to structure of interest. Seeds can
be chosen by an operator, or provided by an automatic seed
finding procedure. In the next step neighbouring pixels are
examined one at a time and added to the growing region, if they
are sufficiently similar based on a uniformity test(also called a
homogeneity criterion). The procedure continues until no more
pixels can be added. The object is then represented by all pixels
that have been accepted during the growing procedure.
Histogram of an Image-In statistics, a histogram is a
graphical representation of the distribution of data. It is an
estimate of the probability distribution of a continuous variable .
A histogram is a representation of tabulated frequencies, shown

K-Means Clustering Algorithm-K-means clustering is a

method of vector quantization, originally from signal processing,
that is popular for cluster analysis in data mining. K-means
clustering aims to partition n observations into k clusters in which
each observation belongs to the cluster with the nearest mean,
serving as a prototype of the cluster.
Determination of Algorithm
All of the above mentioned algorithms had been tested and
compared for best time as well as space. Out of them, Color
Thresholding seems to be appropriate as per our requirement.
Identifying color is one of the easiest approach in segmenting
out an specific object from the background. We can divide a color
component in its primary domain i.e. for a specific color value
from the video feed, we can extract out three component in terms
of RED, GREEN, BLUE quantitatively.
After forming a cluster, the binary image is further filtered
through a modified version of Gaussian filtering.
And ultimately, a weighted mean determination approach has
been deduced to trace out the approximate centre for the cluster in
a 2 Dimensional grid space.
This centre is further used to calculate the grid the object is
into.

III.

FLOW OF CONTROL

There are specific steps that the application covers while in

execution. The Flow has been discussed with a help of flow

diagram, while the concerned modules are discussed in the

upcoming section.
.

IV.

2.

EXPERIMENTAL RESULTS AND

DISCUSSION

V.
CONCLUSION
In this paper, a spatial fuzzy C-means (SpFCM) clustering
algorithm is proposed with applications to MRI image
segmentation. A probability function is employed in a
membership function corresponding to each pixel to generate
local membership values by utilizing local spatial information
within a square window
centered at that pixel. This
Operation
probability function shows the
degree of belongingness of
Reading an Image / Storing in Buffer
neighbouring pixels to a cluster
with respect to the centered
Thresholding
pixel. Finally, a weighted joint
membership function is defined
by utilizing the local and global
Filtering
membership
functions
to
generate final membership
Cluster mean determination
values and cluster centers. The
algorithm is tested both in
Grid determination
qualitatively and quantitatively
on simulated and real-patient
MRI brain images. The main
TOTAL COMPLEXITY
advantage of the SpFCM
algorithm is that it can produce
superior segmentation results from the MRI images which
have high levels of noise and intensity inhomogeneity. The
experimental results indicate that the proposed SpFCM
algorithm effectively outperforms the FCM and sFCM
algorithms for MRI image segmentation in terms of
suppressing noise, cluster validity functions and tissue
segmentation accuracy.

REFERENCES
1.

J. S. Duncan and N. Ayache, Medical image analysis:

Progress over two decades and the challenges ahead,
IEEE transaction on Pattern Analysis and Machine
Intelligence, vol. 1, pp. 85-106, 2000.

M. A Balafar, A. R. Ramli, M. I. Saripan and S.

Mashohor, Review of brain MRI image segmentation
methods, Journal of Artificial Intelligence, vol. 33, pp.
261-274, 2010.
3.
D. L. Pham, C. Xu and J. L. Prince, A survey of
current methods in Medical Image Segmentation,
Technical report JHU/ECE-99-01, Annual Review of
Biomedical Engineering, 1998.
4.
S. D. Olabarriaga, and A. W. M. Smeulders,
Interaction in the segmentation of medical images: A
survey, Medical Image Analysis, vol. 5, pp. 127-142,
2001.
5.
J. C Bezdek, Pattern recognition with fuzzy objective
function
algorithms,
Complexity
Kluwer
academic
publishers,
O(n2)
Norwell,
MA, USA,
O(n2)
1981.
6.
W. Pedrycz,.

O(n2)
2 * O(n2 + n)
Linear Order
O(n2)

Conditional Fuzzy C-Means,Pattern Recognition

Letters, vol. 17, pp. 625-631, 1996.
7.
Y. A. Tolias and S. M. Panas, Image segmentation by a
fuzzy clustering algorithm using adaptive spatially
constrained membership functions, IEEE Transaction
on systems, man and cybernatics. Part A Systems and
Humans, vol. 28, pp. 359-369, 1998.
8.
N. A. Mohamed, M. N. Ahmed and A. A. Farag,
Modified fuzzy C-means in medical image
segmentation, Proc. of IEEE-EMBS, vol. 20, pp.
1377-1380, 2001.
9.
M. Ahmed, S. Yamany, N. Mohamed, A. Farag and T.
Moriarty, A modified fuzzy C-means algorithm for
bias field estimation and segmentation of MRI data,
IEEE Transaction on Medical. Imaging, vol. 21, pp.
193-199, 2002.
K. S. Chuang, H. L. Tzeng, S. Chen, J. Wu and T. J. Chen, Fuzzy
C-means clustering with spatial information for image Segmentation,
Computerized Medical Imaging and Graphics, vol. 30, pp. 9-15, 2006.