2011 Third International Conference on Intelligent Human-Machine Systems and Cybernetics
A SOA-based Fish Recognition System Prototype
Xitao Zheng, Junsheng Zhong, Yongwei Zhang
College of IT, College of Fisheries and Life Science
Shanghai Ocean University, Shanghai 201306, China
xtzheng@shou.edu.cn, Zhangyongwei_108@163.com
Abstract - This paper presents a new system to automatically
recognize fishes based their digital pictures. The system follows
the requirement of SOA (Service Oriented Architecture) in order
to provide a self-service platform for public. The system provides
services like the training mode, the recognition mode, and the
consulting mode. These services can be deployed onto various
application servers at different locations in the world and serve as
the interface system of fish knowledge retrieving and
collaboration. The system services utilize the previous works on
fish morphological ontology construction and digital picture
recognition for carps and other species. Both user and expert can
upload a training fish picture and the name of the fish, the system
will decompose the picture into properties and compose them into
owl with the given fish name. In recognition mode the fish picture
will follow the same procedure and be compared with stored
features to get the fish name and then the other properties and
relations. The system prototype is implemented using web
services from Tomcat. The fish images and owl repository are
saved in MySql database and ontologies are stored in OWL files.
Keywords: SOA, ontology, fish recognition, Web services
I. INTRODUCTION
In order to find a fish property over the internet, the fish
name is necessary. A vague fish name will lead to uncertain
results. For an expert, it is possible to search fish pictures and
descriptions over the Wikipedia or fishnet, read over a lot of
pages of words and picture to determine what a given fish will
belongs to; for a normal user, he/she may not have the expertise
to read through all the links and keywords to reach the
destination. Our solution is to develop a picture based
searching system which can recognize a fish by its looking, and
then translate the recognition into a few keywords, and search
the existing knowledge database with the keywords to get
expected information.
As we know, the recognition of a digital image is highly
dependent on its shooting quality. For a given fish, different
shooting angles, illumination condition and camera resolution
will produce a picture with great difference in its pixel density
and pixel grayscales. When a low quality photo is uploaded to a
system, the system can check the quality first and issue
instructions to retake the photo if minimum requirement is not
met. Also, if the architecture is flexible, the database can be
enriched and weight can be given on different image qualities.
So the fuzzy neural network system can be applied to organize
the fish picture database. Here we assume all the fish pictures
are prescreened by computer and stored in a database. The
failed pictures that have passed the prescreening but are
rejected by feature retrieving and/or matching system will also
be stored in the database and can be periodically reviewed and
maintained by system administrator or an expert. This kind of
prototype can be helpful to build up fish pictures and
978-0-7695-4444-1/11 $26.00
$25.00 © 2011 IEEE
DOI 10.1109/IHMSC.2011.86
recognition experiences. To enable the quick deployment and
global participation of the project, SOA architecture are
adopted, the services and database can be deployed on multiple
servers around the world and service repository can be enriched
by all the participants.
Before elaborating our job, we will have a short review of
our existing jobs and related works in the area.
II. RALATED WORKS
Service-oriented architecture (SOA) is a flexible structure
to design system architecture and integration. A system based
on a SOA will package functionality as a suite of interoperable
services that can be used within multiple, separate systems
from several business domains. The geologically diversified
location of fishes and users require a service based, not just
computer based platform. There are a lot SOA system
deployed around the world, here we just list a few.
Reference [1] presents the Living Human Digital Library
(LHDL) project, which takes advantage of Web Services to
build a biomedical digital library infrastructure that allows
clinicians and researchers to preserve, trace, share data
resources, and to collaborate at the data-processing level. The
paper focuses on transferring existing systems into services.
To utilize the divertive and collaboration feature of SOA,
reference [2] describes a SOA-based precision irrigation
decision-making system which can quickly build a decision
model to meet the needs of different users.
Apart from the architecture side, the service
implementation takes advantage of the previous jobs on fish
ontology and picture recognition [3, 4], and the illumination of
other similar systems [6-8].
For the fish image recognition, there are some papers that
use pattern recognition to do fish identification from a given
set, fish counting, ocean color analysis to find fishing area by
the satellite images, etc. Because of the huge amount of fish
species, there is no new definition of fish ontology so it can be
used for pattern recognition and classification system.
Reference [3] presents a new method to define and obtain
morphological ontology features for carp using its digital
images. The method will be based on digital photograph
processing to analyze the pictures of the fish and a number of
geometrical features of the fish are defined as its ontological
features.
Reference [4] moves further on [3] to make use of these
morphological features. The ontologies based on the
morphological characteristics of the silver carp are defined,
obtained, and compared with those of its family. The
constraint, which is the existence of a certain features or the
66
range of a certain features, is defined for silver carp, common
carp and snakehead fish. The digital geometric calculation is
formalized with pattern recognition and other digital image
processing technique; shared and distinguished features are
discussed for three kinds of samples.
Based on the works on [3] and [4], the author proposed a
method [5] based on ontology formation and fuzzy recognition
of digital pictures. The method provides automatic creation of
the fish geometric ontology and automatic indexing to existing
Semantic Web. Fuzzy set and fuzzy recognition are used to
decide if a new fish picture belongs to an existing training set
or not. Carp is used as an example.
Another example [6] is the works from AOS
(Agricultural Ontology Services) project. The project is
constructing a primitive ontology in fishery area. The ontology
consults ASFA (Thesaurus Aquatic Sciences Thesaurus),
ARGOVOC thesaurus with multiple languages, oneFish list of
keywords, oneFish Topic headings/titles and FIGIS, and other
thesaurus of fishery or fishery related resources. Plenty of
existing jobs for fish information system are organized by
AOS projects.
Reference [7] uses a computer vision machine (The Catch
Meter) to identify and measure different species of fish on the
conveyor, which provide a comparable prototype for the
business layer definition.
So our system will be the combination of the SOA
architecture, the service collaboration and integration of
previous software modules.
III. OUR METHOD
Web Services make it possible to interoperate fish
ontology services across the Internet, though it still remains a
challenge to build a cooperative database environment so the
data can be stored and analyzed in multiple networked
locations. There are ways to handle this issue manually, but
automatic network database updating and synchronizing is still
a problem for a lot of companies, especially when we are
using MySql as our server database engine. There will be
future issues like too many pictures or possible internet attacks
may make the database too big to handle. So the web service
enabled SOA system has more benefit than problems, here are
a list of benefits:
(1) Searching and retrieving resources effectively. The
system is server based and all the image preprocessing and
ontology retrieving are performed on the server, only the
searching results are send back, and the searching using
ontology properties and relations are more effective than harsh
table or other database searching techniques. This is a benefit
of semantics web and also the benefit of SOA structure.
(2) Sharing the algorithm, not just the data. While the
current morphological ontologies are defined and implemented
by our group, the system is open to all contributors so all the
internet users may have the chance to participate in the service
development and upload their services. This is a typical
feature of our system because it targets the public service and
information sharing and technology reuse. It is crucial to be
67
able to expose and share functions provided by such
applications.
(3) Providing efficient and flexible tools for transforming
digital pictures into various set of morphological ontologies,
include definition of new features. This will avoid the problem
when a lot of fish species are input, the existing features will
overlap each other. So new rule and features can be added to
transform the pixel data into meaningful data for pattern
matching. The services of such kinds of transaction are also
open to be added.
(4) Making the good use of our services. The business
layer is open to be re-organized so the services can be reusable
over the internet. There will be technical concerns like the
operations are located in one place and it is operated in another
locations with a completely different hardware/software
platform.
(5) Storing and possibly transferring a large amount of
data. This is because all the pictures the system handled will
be stored in a database, so the data will be a benefit and a
problem, if the database storage is distributed over the internet,
the data synchronization can be complicated. This, as we
mentioned before, will be left for future. The usual picture size
is about 3Mb, if only a few hundred users visit the system each
day, the daily storage increase can be Gb size. So future work
will be focused on database management, data re-organizing
and indexing.
A. System design with SOA
The SOA structure is usually composed of four layers:
the presentation, the business, the service and the data layer.
This is the detailed illustration of these layers:
噝 Presentation layer: Provides user-friendly interaction
interface, takes user’s input, and displays the feedback from
the server. Here the presentation layer is accomplished by
Dreamweaver with JSP
噝 Business layer: Processes business logic, composes
business with several business processes, business function
that calls granularity services.
噝 Service layer: granularity services layout, basic
functioning unit, can pass and receive message from all
directions (upper levels, same levels, or lower levels). Service
repository is stored in the database so the business layer can
visit and get the calling details.
噝 Data layer: Uses a relational database handling, saving
data including multimedia data and repository data.
Fig 1 shows the fish recognition system architecture,
where the presentation layer is the web browser, and the
business layer is the EJB that interoperate with service
repository, and the service listed all the services. Data layer
contain the database management and database storage.
B. Service Analysis
As we know, the fish image needs to be preprocessed:
remove noises, adjust direction, turn to gray-scale and edge
sharpening. There are two kinds of services we are currently
using: the preprocessing service, which include the processes
that are described in the above description; and the ontology
retrieving services, which are digital measurement actions to
get the size, angle, texture, and a lot of other features that are
described in [3-5]. These features are not mandatory and new
features can be added anytime on the basis that the feature can
be automatically retrieved by computer, so the related
automatic retrieving algorithms need to be wrapped with
services and deployed to our application server.
There are five kinds of services in our system as shown in
Fig 1:
Image preprocessing service, remove the background
noises, adjust the fish direction, change the color images to
gray-scale and finally take the edge detection using Canny
operator
Image training service, image saved to database with
serial number and annotation field, morphological ontology
generation service will be called, OWL service will be called
to validate and save ontologies.
Image recognition service, same as image training, OWL
service will be called to retrieve ontologies to perform feature
matching.
Morphological ontology generation service, global and
local sizes will be measured and stored for OWL processing.
OWL service, saving or retrieving of morphological
ontologies and feature validation and cooperation with lexicon
ontologies.
C. System Flowchart
From Fig 2 we can see the system flowchart of our fish
recognition system. The user will choose from the browser to
decide if to send a known fish picture for training (usually
expert or faculty) or for recognition, the JSP will call a system
service to upload the picture. The picture will be prescreened
and then preprocessed to meet the requirement of
morphological ontology generation, if the request is training,
the ontologies will be validated and incooperated with the
ontology of the same fish name and stored in OWL file. If the
Fig1 Fish recognition system architecture
68
request is recognition, the action will be ontology retrieving
from OWL file and feature matching will be applied. Then the
image will be stored in database with serial number and fish
name (recognized or given), necessary information will be sent
back to the browser. If a recognition request failed, the picture
will be tagged with fail and be left for expert to process, which
is the consulting mode.
IV. SYSYTEM IMPLEMENTATION
The system is developed with Tomcat 6.0 on the server
side and Dreamweaver + JSP on the client side. The system
has processed a dozen fish species and the results are satisfied.
The current processing limit the fish image to side picture only,
with clean background and good illumination. The system user
needs to register in order to prevent abuse.
Fig 3 shows the return information when a silver carp is
uploaded to be recognized and the recognition is successful,
The picture on the top right corner is the trained fish picture
and the picture on the top left is the picture is one that we
upload. Although the two fish have different colors, the
matching rate is 90%. We will classify a matching if 75%
percent of more of all the 15 features fall in 10% tolerance
area. In our current models color is not a feature, so the
matching is unbelievingly higher than expected. In some cases
when the color is the only factor to differentiate two samples,
color need to be added as an ontology, currently we can not
handle these kinds of samples.
As we mentioned just now, the current fish recognition
system can only process the side image of fish with a clean
background. This is because that for most fishes their side
pictures provides a better view of their geometrical features.
As the system improves, we need to add multiple view of a
fish, like the top view (over the back), the bottom view (over
the stomach), and the head view (over the head). Also, the
imaging instruction is required and can be intelligently popped
up if the uploaded image failed the prescreening. This is why
we call it a prototype because the current work is still at the
beginning of the fish recognition, a lot of new algorithm and
services need to be developed to handle the hundreds of
thousands of fish species. Also, chemical, cell and genetic
features can be added or linked into the system so the
researchers can share the very expensive feature extraction by
Fig2 System flow chart
Fig 3 The returned information when a match is found
69
a click of the picture. Also, for those that can be efficiently
identified by pictures, suggestions can be presented to help
user to do further identification job.
CONCLUSION
One of the major advantages of our Fish Recognition
System over the other fish ontology systems is that it
preserves and shares not only original pictures of the fish but
also the data processing and visualization algorithms. The
adoption of Web Service technology in the system has
successfully contributed to this aspect. Breaking a recognition
system into loosely coupled services and enable the
redistribution of the data will enable the participation of the
project and the public fish information service. This will
minimize the possible infrastructure requirement for an image
servicing system.
The implementation of morphological and lexicon
ontology services will help the building up of the knowledge
base and also provide the possibility of automatic importing
from existing fish knowledge base.
A published service structure is important prerequisite of
the global participation and contribution to the project.
So the SOA-based fish recognition system can
successfully handle the image uploading, preprocessing,
feature extraction and matching, the knowledge correlation
and presentation. The system will help the global sharing of
the fish information.
REFERENCES
[1] Xia Zhao, Enjie Liu, Gordon J. Clapworthy, Marco
Viceconti, Debora Testi. SOA-based digital library
services and composition in biomedical applications.
Computer Methods and Programs in Biomedicine, In
Press. Available online 17 September 2010.
[2] Liyuan Xu, Liping Chen, Tianen Chen, Yunbing Gao.
SOA-based precision irrigation decision support system.
Mathematical and Computer Modeling, In Press,
Available online 4 December 2010.
[3] Yongwei Zhang,Xitao Zheng, Junsheng Zhong. A New
Morphological Ontology Method for Carp Based on
Digital Image Measurement. Web Information Systems
and Mining, 2010, page23-27.
[4] Xitao Zheng, Yongwei Zhang, Junsheng Zhong. An
Ontology Method for Silver Carp Auto-Recognization
Based on Digital Image. Web Information Systems and
Mining, 2010, page331-334.
[5] Xitao Zheng, Yongwei Zhang. A Fuzzy Recognition
Method for Fish Ontology Retrieving. Journal of
Measurement Science and Instrumentation, 2010 Vol.1
Supplement
[6] Qian ping, Zheng Yelu. The Research and Application of
Agricultural Ontology. China agricultural science and
technology press, 2006, Pages 51-52.
[7] D.J. White, C. Svellingen, N.J.C. Strachan. Automated
measurement of species and length of fish by computer
vision. Fisheries Research, Volume 80, Issues 2-3,
September 2006, Pages 203-210.