International Journal of Computer Engineering & Technology (IJCET)

Volume 7, Issue 3, May-June 2016, pp. 4754, Article ID: IJCET_07_03_004

Available online at
http://www.iaeme.com/IJCET/issues.asp?JType=IJCET&VType=7&IType=3
Journal Impact Factor (2016): 9.3590 (Calculated by GISI) www.jifactor.com
ISSN Print: 0976-6367 and ISSN Online: 09766375
IAEME Publication

IMPLEMENTATION OF SASF CRAWLER

BASED ON MINING SERVICES
Rakesh Shastri
Asst Professor, Dept. of Information Technology,
JSCOE, Hadapsar, Pune, India
Shubham Age, Tushar Indorkar, Shital Kokate, Manisha Shitole
UG Students, Dept. of Information Technology,
JSCOE, Hadapsar, Pune, India
ABSTRACT
As we know there are multiple users and they access various websites and
every website contains multiple web pages. It is difficult for the users to find a
document that is equal to their needs. Users browse their request by
submitting their queries to any search engine like Google or Yahoo and it is
has been found that most of the results given by the search engines are
irrelevant most of the time. Web crawlers are most of the important
components used by a search engine that collects web pages from the web. It
is not easy for a crawler to download specific web pages. The ontology,
created from the browsing history, was then parsed for the entered search
query from users and the corresponding results were returned to the user
providing a semantically organized and relevant output. An ontology-based
web crawler uses ontological concepts for improving its performance. So it
will be easy for the users to get relevant data what as per the user search
request. We have focused on self-adaptive semantic focused crawler, SeSM
and StSM algorithm.
Keywords: Ontology Learning, Web Crawler, Heterogeneity, Ubiquity,
Ambiguity.
Cite this Article: Rakesh Shastri, Shubham Age, Tushar Indorkar, Shital
Kokate and Manisha Shitole, Implementation of SASF Crawler Based On
Mining Services, International Journal of Computer Engineering and
Technology, 7(3), 2016, pp. 4754.
http://www.iaeme.com/IJCET/issues.asp?JType=IJCET&VType=7&IType=3

http://www.iaeme.com/IJCET/index.asp

47

editor@iaeme.com

Rakesh Shastri, Shubham Age, Tushar Indorkar, Shital Kokate and Manisha Shitole

1. INTRODUCTION
Internet has become the most convenient way to search the information, book pdf,
softwares, online shopping etc. User needs fast, accurate and easy search results
according to their requirements. But if it takes too much time to look for the accurate
results, then user may get frustrated. When user submit his query to the search engine
then search engine displays number of results related to the users query, many times it
happens that user searches number of pages but still dont get the exact information he
wants. So, to get easy and accurate search results we have developed a web
Application called SASF- self Adaptive semantic focused Crawler which will help the
user to get the exact results that they want and within a short period of time. The idea
behind developing this application is to save users time and make search easy for him
to get accurate results within limited pages. Our application will have an external
called notification feature i.e. when user is searching on our application and the
information and data is not available at that time then it will notify the admin to
upload the required data and as soon as admin uploads it then it will notify the user
that data or information is available now. We are using the concept of ontology
learning with SASF crawler [13].
It is decently perceived that data innovation has a significant impact on the way
business is led, and the Internet has turned into the biggest commercial center on the
planet. Inventive business experts have understood the business applications of the
Internet both for their clients and key accomplices, transforming the Internet into a
colossal shopping center with a colossal inventory. Purchasers have the capacity scan
a tremendous scope of items and administration notices over the Internet, and
purchase these products specifically through online exchange frameworks.
Administration notices structure an impressive piece of the publicizing which happens
over the Internet and have the accompanying peculiarities [12][4][10][11]:

A. Heterogeneity

Figure 1 Heterogeneity

Today internet is used by every person or organization to advertise their product,

so the internet contains data of lots of different backgrounds i.e. Heterogeneous.
Given the diversity of services in the real world, many schemes have been proposed
to classify the services from various perspectives, including the ownership of service
instruments, the effects of services and the nature of the service act and many more.
There is not a single scheme that is agreed publicly and available for classifying
services over the Internet. Many commercial product and service search engines
provide classification schemes of services with the purpose of facilitating a search,
they do not really distinguish between the product and the service advertisement;
instead, they combine both into one taxonomy.

http://www.iaeme.com/IJCET/index.asp

48

editor@iaeme.com

Implementation of SASF Crawler Based On Mining Services

B. Ubiquity

Figure 2 Ubiquity

Ubiquity is the property of being present everywhere. Service providers can

register any service advertisements with the help of various service registries,
including global business search engines, such as business.com, local business
directories such as Google and JustDial, search engine advertising, such as Google
and Yahoo!. These service registries are widely distributed over the internet. There
are lots of people / organization working on same domain will have their data on the
internet, but service user knows only about the domain so he can get confused with so
much repetition of data on internet.

Ambiguity

Figure 3 Ambiguity

Ambiguity is nothing but the uncertainty in which several interpretations are

possible. So much use of internet for users own benefit, user who use the service will
struggle to find exactness in data i.e. data is ambiguous. It is an attribute of any idea
or statement whose meaning cannot be completely resolved according to a rule with
finite number of steps. Many online service advertising information is embedded in a
large amount of information on the WWW and is described in natural language,
therefore it may be ambiguous. Moreover, online service information does not have a
consistent and standard format, which varies from Web page to Web page.

2. RELATED WORK
Consumer privacy concerns about Internet marketing: They have outlined a
taxonomy that helps to describe, categorize, and analyse consumer privacy concerns.
They have also reviewed the current state-of-the-art technology and also pointed out
the integration of business self-regulation, regulated enforcement law and the users
ability to enhance individual privacy protection through the use of technology [3].
Focused crawling for automatic service discovery, annotation, and classification
in industrial digital ecosystems: There was an crucial issue in Digital Ecosystem to
resolve this issue, in this paper they present a conceptual framework for a semantic
focused crawler, with the use of automatically discovering, annotating and classifying
the service information with the Semantic Web technologies [2].

http://www.iaeme.com/IJCET/index.asp

49

editor@iaeme.com

Rakesh Shastri, Shubham Age, Tushar Indorkar, Shital Kokate and Manisha Shitole

A service search engine for the industrial digital ecosystems: There was an issue
between service providers and service requesters to solve this issue, they had design a
conceptual framework of ontology based semantic service search engine. Apart from
these function with a novel search model, these framework provides a quality-ofservices (QoS)-based service evaluation and ranking methodology. To evaluate the
feasibility of the framework, they implemented a prototype in the transport service
domain, and compare the performance of the search models [1].
SHARDIS -A privacy-enhanced discovery service for RFID-based product
information: They had introduces SHARDIS, a privacy-enhanced discovery service
for RFID information based on the peer-to-peer paradigm. Their idea was to enhance
confidentiality of the clients query against profiling by cryptographically hashing the
search EPC by splitting and distributing the service addresses of interest [4].

3. SYSTEM ARCHITECTURE

Figure 4 system Architecture

In the existing system, we found out that users are unable to get proper service
information and data service facing the problems of heterogeneity, ubiquity, and
ambiguity.
So we are proposing a self-adaptive semantic focused crawler i.e. SASF crawler,
with the purpose of accurate and efficient discovery of information related to service
over the Internet, by dealing with the three major issues. Our aim is to recover the
drawbacks from the existing system for this we are going to use SASF crawler with
Ontology learning with the help of which whenever user will submit his query to the
search engine he will get the relevant data related to his search. Here we present the
idea of Self Adaptive semantic focused crawler to give users the best way to search
and get the required results. We have focused here to provide the users with the exact
search results within minimum page. Our system consists of Users and Admin the
work of admin is to provide necessary data to the users. Users will register first to our
application to get full access of the data. As user input his query to the application it
will search into the database for the required results. SASF crawler will fetch the data
exactly matching with the user query and displays the results to the user. In addition
with this we have provided a notification feature so that if the required data is not
present in the database it will notify the admin to upload necessary data.
We overcome the drawbacks of the existing system and providing users with the
exact results. We used Semantic crawler and ontology learning to achieve our goals.
Users will find it easy to get the data in our application.

http://www.iaeme.com/IJCET/index.asp

50

editor@iaeme.com

Implementation of SASF Crawler Based On Mining Services

4. ALGORITHMS
Semantic based String matching algorithm.

Figure 5 SeSM algorithm flow.

The major goal of this algorithm is to compare semantic similarity between

concept description (user search) and service description (ie. Already present in
database). Here we compare similarity between what user is expecting and what there
in database [13].
Here w is contains pair of (p,q)
Where p = user input.
q= data in database.
p = {p1,p2,p3..}
q = {q1,q2,q3..}
p1 is compared with q1,q2,q3
p1 is matching with q with rate 1.0(look in above diagram) greater than
q2(0.9),q3(0.7),
so for p1 i/p q1 is the o/p
same matching performed for all i/p.
Statistic based string matching algorithm.
StSM algorithm is a complementary solution for the SeSM algorithm, in case the
latter does not work effectively in some circumstances.
If above SeSM fails we use StSM. In StSM we check search history of user what
he has searched in past, if suppose For p1 we have q1(1.0) and q2(1.0) and q3(1.0)
All matching rate are similar so we use statistical approach and check if q1 has been
searched by user lot more times in past search history than q2, q3 then we set o/p of
p1 as q1[13].

Steps for Semantic and Statistic algorithm

I/P user search keyword.
O/p search result.
Step1: user enters the search keyword
Step2: check for user search history and get ontology list of keyword
Step3: if list is null or zero

http://www.iaeme.com/IJCET/index.asp

51

editor@iaeme.com

Rakesh Shastri, Shubham Age, Tushar Indorkar, Shital Kokate and Manisha Shitole

Show the result to user with matching

result
Step4: if list is not null or not zero process step5
Step5: get list of keywords, user search
keyword, past search history (ontolist)
Step6: set the threshold value for max
similarity.
Step7: put keyword list, search keyword, onto term into the new list.
Step8: compare new list elements with each other, if matches increment the count.
Step9: if match count is equal or greater than threshold value.
Step10: display the search result to user.

5. TEST RESULTS
1. Login module
Table1 Test cases for Login module

2. Register module
Table2 Test cases for Register module

http://www.iaeme.com/IJCET/index.asp

52

editor@iaeme.com

Implementation of SASF Crawler Based On Mining Services

3. Database
Table3 Test cases for database

6. TEST OUTCOMES

The algorithm for searching is giving the exact results according to the user queries.
Using the concept of web crawler and ontology learning searching is improved.
All the irrelevant pages will be discarded.

7. CONCLUSION
We have presented our idea of using SASF crawler with ontology learning to give
users the exact results according to their needs. Searching will be quick and easily
available to the user.

ACKNOWLEDGMENT
I would like to express special appreciation and thanks to my Guide Prof. R.V.Shastri
for mentoring me. I would like to thank Prof. P. D. Lambhata for encouraging my
research work and for allowing me to grow as a research scholar.

REFERENCES
[1]
[2]

[3]

[4]

[5]

[6]

H. Dong, F. K. Hussain, and E. Chang, A service search engine for the industrial
digital ecosystems, IEEE Trans. Ind. Electron., 58(6), pp. 21832196, Jun. 2011.
H. Dong and F. K. Hussain, Focused crawling for automatic service discovery,
annotation, and classification in industrial digital ecosystems, IEEE Trans. Ind.
Electron., 58(6), pp. 21062116, Jun. 2011.
Huaiqing Wang, Matthew K.O. Lee, Chen Wang, Consumer privacy concerns
about Internet marketing in COMMUNICATIONS OF THE ACM March
1998/Vol. 41 , pp. 6370.
Benjamin Fabian,Tatiana Ermakova, and Cristian Mller, SHARDIS -A
privacy-enhanced discovery service for RFID-based product information, IEEE
Transactions On Industrial Informatics, 8(3), August 2012, pp. 707718.
Swati Ringe, Nevin Francis, Palanawala Altaf H.S.A., Ontology Based Web
Crawler, International Journal of Computer Applications in Engineering
Sciences II (III) September 2012, pp. 194197.
Manvi, Ashutosh Dixit, Komal Kumar Bhatia, Jyoti Yadav, Design and
Implementation of Domain based Semantic Hidden Web Crawler, International

http://www.iaeme.com/IJCET/index.asp

53

editor@iaeme.com

Rakesh Shastri, Shubham Age, Tushar Indorkar, Shital Kokate and Manisha Shitole

[7]

[8]

Journal of Innovations & Advancement in Computer Science IJIACS ISSN 2347

8616 Volume 4, Special Issue May 2015 pp. 73-84.
J. Madhavan, D. Ko, L. Kot, V. Ganapathy, A. Rasmussen, A. Halevy, Googles
Deep Web Crawl. In proceedings of Very large data bases VLDB endowment,
pp. 12411252, Aug. 2008.
H. Dong, F. Hussain, and E. Chang, O. Gervasi, D. Taniar, B. Murgante, A.
Lagana, Y. Mun, and M. Gavrilova, Eds., State of the art in semantic focused
crawlers, in Proc. ICCSA 2009, Berlin, Germany, 2009, 5593, pp. 910924.

[9]

W. Wong, W. Liu, and M. Bennamoun, Ontology learning from text: A look

back and into the future, ACM Comput. Surveys, 44, pp. 20:136, 2012.

[10]

M. Ruta, F. Scioscia, E. D. Sciascio, and G. Loseto, Semantic-based

enhancement of ISO/IEC 145433 EIB/KNX standard for building automation,
IEEE Trans. Ind. Informat., 7(4), pp. 731739, Nov. 2011.

[11]

I. M. Delamer and J. L. M. Lastra, Service-oriented architecture for distributed

publish/ subscribe middleware in electronics production, IEEE Trans. Ind.
Informat., 2(4), pp. 281294, Nov. 2006.

[12]

Mr.Muneerkhan Aslam Bandar and Prof. Y.B.Gurav, Implementation of Hsasf

Crawler for Information Discovery, International journal of scientific research,
4(8) Aug 2015,ISSN No 2277 8179.
Shobha B. Patil, S.K. Shirgave, Enriching Search Results Using Ontology,
International Journal of Computer Engineering and Technology, 4(2), 2013, pp.
500507.
Sinan Adnan Diwan Alwan, Dr. Enas Hadi Salih and Ammar J.Fatah, Ontology
Based Java Platform Personalization to Host Environment, International Journal
of Computer Engineering and Technology, 5(7), 2014, pp. 0110.
Jaytrilok Choudhary and Devshri Roy, Priority Based Focused Web Crawler,
International Journal of Computer Engineering and Technology, 4(4), 2013, pp.
163169.
P. Resnik, Semantic similarity in a taxonomy: An information-based measure
and its application to problems of ambiguity in natural language, J. Artif. Intell.
Res., 11, pp. 95130, 1999.

[13]

[14]

[15]

[16]

http://www.iaeme.com/IJCET/index.asp

54

editor@iaeme.com