IntentSearch: Capturing User Intention for One-Click Internet Image Search

Presented by

Nayana.K.Raj 0941943 PAACET

26-07-2012

engine which helps to interpret users search intention by using ONE-CLICK query image
Search Uses Text

4 steps for image searching

based information of query word and visual content of query image to expand the image pool

User search intention only by query keywords is difficult because text based image search suffers from.. Ambiguity of query keywords User doesn't have enough knowledge Hard for users to describe the visual content of target images

Easier search by using both textual and visual content of query

Web-scale image search engines mostly rely on surrounding text features. Users search intention by only by query keywords

Image search on the basis of both textual and visual content of images

Image pool is re-ranked based on textual and visual features

Fig. 1: Top-ranked images returned from Bing using apple as query

Key contribution is to capture the users search intention from this one-click query image in four steps.
Adaptive similarity

Keyword expansion
Visual query expansion Image pool expansion

The user first submits query keywords q. A pool of image is retrieved by text-based search User is asked to select the query image from image pool The query image is classified as one of the predefined adaptive weight categories Images in the pool are re-ranked based on their visual similarities to the query image Similarities are computed using the weight schema

Visual feature design

Existing features : Gist , SIFT, Daubechies Wavelet , Histogram of Gradient (HoG)

New features : Attention guided Color Signature, Color spatialet (CSpa) , Multilayer Rotation Invarient ( EOH), Facial Feauter

Adaptive Weight Schema

Weight Lets

schema is used for similarity calculations

take image i and j Adaptive similarity between i & j

Sq(i , j) = fm=1 mq sm(i,j) where sm(i,j) similarity between I and j on feature m f is the visual feature mq is the express the importance of feature m for measuring similarity

Features for query categorization

existence of

faces, the number of faces in the image the image frame taken up by the face region face center relative to the centre of image

percentage of

coordinate of

Directionality

Color Spatial Homogeneousness (variance of values in different blocks of Color Spatialet)

Total energy

of edge map obtained from Canny Operator

Edge

Spatial Distribution

Image Clustering
Image

is divided into clusters Each word wi has ti clusters C(wi)= { ci,1 ,.............,ci,ti }
Visual

distance between the query image and a cluster c is calculated as the mean of the distances between the query image and the images in c.
The

cluster Ci,j with the minimal distance is chosen as visual query expansion and its corresponding word wi . q = wi + q

 User friendly

Easy search for a particular image(on the internet)

Can find the image is real or not

duplicate images

query log data, which provides valuable cooccurrence information of keywords , for keyword expansion Can be improved by including duplicate detection in the future work

Internet image search approach which only requires one-click user feedback

Intention specific weight schema

Without additional human feedback Possible for industrial scale image search by both text and visual content

J. Cui, F. Wen, and X. Tang, Real Time Google and Live Image Search Re-Ranking, Proc. 16th ACM Intl Conf. Multimedia, 2008. J. Cui, F. Wen, and X. Tang, IntentSearch: Interactive On-Line Image Search Re-Ranking, Proc. 16th ACM Intl Conf. Multimedia,2008. Bing Image Search, http://www.bing.com/images http://www.google.com/imagesearch