Mobile Visual Search: Architectures, Technologies, and the Emerging MPEG Standard

Girod, B. Chandrasekhar, V. Grzeszczuk, R. Reznik, Y.A.

IEEE Multimedia, VOL.18, NO.3, March 2011

Reporter : han-nung , hsu Data : 2012/03/07

2012/3/7

mclab

OUTLINE

Introduction Mobile Image-Based Retrieval Technologies Performance of Mobile Visual-Search Systems Emerging MPEG Standard Conclusions & Outlook

2012/3/7

mclab

Introduction

As mobile phones have evolved into powerful device, enabling a class of applications that use the phones built-in camera to initiate search queries about objects in the users visual proximity.

2012/3/7

mclab

Introduction

What part of the processing should be performed on the mobile client, and what part is better carried out at the server? The processing on the mobile device must be fast and economical in terms of power consumption. The algorithms used for retrieval must be scalable to potentially very large databases, and capable of delivering accurate results with low latency.

2012/3/7

mclab

OUTLINE

Introduction Mobile Image-Based Retrieval Technologies Performance of Mobile Visual-Search Systems Emerging MPEG Standard Conclusions & Outlook

2012/3/7

mclab

Mobile Image-Based Retrieval Technologies

Most successful algorithms for image-based retrieval today use an approach that is referred to as Bag Of Features (BoF) or Bag Of Words (BoW).

2012/3/7

mclab

Mobile Image-Based Retrieval Technologies

Feature extraction
Scale invariance rotation invariance

2012/3/7

mclab

Mobile Image-Based Retrieval Technologies

Feature extraction
Scale Invariant Feature Transform (SIFT) Speeded Up Robust Features (SURF) Compressed Histogram Of Gradients (CHoG)

2012/3/7

mclab

Mobile Image-Based Retrieval Technologies

Feature indexing and matching

Vocabulary Tree (VT)

2012/3/7

mclab

Mobile Image-Based Retrieval Technologies

Geometric verification

2012/3/7

mclab

10

OUTLINE

Introduction Mobile Image-Based Retrieval Technologies Performance of Mobile Visual-Search Systems Emerging MPEG Standard Conclusions & Outlook

2012/3/7

mclab

11

Performance of Mobile Visual-Search Systems

For evaluation, we use a database of one million CD, DVD, and book cover images, and a set of 1,000 query images with a range of photometric and geometric distortions.

Device:
Nokia 5800 mobile phone (with 300-MHz CPU) Linux server (with a Xeon E5410 2.33-GHz CPU & 32 Gbytes of RAM)

2012/3/7

mclab

12

Performance of Mobile Visual-Search Systems

2012/3/7

mclab

13

Performance of Mobile Visual-Search Systems

2012/3/7

mclab

14

OUTLINE

Introduction Mobile Image-Based Retrieval Technologies Performance of Mobile Visual-Search Systems Emerging MPEG Standard Conclusions & Outlook

2012/3/7

mclab

15

Emerging MPEG Standard

Ensure interoperability of visual search applications and databases, Enable a high level of performance of implementations conformant to the standard, Simplify design of descriptor extraction and matching for visual search applications, Enable hardware support for descriptor extraction andmatching inmobile devices, and Reduce load on wireless networks carrying visual search-related information.

2012/3/7

mclab

16

OUTLINE

Introduction Mobile Image-Based Retrieval Technologies Performance of Mobile Visual-Search Systems Emerging MPEG Standard Conclusions & Outlook

2012/3/7

mclab

17

Conclusions & Outlook

In the near term, the availability of standard data sets and testing conditions is expected to foster competition and collaboration to develop a best-ofbreed solution.

Longer term, the existence of a widely accepted standard will provide the certainty for industry to make major investments in mobile visual-search technology that will lead to a broad deployment.

2012/3/7

mclab

18

Thank you

2012/3/7

mclab

19