Mobile Augmented

Reality
Hai To
Technische Universität Berlin
Open Communication Systems 1
 Agenda

What is mAR?
➔
A brief history of AR
➔
Definition & Taxonomy
Where to find mAR?
➔
Former approaches
➔
Current use cases
➔
Future Scenarios
How to make mAR?
➔
Software & Hardware
➔
Android for the win
➔
Technology is the key

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 2
 What is mAR?

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 3
 Background

●
AR derived from Virtual Reality
term was coined by engineers at Boeing
➔

(1990, Thomas Caudell & David Mizell)

VR already used in aircraft
➔

manufacturing
(Boeing 777 completely designed through 3D CAD)
➔
AR developed to aid assembling wiring
harness
➔
prototype has been realised with support
of DARPA & Honeywell Military Avionics
Division
➔
AR-system proved to be 25% to 50%
faster than conventional methods

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 4
 Definition

“to augment” = to increase, make larger or

supplement
“reality” = (subjective perception of)
the real world
↳“AR” = extended perception of one's
surrounding through
ancillary
information

●
Augmented Reality is a computing
system:
➔
combining reality with virtual objects
➔
interacting in real-time
Introduction
➔
registered
What is in 3D
Where to How to
Conclusion
mAR? find mAR? make mAR?
(Ronald T. Azuma)

Technische Universität Berlin

Open Communication Systems 5
 Taxonomy

●
Augmented Reality or Virtual
Reality
utilising real vs generating
artificial
surrounding environment

➔
compartmentation depends on the
referenced world model
➔
independent from the perceived images
(Paul Milgram)

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 6
 Where to find mAR?

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 7
 Former Approaches

●
MARS – at Columbia University:
➔
mobile computing system combining
HMD, handheld and wireless
communication technology
➔
developed to help navigate and receive
information about points of interest

➔
motivation: create an advanced, context-
aware, mobile experience; define
reusable interface components
➔
challenges: ergonomics (miniaturisation,
power supply, thermal dissipation);
robustness & outdoor usability;
commercialisation
What is Where to How to
Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 8
 Current Use Cases

●
Location based mAR:
➔
e.g. Wikitude, Layar, TwittARound
➔
utilizes build in GPS and electronic
compass system for tracking
➔
ancillary information for POIs shown
on display

●
mAR Gaming:
➔
e.g. Arhrr, Ghostwire, Treasure Hunting
➔
uses tag recognition for tracking
➔
window to another world

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 9
 Smartphone Apps

TAT Augmented ID

• The Astonishing Tribe

in Malmö, Sweden
• Android based
• uses the Flickr facial
recognition technology
provided by Polar Rose

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 10
 Future Scenarios

U-Cities

* total inter-
connectedness
* largest
integrated
technology
testbeds
* 297 Mio
RFID research
centers
* personal
smartcard for
purchases &
access

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 11
 How to make mAR?

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 12
 Mobile Platforms

●
Software
libraries: ➔
D’Fusion (TI)
➔
ARToolKit (C/C+ ➔
AndAR, mixare
+) ➔
DRAFT (TUM)
➔
Studierstube ES ➔
ARML (Mobilizy)
(C/C++)
1% 2% 8%
8%
●
OS-market- 52%
Symbian
12% RIM

share: Windows M
Mac OS X
Linux
Palm OS
Other
17% Worldwide: Smartphone Sales to
End Users by Operating System,
2008 (Gartner)

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 13
 Android

●
“Softwarestack”
➔
Mobile OS +
Middleware +
Key Applications
●
mAR relevant:
Applications
➔
creation +
➔
Sensor Manager
emulation + ➔
Location
Applications
debugging tools Manager Framework
➔
APIs, APIs, APIs ➔
Surface
➔
free Manager Libraries RT
documentation ➔
OpenGL + SGL
➔
online market API
Linux Kernel
place ➔
Camera +
➔
open source Geomagnetic
Field API
What is Where to How to
Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 14
 Android

●
benefits: concerns:
●

➔
licensing ➔
limited
➔
developer commercial
freedom distribution on
➔
hardware marketplace
options ➔
internationalisat
➔
standardisation ion and
localisation
➔
multi tasking ➔
premature
➔
component
reuse
➔
security
➔
easy
➔
privacy
customizability ➔
fragmentation
➔
big community
What is Where to How to
Introduction ➔
momentum
mAR? find mAR? make mAR?
Conclusion

Technische Universität Berlin

Open Communication Systems 15
 Technologies

●
Key ●
Add-ons:
technologies: ➔
haptic Displays
➔
GPS localization ➔
RFID Readers
(indoor, ➔
3D Display
precision) ➔
acoustic/
➔
Head tracking aromatic
(realtime, augmentations
fluent) ➔
Projector
➔
Display .
(glare, contrast) .
➔
Computer .
(processing,
miniaturization)
➔
Network
What is Where to How to
Introduction Conclusion
mAR?
(QoS, speed) find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 16
 Conclusion

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 17
 Prospect

●
mAR present state: in its infancy
➔
comprehensive research environment
➔
advanced development status
➔
early commercialisation stage
●
barriers: content, critical mass, technology
●
mAR promises: host of everyday
uses
➔
endless virtual information in evanescent
real objects
➔
seamless mergence of reality and
virtuality
➔
personalised information management &
Introduction filtering
What is Where to How to
Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 18
 Challenges

●
social issues:
➔
technological discrepancy →
informational discrimination
➔
privacy, autonomy, liability

●
systematic issues:
lot of challenging systems → difficult to
➔

unify information access

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 19
 Q&A

Mobile Augmented Reality

Virtual Reality, Azuma & Milgram

RV-Continuum,
MARS, Wikitude, Arhhr, TAT AID, U-Cities
wearble vs mobile
ARToolKit, ARML, Studiestube
Android
Symbian, Apple,

key technologies, add-ons

barriers, promises
challenges

What is Where to How to

Introduction Conclusion
mAR? find mAR? make mAR?

Technische Universität Berlin

Open Communication Systems 20