Virtual Reality (Artificial Reality)

Definition
A computer system used to create an artificial world in
which the user has the impression of being in that world
and with the ability to navigate through the world and
manipulate objects in the world.

-C. Manetta and R. Blade
Motivations
A display connected to a digital computer gives us a
chance to gain familiarity with concepts not realizable
in the physical world. It is a looking glass into a
mathematical wonderland.

The ultimate display would, of course, be a room
within which a computer can control the existence of
matter.

- Ivan E. Sutherland
Two Requirements of VR
Immersive

Interactive
Two Related Courses of VR
Computer Graphics

Computer Vision
Two Ways to construct VR

(Geometry-based) (Image-based)

3D

Two Types of VR
Immersive : 3D environment seen through a head-mounted
display (HMD). In a completely immersive system the user has
no visual contact with the physical world.

Semi-immersive : Most advanced flight, ship and vehicle
simulators are semi-immersive. The cockpit, bridge, or driving
seat is a physical model, whereas the view of the world
outside is computer-generated (typically projected).

Two Parts of VR
Augmented Virtuality

Augmented Reality
Adventure games, MUD(Multi-Users Dungeon) : Textually
described virtual worlds where the user perceives the virtual
environment through mental images like reading a novel.

Desktop : 3D virtual environment graphically displayed on a
desktop computer monitor.
Evolution of VR
Evolution of
VR

Projected : 3D environment projected onto a screen. Enables
a single user to demonstrate concepts to a group of people. A
CAVE (developed by University of Illinois), where several
screens are used to surround the user with images, is the most
advanced form of projected VR in use today.
VR Technology
Hardware capable of rendering real-time 3D graphics and
high-quality stereo sound.

Input devices to sense user interaction and motion.

Output devices to replace user's sensory input from the
physical world with computer-generated input.

Software that handles real-time input/output processing,
rendering, simulation, and access to the world database in which
the environment is defined.
Can display either stereo or mono
images
Stereo images (binocular
disparity)
Same image twice (binocular
concordance)
Single image (uniocular)
May be totally immersive or semi-
immersive (see-through)
May include a built-in head-
tracker
May have built-in stereo
headphones
Head Mounted Display (HMD)
Each eye
sees a
different
field of
view
LCS (Liquid Crystal Shutter)
glasses
Display shows left and right
images alternately, switching at
high speed between images

Stereoscopic image is seen when
the display is viewed with special
glasses

Typical 'Fishtank VR'
Particularly good for large
audiences in a theatre
Alternative Displays
BOOM(Binocular Omni-Orientation Monitor) by Fakespace
Uses a CRT to provide high-resolution display
It is comfortable to use, since it does not have to be worn
Has fast, accurate, built-in tracking
Alternative
Displays
Alternative
Displays
VRD (Virtual Retinal Display)

Image is projected directly
onto the retina (by Microvision)
Mechanical
Usually a mechanical arm attached to the tracked
object
Very accurate, short lag, but restrict movement

Electromagnetic
Measures strength of magnetic fields in coils
attached to objects
Fast, short lag, but often prone to interference
Limited range
Motion Tracking Types
Motion Tracking Types
Optical
Typically, pulsating LEDs monitored by a camera
at a fixed position
Fast, reasonably short lag, but often prone to
interference caused by ambient lighting
conditions
Line of sight problems

Acoustic
Use ultrasound waves to measure position and
orientation
Slow and often imprecise
3D Mice

Spaceballs

Eye tracking

Video camera and shadows

Voice recognition

Biological sensors
Interaction Devices
Interaction Devices
Light Pen
Data Gloves
Interaction Devices
Data Gloves

Hand and gesture tracking

Enables natural interaction with objects

Can use hand-signs to execute actions

Full body suits

Motion capture
Interaction Devices
Feedbacks
Motion platforms

Tactile feedback provides a sense of touch through,
typically, vibrating nodules or expanding air bubbles
inside a glove or suit

Force feedback provides physical constraints

Exoskeletons

Joysticks, Hand controllers
Sound
Important to create a sense of atmosphere

Can greatly enhance feeling of presence

Can be used to provide valuable depth cues,
aiding navigation

Enables the user to perceive events that occur
outside the immediate field of view

Audio feedback
Need modeling tools to create objects (AutoCAD,
3D Studio, etc.)

Designing objects is time-consuming

Objects often need to be optimized for VR use

Software
Applications
1. Architecture

2. Design and Prototyping

3. Education & Conferencing

4. Training

5. Medical

6. Business and Visualization

7. Art and Leisure
Architecture
Walkthroughs to evaluate design decisions and
present designs to customers

Demonstrate how a planned construction fits into
the environment in which it is intended to be built
Use to create rapid prototypes rather than make clay
models or full-scale mock-ups

Simulate assembly lines. For example, to evaluate
the risk of interference or collision between robots,
cars, and car parts
Design and
Prototyping
Education & Conferencing
Education
Visualize concepts
Visualize the past ('Virtual
Heritage')
Virtual lectures and
classrooms

Conferencing
Collaborative work over the
Internet
Virtual work groups
Virtual conferences
Civilian and
military training
simulators

Driving
simulators

Flight simulators

Ship simulators

Tank simulators
Training
Training
Train for hazardous or difficult operations
Nuclear plant maintenance
Learn to move in zero gravity
Practice locating and fixing faults in equipment
Surgery
Practice performing surgery
Perform surgery on a remote patient

Rehabilitation
Phobia therapy
Use VR input devices and tele-presence to enable
handicapped people to do things that would otherwise be
impossible for them to do
Enable people to visit/see places that they would be
otherwise unable to experience
Use VR to teach new skills in a safe, controlled, environment
Medical
Business
3D visualization of complex financial information
Demonstrate customizable products to customers

Scientific Visualization
View complex data sets to gain greater insight and
understanding of structure
View complex molecular and geological structures
Business and
Visualization
Art
Virtual galleries
and museums
Virtual theatres

Leisure
Games
Sport simulators

Art and Leisure
HSCT
Passengers Passengers Seats Seats 300 300
Cruise Speed Cruise Speed 2.4 2.4 Mach Mach
Range Range 5000 5000 nmi nmi
High Speed Civil Transport (HSCT) Aircraft High Speed Civil Transport (HSCT) Aircraft
XVS
eXternal Visibility System eXternal Visibility System
Challenges of VR
Hardware Technology
More realistic graphics and audio (this is also a
software problem)

Greater processor power

Less lag

Less obstructive input/output devices

Better display technology

Creating models takes a long time and is a skill that
needs to be acquired

We need better software tools to create objects, add
behavior to objects, and to handle interaction

Optimizing models takes time and synchronizing
modifications can be difficult (CAD tools to/from VR
environment builder)
Building worlds
Building worlds
Implementing scripts and interaction is often
difficult for non-programmers

Making accurate models of the physical world is
difficult. Devices such as laser scanners are making
this easier, but human intervention is still a necessary
and time consuming part of the modeling process

Can we build knowledge-based systems to help us
capture the physical world?
User Interaction Design
The user interface for a virtual environment needs to be
carefully designed to take into account the functional
requirements of the application and the limitations of
current technology

How should the user interact with the virtual
environment?
How should the computer provide feedback?
What is the most meaningful way to visualize an
abstract object/process/structure?
What does cyberspace (the Internet) look like?
How do we improve navigational and spatial awareness?
Human Factors
There are a number of factors that should be taken into
consideration when designing the user interface of a
virtual environment in order to minimize physical
discomfort for the user

Latency causes user discomfort (simulation sickness,
nausea)
Well design and optimized models reduce latency and
improve rendering consistency
Eye strain problems?
Long-term physical and psychological effects?
How should users communicate?

How do I know you are real and not an agent?

Social effects of distributed VR?

Technological: Lag, large numbers of users
simultaneously, etc.
Distributed Virtual Environment
VR is a powerful user interface technology

Choosing the best way to visualize information is
important

VR enables the user to interact directly with information

VR enables the user to see/experience things in new
ways

VR does not have to be immersive to be useful
Conclusion