Monash University

School of Computer Science and Software

Engineering

Design Issues in Human Visual Perception

Experiments on Region Warping

Yang-Wai Chow
Ronald Pose
Matthew Regan
Overview

• Background
 Address Recalculation Pipeline
 Priority Rendering
 Large object segmentation
 Scene tearing artefacts
 Region Warping
• Related Work
 Image/Video compression
 Level-of-Detail (LOD) management
 Visual attention models
• Design Issues
• Conclusions
• Background
Address Recalculation Pipeline
Priority Rendering
Large object segmentation
Scene tearing artefacts
Region Warping
The Address Recalculation Pipeline (ARP) was designed to reduce the
end-to-end latency due to user head rotations for immersive Head
Mounted Display (HMD) virtual reality systems

The Address Recalculation Pipeline (ARP)

• Latency is a major factor that plagues the designing of
immersive Head Mounted Display (HMD) virtual reality systems
• End-to-end latency is defined as the time between a user’s
actions and when those actions are reflected by the display

Head Image Buffer Image

Tracking Creation Swap Valid

Average latency to head rotations WITHOUT pipeline

Head Image
Tracking Valid

Average latency to head rotations WITH pipeline

Priority Rendering allows different section of the scene that surrounds
the user’s head to be rendered onto separate display memories and
therefore can updated at different update rates

Priority Rendering
• Different section of the scene can be rendered onto separate
display memories before being combined to form an image of
the whole scene
Segmentation of large virtual world object in conjunction with Priority
Rendering can further reduce the overall rendering load

Large object segmentation

• Fractal terrain example – A fractal terrain typically consists of
thousands of polygons. If the terrain were to be segmented for
priority rendering, different sections of the fractal terrain could
be updated at different update rates
 However the implementation of object segmentation in conjunction with
priority rendering introduces a potential scene tearing problem

Scene tearing artefacts

• Scene tearing artefacts will completely destroy the illusion of
reality, and therefore have to be addressed before object
segmentation can be used effectively with priority rendering
Region Warping was designed to hide the scene tearing artefacts
resulting from object segmentation with Priority Rendering

Region Warping
• Region Warping essentially involves the perturbation of object
vertices in order to hide the tearing artefacts
An example showing the scene tearing effect

Scene tearing
The exact same frame, this time with Region Warping

Region Warping results

Initial Region Warping experiments and analysis do not portray the
human perception or response to the warping

Initial region warping experiments and analysis

• Some initial experiments on region warping have been

conducted
(Please refer to our paper entitled “Region Warping in a Virtual
Reality System with Priority Rendering” – IADIS Applied
Computing 2005)
• However the analysis that was performed were mathematical
analysis of the distortions and did not portray the human visual
perception to region warping
• Therefore further experimentation involving human subjects
are necessary
• Related Work
Image/Video compression
Level-of-Detail (LOD) management
Visual attention models
A number of other computer graphics techniques have also been
developed that take advantage of the limitations in the psychophysical
aspects of the human visual perception

Related work

• The human visual system has a finite perception to detail

• A variety of computer graphics techniques attempt to reduce
the computational effort and time required in the rendering of
scenes or generating of images without greatly affecting a
user’s perception of the displayed image quality
• Examples of human visual perception exploits can be found in
lossy image/video compression techniques
• JPEG and MPEG coding are lossy compression techniques that
attempt to decrease the amount of data in the representation of
an image/video
Level of Detail (LOD) management used in computer graphics attempts
to decrease the amount of detail in a scene without the user noticing
the difference

Level of Detail (LOD)

• Level of Detail (LOD) management have been used by

degrading the peripheral detail in a HMD virtual reality system’s
display, with no adverse effects to the user task performance
• LOD methods also include the use of object meshes with
varying degrees of complexity
• A number of criteria for switching between varying LODs have
been proposed and implemented
• Simpler object meshes can be switched into the display
without the user noticing any alterations in the scene
Others have experimented with visual attention models to selectively
concentrate computational effort on important sections of a scene

Visual attention models

• A user will typically focus on prominent or conspicuous

objects in a scene and will pay less attention to the details
contained in the rest of the environment
• This is a fundamental feature of the human visual system
known as inattention blindness
• Some researchers have conducted visual perception
experiments based on visual attention models in order to
predict where the user will be looking in a scene and focus
rendering effort on those sections of the scene
• Design Issues
Stereoscopy versus Monoscopy
Distance from the user
Head tracked displays
Scene complexity
Virtual environment characteristics
Scene familiarity
Ability to interact with surrounding
environment
As part of the human visual system, stereo vision is one of the main
factors that contribute to a user’s perception of realism within a virtual
world

Stereoscopy versus monoscopy

• The human visual system is based on stereo vision

• A human perceives information such as depth based on the
disparity between left and right eyes
• It is therefore conceivable that a user might perceive differently
the effects of the distortions caused by region warping in a
virtual world viewed through a stereoscopic display or a
monoscopic display
A human perceives less detail in objects that are located further away
compared to closer objects

Distance from the user

• Objects that are further from the users will appear to be smaller
from the user’s viewpoint
• Human binocular disparity/stereo perception fails after a
distance of more than 30 meters
• It is therefore possible that distortions that are concentrated at
a distance away from the user will be less noticeable
Upon user head movements or rotations, based on feedback from the
motoric senses the human brain will expect the scene to change in
certain ways

Head tracked displays

• The human eye is less sensitive to details that move rapidly over
the retina
• Perception to distortion might vary between head tracked virtual
reality systems and non-head tracked systems
The human visual system depends on the retina to cope with the
wealth of information contained within the scene

Scene complexity

• The user visual attention will typically be focus or drawn to

certain interesting areas of the scene
• High levels of detail contained within the scene will distract and
possibly overwhelm the user’s perception
The makeup of the virtual world might also potentially affect a user’s
perception to the region warping distortions

Virtual environment characteristics

• Distortions will appear less obvious on scenes containing

random nature objects compared to scenes containing rigid
structured architectures
• The reason for this is because highly structured scenes
contain straight lines can might noticeably bend as a result of
the warping
The human brain might unconsciously correct perceived errors when
presented with a familiar scene

Scene familiarity

• The human visual system is governed by real world

experiences, as the brain interprets images presented to the
retina based on these experiences
• The human brain might overlook errors contained in familiar
scene
• Alternatively the converse might apply, in that errors might
appear more prominent when presented with a familiar scene
A user’s focus of attention will be divided with allowed to interact with
the surrounding environment

Ability to interact with surrounding environment

• Interactability with the surrounding environment adds to the

user’s feeling or sense of presence in the virtual world
• When a user is immersed in a virtual world, his/her mindset and
focus of attention will be drawn towards trying to manipulate
objects or to navigate through the virtual world
Conclusions
• Region warping attempts to take advantage of the limitations in
the human visual perception, to reduce the overall rendering
load of a ARP virtual reality system with priority rendering
• However while region warping appears to works well on
computer monitors and perform well by analytical measures,
the real test can only be with real human perception
• Therefore before this technique can be used effectively, it is
important to characterize and understand the level of
distortions that might be perceivable to the user
• This paper has outlined our work and future approaches
 Questions
or
Suggestions?