Fragments of Laura:

Incorporating Mobile Virtual Reality in

Location Aware Mobile Storytelling
Experiences

Mara Dionisio, Paulo Bala, Valentina Nisi,

Nuno Nunes

Madeira Interactive Technologies Institute

Faculty of Science and Technology, Nova University

PD/BD/114142/2015
Madeira Interactive Technologies Institute

M-ITI
 BEANSTALK – Research Team
• Beanstalk is a multidisciplinary project team working at the
M-ITI, in partnership with the Associação de Promoção da
Madeira (AP Madeira).
• Our goal is to design and deliver research on the wide topics
of tourism and new marketing trends.
 Creating innovative experiences for tourists
in Madeira Islands

Tourism industry is growing and demands and expectations

from visitors are rising with it. The design and staging of
meaningful and satisfying experiences is a pressing need of
tourist destinations!

‘Buying experiences makes people happier, with a greater

sense of well- being, than purchasing a good’.
Pine & Gilmore (2011)

‘The use of innovative technologies has the potential to

differentiate destinations and create unique experiences
valued by the tourists’.
Neuhofer (2014)
What kind of innovative
technologies?
 Immersive Virtual Reality

360 immersive VR has reached a wider public,

displaying VR experiences and environments is no
longer pressing issue
Image credit: http://blog.thomasstreet.com/post/140648439373/the-evolution-of-app-design
Old & New Generation
of VR devices
 Mobile VR (MVR)
• VR is becoming more and more mobile…

‘System that creates the illusion of participation in a simulated environment,

rather than external observation of such an environment, by replacing real
sensory signals that the user perceives with simulated sensory signals through
the use of portable technology’
Poppe (2017)
But its been used mainly indoors …
 What if we bring VR into the outside
world??

‘Users immersed in a VR experiences

perform better if it displays the sensory
data related to their surroundings’
(Mel Slater and Martin Usoh)

Mobile VR potential applied to tourism?

 MVR + LAMS
• What if we ally Mobile VR with Location
Aware Multimedia Storytelling (LAMS)?
– Sub-genre within the wider field of locative media

‘LAMS combine the mobility of the

audience with the spatial distribution of
the story contained in interactive, multi-
threaded narrative experiences, to
create a sense of place from otherwise
unknown spaces’.
Nisi (2004)
grammed to interact through Bluetooth technology with two art pieces that can be
ountered along the course of the users discovery journey. One of the most

Location Aware Multimedia streets

ovative features of this system is the presence of interactive
med Easter eggs, in game design jargon) along the tour. Suchplay,
of Aalborg.
content
in that
surprises
The system was designed for computer supported collaborative
surprises
two participants
function as experienced the story by working cooperatively to

Storytelling Applications unravel when

ards for the audience exploring the space (Figure 1). For example,
ous users explore an abandoned greenhouse they encountercharacter
the mystery
in the
a Bluetooth
while on the move (Figure 6). Each participant plays a different
the most
story, and these two characters must work together to solve the crime.
triggered
mocromatic painting of a rampicant plant that lights up and The system
slowly simulates
unfolds on the a location-based system, by using Wizard of Oz techniques to

• Several projects over the last two decades

de of the greenhouse. The HU experience was evaluatedrespond throughtosurveys
the of
s interviews, users appreciated the most the following features
the user’s
storyline
andcurrent location, and then the system provides the episodes of
related
the project: Theto that location. The interactive story, accessed by using two
have explored the association of digital media networked
ent presented as narration and not as information; The presence of PDAs, introduces to the two participants the characters they are playing
interactive
rises along the tour; The integration between real andand the other
virtual fictional
worlds, that characters in the story. The players are also introduced to
to specific locations or objects episodicand
ances the possibility to understand interact with the real landscape plotwith
lineslocal
that interweave both fiction and fact, clues and logical puzzles that
ple lead them through the story and through the city.

ure 1 - The Bluetooth activates an electronic circuit and the profile of the landscape appears
Figure
the clothes (Left); The interface: at any moment, visitor can look at the map and 6 – Ilustrative
choose to images of the project “Who Killed Hanne Holmgaard?”
listen one of the five characters (Right).

From this project we learned the importance of coupling location clues of where the
m the HU project we learned that users particularly appreciate
user isaccounts of the
situated with a story element delivered through the mobile experience in order
l characters in the form of stories, how Easter Eggs are a powerful
to createtoola to reward
fuzzy boundary between reality and fiction. Furthermore, from the
audience for their exploration efforts and how real locationaudience
coupled point
with virtual
of view, the experience of the story as a pair can be exploited has

10 information sharing and social values.

Figure 8 - Visitors scanning a QR-Code (Left) and visitor listening to a memory (right)
Mobile Virtual Reality Experiences for
tourism
• Most of VR in tourism industry focuses on virtual
travel;

3D virtual geology field trip Google Expeditions MVR GEs during physical
to Skiddaw mountain (GE) use interactive field trips.
The Open University, UK 360º videos for Minocha et al.
immersive virtual
journeys
MVR has the potential to offer this to tourists a way to enhance
the experience while visiting a location.
 Research Opportunity
• VR hasn’t been used as a medium in
LAMS.
• Study MVR “in the outside world
context”
• Explore the entertainment potential of MVR,
and how it can be used to enhance the
cultural heritage sites by using it within
location aware multimedia stories (LAMS).
 SO …
HOW ARE WE INCORPORATING
MVR & LAMS?
Goal of the work presented today

Study the different aspects of User Experience

while interacting with MVR coupled with a LAMS
tour.

Compare two different VR delivery methods: A

Screen based VR interface and HMD VR interface
in order to find the most suitable way to
experience such MVR experience.
STUDY SETUP
Adapted and prototyped a shorter version of Fragments of Laura
to the University campus

Experience that we will focus on today!

 2D Motion Comic 2D Motion Comic

3D VR Interactive Scene 2D Motion Comic

s the user experience with Mobile VR in a public
Figure 1. Screenshot of mobile application map interfac
s we identified insufficient literature concerning
(top); Screenshots of the motion comic (story points: 1,2 a
and use of Mobile VR technologies outside of the 4) Screenshot of the interactive scene (story point:3)
y context.
ENTS OF LAURA”: A LAMS FOR TOURISM
LTURAL HERITAGE
Methodology LAMVR EXPERIENCE DESIGN AND IMPLEMENTATIO
Motivated by the design of conventional LAM
“Fragments of Laura” is delivered as a mobile applicati
the challenges of designing and testing LAMS
which makes use of a map interface with icons represent
with MVR, we created “Fragments of Laura”
meaningful locations. Each location is associated with
2 conditions: C1Screen & C2HMD
L was designed with the goal of raising awareness
pants regarding the natural and cultural heritage of
story point and its icon is representative of this associati
Participants, supported by the map interface, must find
d of [removed blind rev.]. For this purpose, the
Independent Measures C1Screen
story is based on a combination of historical
desired points by walking to a specific location. Once in

Figure 2. Differences between C1Screen and C2HMD in terms of the user experience
C2HMD
 Yes… we know that:
• ….. they are not tourists ( the intended target
audience)
• …test location isn’t the setting that he experience
was designed for

But we all know that VR can fail big time!

So we need to start in a more controlled environment and the
prototype was in a early stage to test with tourists.
 Participants Task
• Recruited using snowball sampling methodology
• Do a walking tour around the campus while interact
Fragments of Laura LAMS tour
• Uncover 4 locations and their stories
Interactive VR point
 Participant Task: C1Screen
• In condition C1Screen participants will have to
find a clue that will help them interact with
the 3D “Pharmacy” scene.
Condition 1: C1Screen
 Participant Task: C2HMD
• In condition C2HMD participants will have to
follow some clues to find a Google Cardboard
and the recipe that will be hidden
Condition 2: C2HMD
tour for condition C2HMD, the HMD’s could either be
hidden in the experience location or given to the participant
at the beginning. We chose to hide the cardboard together
C1Screen VS C2HMD
with the clue so that participants in condition C2HMD
wouldn’t be influenced by it from the beginning of the
experience.
Interaction
Medium Clue
required
Walk: Continuous touch
C1Screen

in “Walk button”
Mobile
Object Selection: Recipe
Phone Screen
Touch input on virtual
object
Mobile Phone Walk: Continuous
Screen Pressing in Cardboard Recipe
C2HMD

+ Button +
Google Object Selection: target Google
Cardboard virtual object and press Cardboard
HMD the Cardboard Button
Table 1. Summary of differences between conditions C1Screen
and C2HMD
 Participants
• Randomly assigned between the conditions

Total
13 11
24
C1Screen C2HMD
 Procedure and Data Collected

Introduction
Quantitative data
to the •5m – Shadowing
experiment
– Semi-structured interviews
MV LAMS
• ~20m Qualitative data
Tour
– Flow Short Scale
Post- – User Experience
Experience Questionnaire (UXQ)
• ~ 20 m – Game Experience
Questionnaire (GEQ)
RESULTS
QUANTITATIVE
RESULTS
 FoL LAMS: Overall User experience

Experience was very well perceived by all users with above average
median scores for all components from the UXQ
Comparing Mediums: C1Screen VS
C2HMD
C1
C2

A significant difference was found in terms

of Perspicuity.

C1 was easier to get familiar with and to

interact than C2

We did not found significant

differences in any of the other user
experience components.
Comparing Mediums: C1Screen VS
C2HMD

Above average median scores

We did not find

significant differences
in any of the other
game experience
components.
Low median scores

Game Experience Scores

 C1Screen VS C2HMD
Flow Scale Scores

We did not find

significant differences in
Flow scores for Slightly Higher
participants in C1Screen
(Mdn=47.0) and
participants in C2HMD
(Mdn =49.0)
QUALITATIVE
RESULTS
 FoL LAMS: Overall User experience

7 participants really enjoyed was the mix of media between the

multimedia videos and the 3D interactive scene

10participants favorite aspect was the pharmacy

interactive scene
4participants especially pointed out enjoying the combination of using a
map interface with physical markers spread out to mark the story
points

9 participants appreciated how simple and intuitive the

application was
 MVR – Pharmacy scene
8 participants were Surprised & 5 seemed Confused
8 participants seemed to be uncomfortable while interacting
with the interactive scene, four from each condition .

4 participants holding back on the interaction movements

while other 4 participants seemed to be immediately
comfortable while interacting in the pharmacy scene

Pharmacy MVR scene instigated walking among the

participants. 5 in C2HMD and 6 in C1Screen
 Location & Social context

7 participants weren’t uncomfortable in that specific context

[university campus] but if they were in another public space they
would be.

8 participants mentioned that they would be worried

about others observing them if it was in another context.

2 participants mentioned feeling pressured to achieve

the task, as they didn’t want to look like “fools”.
INSIGHTS
 Balance between Multimedia an
immersive multimedia content
• Although participants enjoyed the
combination of media between the videos and
the interactive scene the VR scene absorbed a
lot of attention from the participants. We
need to be careful with MVR it should not
become something mentally draining or
exhaustive
 Challenges in embracing Mobile VR
• Mobile VR is still not widely accepted
socially; In order to have MVR in public space
(outside) we need to choose the locations
wisely to allow a confortable experience. For
example locations without many passers by
or any traffic etc..
 Provide “onboarding” time for MVR
• Participants needed to be put in the right
mind-set. For example incorporate the HMD
as a designed part of the experience; as
“Magical device” that allow the users to
travel back in time and embody the character
of Laura;
 Limitations
• The university campus could have been seen
as a “safe setting”; the shadowing protocol
might also contributed to this felling of
“safeness”
• Not tested with the intended target users –
Tourists
• Not tested in the urban setting that the
experience is design for.
 Conclusion and Future Work
Results in this study encourage us to pursuit investigation in
LAMS combined with immersive VR technology, particularly
using different types of tasks and interactions.

Redesign of certain aspects of the

experience, such as reconsidering the
balance of different multimedia.

Explore how to make MVR “more

aware”/reactive of its
surroundings/location by for example
matching the MVR 3D environment
conditions with the real world weather
conditions, etc..
 Feedback &
Questions?
PD/BD/114142/2015
Thank you!
Mara Dionisio
msgdionisio@gmail.com