Escolar Documentos
Profissional Documentos
Cultura Documentos
Eduardo Veas*
Ernst Kruijff
Figure 1: Two-handed VespR (A), single-handed VespR assisted by non-dominant hand (B),
electronics of small handle (C) and BatPack (D)
ABSTRACT
This paper focuses on the design of devices for handheld spatial
interaction. In particular, it addresses the requirements and
construction of a new platform for interactive AR, described from
an ergonomics stance, prioritizing human factors of spatial
interaction. The result is a multi-configurable platform for spatial
interaction, evaluated in two AR application scenarios. The user
tests validate the design with regards to grip, weight balance and
control allocation, and provide new insights on the human factors
involved in handheld spatial interaction.
CR Categories and Subject Descriptors: I.3.6 [Computer
Graphics]: Methodology and Techniques - Interaction Techniques.
Additional Keywords: Augmented Reality, handheld devices,
mobile computing, 3D user interface, garage interface design
1
INTRODUCTION
43
Task
Accuracy
Speed
Frequency
Duration
Input
Handedness
Typical
DOF
6DOF
(constr.)
Isometric
isotonic
isotonic
isometric
Viewpoint
manipulation
low
high
long
continuous
single
bimanual
Maneuvering
medium
low to
medium
(fast)
low
medium
short
continuous
single
bimanual
single
bimanual
6DOF
(constr.)
1/2DOF
isotonic
isometric
isometric
isotonic
Move
large datasets
(maps)
System control
low to
medium
low to
medium
vary /
medium
medium
to long
continuous
low
low to
high
vary
short to
medium
discreet
continuous
single
bimanual
1/2DOF
6DOF
isometric
(isotonic)
Visualization
mode change
low
low to
high
vary
short
discreet
single
1/2DOF
(6DOF)
isometric
(isotonic)
Object selection
low to
high
low to
high
vary
short
discreet
single
bimanual
1/2DOF
isometric
(isotonic)
Object
manipulation
medium
to high
low
vary
short to
long
discreet
continuous
single
bimanual
1/2/6DOF
(constr.)
isometric
isotonic
Move lenses
medium
to high
low
vary /
medium
medium
to long
continuous
single
bimanual
1/2/3DOF
isometric
isotonic
Numerical / text
input
low
medium
low
medium
discreet
single
bimanual
1DOF
isometric
Typical control
human motion sensing
(micro)joystick
human motion sensing
(micro)joystick
(micro)joystick
pen / touch
human motion sensing
button / slider
pen / touch
(micro)joystick
human motion sensing
button / switch
pen / touch
(micro)joystick
human motion sensing
Button
pen / touch
(micro)joystick
(micro)joystick
pen / touch
human motion sensing
(micro)joystick
pen / touch
human motion sensing
keyboard
pen / soft-keyboard
Table 1 Overview of typical handheld AR tasks and characteristics defining the interaction requirements (italic tasks are subtasks)
RELATED WORK
TASK ANALYSIS
44
based solutions, or a combination of both. LBS provide contextbased information delivery, and mostly simplified interaction:
the information is either filtered based on location, or can be
filtered manually by the user. LBS can apply push or pullservices, varying the level of user involvement in information
delivery. A typical example of sensor-based interaction is a map
application on top of which information is layered. Camerabased interaction provides static or dynamic detection of
location to trigger information delivery. It may be associated
with complex applications, like the map example, or just display
web pages. From the AR perspective, it is most relevant that the
camera image can be used to overlay registered data. Most
dynamic camera-based interaction is limited to simple motionbased actions, providing viewpoint-specific content. Location
tracking can be based on optical tracking (using the camera) or
any other location sensing method.
Looking closer at handheld AR interfaces, one can notice a
clear dependency between the system platform and the range
and complexity of interaction. For cell phones, the addition of
sensors is mostly out of the question. For this reason, they make
use of the camera for viewpoint manipulation, and one or two
(integrated) buttons or the micro-joystick for system control and
selection purposes. The tasks afforded by a cell phone are very
restricted: typically, manipulation actions are constrained to 2D,
due to the fact that most controls are small and difficult to use.
Besides, interaction is restricted to short duration tasks.
Applications with reduced interaction requirements and simple
tracking solutions (indoors, low computation vision-based, etc)
are perfect for these unmodified devices. On the other hand,
applications with higher complexity of tracking or interaction
require extending the platform with more accurate sensors.
UMPCs simplify extension through standard interfaces (USB,
Bluetooth), affording other control possibilities and the additions
needed to support outdoor applications.
In our task analysis, based on [11-14], we investigated a range
of handheld AR applications and device setups (real setups and
literature), including [5, 15-18]. Table 1 provides an overview
of representative actions performed with handheld AR setups.
4.1
Requirement analysis
The device construction presented in this paper evolved out of
requirements from an outdoor AR project for field workers
called Vidente [18]. The application overlays underground
structures over live video images, and makes use of several
tools: an excavation tool (magic lens), an x-ray tool, a filtering
tool (changing visualization modes), a labeling tool, and a
snapshot tool. The tools resemble standard handheld AR tasks as
stated in Table 1, and can be compared to handheld applications
of higher interaction complexity.
45
Outcomes
Following outcomes directly affect (improve) the performance
of a handheld AR construction:
46
Figure 3 Unnatural wrist turn (A), better wrist turn with button
access problems (B)
The UMPC can be held with one hand for a short time, but the
uneven weight distribution will cause the device to tilt to the
opposite side. Balancing the tilt can cause noticeable fatigue: the
fulcrum (balance) of the arms and wrist lever system
biomechanics causes counterbalance muscular activity which
usually pushes the user to apply a two-handed grip. Hence, onscreen interaction, for example with a pen, is tedious and short.
In addition, when external devices are packed to the back of the
UMPC, the construction does not only tilt to one side, but also to
the back during one-handed usage. The balance further restricts
use of controls: accessing the controls at the top side can be
tedious and causes even more fatigue itself, strengthened by the
un-ergonomic turn of the wrist. To use the rather small and flat
controls on the (planar) surface of the device, the user has to
angle the thumb flat on top of the device (Figure 3A). Instead of
using a non-straight, more ergonomic angle (Figure 3B), the
wrists need to be turned towards the device. Slightly bigger
controllers would be beneficial to avoid these access problems.
Outcomes
Grip design
As we found out before, users need to hold the construction in
a power grip. The usage of a power grip fits with the mostly
viewpoint oriented usage of handheld AR, in which the device is
moved over the real world to change the viewpoint. To
improve the way users hold the UMPC and the additional
devices, we performed a grip study. Using clay and plastic
parts, we experimented with different forms and observed their
relationship with the placement of controllers. We started
47
space to put in all the hardware into a grip. Due to the sizes of
the boards and the hidden parts of controllers, the grips had to be
greatly optimized. Finally, the grip is more effective when it
allows the user to grasp around the handle, supporting a more
powerful grip (as previously noted, 76mm from finger to palm).
Some distance between the UMPC and the grip ensures that the
user can really grasp around the grip without being restricted by
the UMPC case.
Configuration of the construction
Having an idea how the handles would look like, we made
mock-ups with foam to find out how the handles could actually
connect to a construction holding all the peripherals. We started
placing handles around the UMPC in different configurations
and analyzed how users would hold the construction, focusing
specifically on the effects the pose would have on limiting
fatigue. The configurations had to comply with several
restrictions. First, the GPS antenna and the external camera had
to be placed at specific locations. Second, the placement
configuration of all devices should be such that weight balance
would be optimal: the construction should not tilt to one side,
leading to user fatigue.
The first device configuration idea was to load all peripherals
below the UMPC like a SLR battery pack, but space turned out
to be a problem. Especially the cables take considerable space
that could not be accommodated, and the camera and GPS
antenna could not be attached easily. We returned to the initial
idea of mounting most peripherals behind the UMPC. The basic
form of the base ended up resembling an L, giving the UMPC
a steady stand.
Ideally, the handles should be attached in the horizontal
weight equilibrium: this equilibrium runs through the middle of
the base, while behind, in front, left and right the weight is about
the same. Calculating the approximate weight of the case and the
peripherals against the weight of the UMPC, we placed the
handles directly behind the back of the UMPC. In addition, we
made some basic arrangements to mount the camera and the
GPS receiver externally. The final form of the construction
resembles a Formula 1 or airplane steering wheel working
well for navigating around objects: the user feels like flying
around objects (see Figure 1A).
Subsequently, we worked on the idea of a single-handed
version. The main idea would be to detach both handles from the
sides. One handle could be mounted below; the second handle
could be put away, or used for freehand (spatial) interaction.
Following the weight distribution factors, we tried to mount the
handle directly below the center of weight. However, due to
limitations in accessing controls at the top of the handle, it had
to be mounted slightly in front of the weight equilibrium. In the
second development stage, we developed a new handle for
single-handed usage that can be mounted directly below the
center of weight.
Outcomes
48
4.5
Electronic design
Based on the controller allocation plan, we mounted an analog
and a digital joystick, usable by index finger and thumb
respectively. The idea was to map constrained interaction
techniques on both micro-joysticks to control specific axes in
translational task with a dedicated controller (see Figure 6). We
also included 3 thumb-operated buttons, and one that could be
reached by either the index finger or the middle finger. All
controls are mapped to a USB board from an off-the-shelf
joystick. The second handle is a test bed for alternative,
unconventional MIDI controllers, including quasi haptic input
methods relying on touch sensitive Piezo sensors. Initial
experimentation shows that the usage of Piezo based elements
has limitations: in single handed configuration, the force needed
by the fingers to balance the construction prevent fine-grained
control. Thus, only the secondary control unit includes a Piezo
sensor accessible by the thumb, and a bend sensor that can be
used by the index finger. The latter can be used well to adapt the
visualization mode. In addition, but not further discussed in this
article, a wireless camera and laser pointer are mounted in the
joystick for additional tracking and interaction purposes.
4.6
Assembly
The last stage consisted of fine tuning the models in CAD,
adapting the forms and getting the grips right. The models were
smoothened, getting its final form now known as BatPack (the
case holding the peripherals), and the Wings (the handles). We
named the device construction VespR, derived from the Latin
word for Bat. The models were printed using stereolithography,
and layered with a rubber-like material, to improve the grip on
the devices. The rubber is a velvety material, comfortable in the
hands and hygienic (cleanable with water). Its softness prevents
extended usage since it peels a bit, but it is easy to process in
production. The rubber is extremely good for providing a nonslippery grip, a real advantage for the weight imposed on the
hands and fingers.
5
EVALUATION
Question
Avg
Stdev
Correlation
with second
study
Overall weight
3,93
1,58
+/-
Weight balance
4,07
1,10
+/-
Ergonomics of grip
4,27
1,16
Grip material
4,80
0,94
Fatigue
5,21
1,58
Placement of controllers
5,07
1,39
+/-
5,20
1,37
Switching nuissance
5,38
1,56
Control effectivity
4,80
1,42
+/-
Overall, the results of the test were quite positive, even though
many ratings were mid-range. The explanation, supported by the
second evaluation, is quite simple: single-handed interaction is
tedious, and clearly affects the rating. The weight and associated
fatigue (even when not rated dramatically in test 1) lowers the
overall attitude towards the device.
5.2
Second evaluation
This evaluation compared traditional UMPC setups with the
different configurations that VespR affords. 15 users (12 male,
3 female, 14 right handed, one left handed) participated in the
test. All users had a computer science background, 4 people
were non-specialists in the field of AR/MR. Every condition
took about 5 minutes to perform, totaling the test time to about
40 minutes (including answering the questionnaire). In total, we
used 5 different conditions: UMPC only (535 grams), the old
UMPC device construction shown on Figure 2 (using the buttons
on the UMPC and a small camera, 739 grams, referred to as
UMPC with plastic case), VespR with two large handles
mounted on the sides (totaling 1249 grams, referred to as
handles at side), VespR with one large handle mounted below
(the one with micro-joysticks, 1091 grams, referred to as big
handle below), VespR with the newer, smaller handle mounted
below (1105 grams, referred to as small handle below). The
different weights provided us with insights in the weight balance
and ergonomic factors influencing fatigue.
The test was laid out in two spatial areas: a selection and a
placement area separated by about 5 meters. The selection area
consisted of a poster placed at eye-height with markers over
which different buildings were overlaid. The placement area
consisted of two posters with a city map. One poster was placed
on the wall at eye sight (wall-mode), one on a desk (tablemode). These different modes allowed us to specifically focus
on how pose affects the usage of the different constructions.
49
Mode Fatigue
avg
Device
Mode
Overall
weight
avg
UMPC
only
UMPC
with
plastic
6.53
Weight
Balance
stdev Avg
0.83
T
W
6.07
1.03
W
Handles
at side
T
Big
W
handle
below
T
Small
W
handle
below
T
5.00
3.73
3.67
1.36
1.91
2.02
6.67
Operate 1
Hand
stdev avg
0.82
2.10
1-handed
view
stdev avg
1.73
5.70
stdev
1.95
6.73
0.59
2.10
1.64
5.60
2.12
6.00
1.20
1.80
1.65
4.70
2.21
6.07
1.10
1.90
1.90
4.90
2.42
6.13
1.30
6.20
1.01
4.07
1.75
4.60
1.64
6.07
1.58
4.27
1.75
4.07
1.67
5.60
1.76
4.00
1.81
4.53
1.46
6.33
1.35
4.07
1.83
3.93
1.39
6.00
1.41
As expected, the weight balance was very good for the UMPC
only, independent if used in table or wall-modes. Through its
light weight, the UMPC-plastic case combo was also rated well
in both modes. The two-handed VespR received high ratings,
confirming our work on weight balance. Again, considering that
the two-handed version is considerably heavier than the
traditional light weight UMPC construction, we believe this
result is very good. In line with the first evaluation, the singlehanded configurations performed less good. Users did not notice
a major difference between the two handle placements, event
though the smaller handle is placed far closer to the weight
equilibrium. Possibly the weight was too high for one hand that
a difference could not be clearly noticed, even when the second
hand was used to support the BatPack.
Single-handed operation ratings changed the appreciation of
quality for some devices considerably. Users rated the usage of
both UMPC and UMPC with plastic case as extremely bad for
one handed interaction. Both single-handed VespR scored much
better with acceptable, but not impressive scores. This indicates
that we improved the construction to such extent that singlehanded interaction (control) of applications is better than with
the traditional configurations. Nonetheless, users believed that
50
UMPC
only
UMPC
with
plastic
Regrasp
weight
stdev avg
5.87 1.19
Regrasp
control
Comfort
stdev avg
stdev avg
stdev
6.73
0.46
6.13
1.46
5.33
1.54
1.54
6.00 1.07
6.73
0.46
6.20
1.42
5.33
5.20 1.70
5.67
1.99
5.40
2.23
4.00
1.89
5.47 1.41
5.80
1.86
5.47
2.10
4.07
1.79
5.27 1.49
6.33
1.11
6.47
0.74
6.07
1.39
1.36
W
Handles
at side
T
Big
W
handle
below
T
Small
W
handle
below
T
5.60 1.35
6.40
1.05
6.47
0.74
6.00
4.07 1.79
4.47
1.84
5.07
1.71
4.27
1.94
4.00 1.81
4.53
1.77
5.00
1.73
4.13
1.84
4.00 1.81
4.27
1.91
4.53
2.07
4.60
1.80
3.93 1.62
4.40
1.80
4.40
2.03
4.27
1.53
not mind holding the UMPC, which was rated about the same as
the big handles we used beside and below the BatPack. The
small handle received a lower score, whereas the UMPC with
the plastic case performed worst. We have the strong impression
users only rated ergonomics of the construction as one instead of
the joystick: hardly without exception, the large joystick
performed extremely well in the two-handed configuration,
while the same joystick received lower ergonomics values in the
single handed configuration.
The material of the joysticks, the velvety rubber, was highly
appreciated by the users (avg. 6.00 / stdev 1.20). The low rating
of the UMPC with plastic could be higher if it had a better
mounting for connecting the UMPC: multiple users reported on
being afraid to drop the UMPC from the casing.
Users found the UMPCs buttons easy to reach. Both the big
and the small handle scored about equally well. Surprisingly,
when asking the users about the placement of the controllers on
the big handles in two-handed configuration, the score was much
higher. Obviously, the weight balance, pose and force on the
hand / wrist have a large effect on how the user reaches and uses
the controllers.
Device
Grip
Mode ergonomics
avg
UMPC
only
UMPC
with
plastic
Handles
at side
Big
handle
below
Small
handle
below
stdev
5,30 1,77
Control
placement
stdev avg
stdev
6.11
1.27
1.18
T
W
4,00 2,16
T
W
3.73
1.10
3.53
1.30
3.67
1.29
6.00
0.88
6.13
0.64
6.20
0.56
5,20 1,82
5.47
1.68
5.00
1.77
4.93
1.79
4,80 1,86
5.33
1.72
4.60
1.64
4.33
1.80
T
W
3.60
5,20 1,82
T
W
Effectivity
avg
The outcomes were in line with our hypothesis that the twohanded VespR is a worthy alternative to the older UMPC
constructions. The second evaluation clearly showed that the
increased weight of the installation can be dealt with in an
ergonomic way: users were very comfortable. It should also be
stated that in simple scenarios, the UMPC without extended
hardware might be useful some of the ratings were more than
acceptable. Particularly if the interaction can be divided in short
subtasks allowing the user to lower the device in between, the
UMPC-only can be effective. However, fine grained interaction
using the UMPC controls is quite tedious. Many users reported
frustration when unable to accurately fine-tune the objects
position (second test) in UMPC only modes. Fine-tuning as
afforded by all VespR configurations was well received.
Although not evaluated in terms of accuracy or duration, finetuning was, in UMPC cases, often aborted by users, while in the
VespR cases it was eagerly performed. The bottom-line is
simple: for complex AR applications, the UMPC-only version
close to useless, since it lacks the necessary quality or
functionality of several devices needed for outdoor usage.
Similarly, some indoor installations require additional tracking
devices that automatically lead to bulkier constructions.
Interaction with the single-handed version is possible, but not
ideal. Both evaluations showed that holding the device singlehanded is not very ergonomic for longer sessions. Leaving the
second hand free for pen-input or real-life communication would
require another approach, particularly for longer sessions. In
single-handed conditions we also noticed some effects described
by Guiard [13]: when placing objects, the non-dominant hand
stabilized the device. The non-dominant hand adjusts the spatial
frame of reference for the dominant hand, supporting it while
performing fine grained actions.
Interestingly enough, users were able to perform fine grained
actions when using the power grip and a thumb based controller.
Furthermore, they were able to ergonomically perform fine
grained actions with their non-dominant hand, which was a big
surprise to us. Currently, the joystick with the index and thumbcontrollers is mounted to the left side of the VespR in the twohanded configuration. Most users, though right-handed, could
control the application perfectly. We do not have an explanation
for this phenomenon, even though we believe that similar
performance can be found in game pads, where users also make
use of the non-dominant hand to perform fine grain interaction.
Finally, comparing the two interaction modes (wall vs. table),
all two handed versions rated higher for interactions in table
mode, while for the single-handed VespR the ratings were
higher in wall mode. In single-handed conditions, the pose
causes the weight balance to tilt to the back when lowering the
device (table mode), deteriorating interaction. Two-handed
setups have the advantage that the interaction mode does not
affect weight balance, while the lower position of the device in
table mode causes less strain [19], as noted in section 4.2.
Take home message
51
CONCLUSION
VespR significantly advances ergonomics of handheld
devices in such a way that users perform comfortably even with
double the weight of other devices. Extra weight is allocated to
higher quality sensors providing developers the opportunity to
improve registration, presentation, and interaction for AR
applications. VespR has been continuously in use since its
construction by researchers and end-users, for demonstrations
and practical applications.
Even with the trend in miniaturization and integration of
devices, the results of this study are valid for further
developments in the area. Future UMPCs will likely integrate
more or better devices that are currently externalized. Still, AR
will keep putting requirements that cannot be completely
matched, including special tracking requirements. Furthermore,
technological improvements can not ignore human factors if
acceptance is expected in the long run.
We are currently optimizing the VespR by improving the size
and internal organization of the BatPack, and performing further
experiments with the MIDI controllers. In addition, we are
planning additional evaluations of longer duration outdoor
sessions.
ACKNOWLEDGEMENTS
We would like to thank Erick Mendez and Gerhard Schall for
aiding us with the hardware setup material, ideas and the
evaluations at Ubicomp. We also thank Albert Walzer and
Stefan Zedlacher for aiding in the construction of the devices.
This work was partially funded by the Austrian science fund
FWF under grant Y193.
REFERENCES
[1]
[2]
Microsoft Research.
[10] Jones, M. and G. Marsden, Mobile interaction design. 2006: John
Wiley & Sons Ltd.
[11] Bullinger, H., P. Kern, and M. Braun, Controls, in Handbook of
Human Factors and Ergonomics, G. Salvendy, Editor. 1997, John
Wiley & Sons.
[12] Zhai, S., Human Performance in Six Degree of Freedom Input
Control, PhD Dissertation, Dept. of Computer Science. 1995,
University of Toronto.
[13] Guiard, Y., Symmetric Division of Labor in Human Skilled
Bimanual Action: The Kinematic Chain as a Model. The Journal
of Motor Behaviour, 1987. 19(4): p. 486-517.
[14] Kabbash, P., B. Buxton, and A. Sellen. Two-Handed Input in a
Compound Task. In Proceedings of the 1997 ACM Conference on
Human Factors in Computing Systems. 1997: ACM Press.
[15] Pering, T., Y. Anokwa, and R. Want. Gesture connect: facilitating
tangible interaction with a flick of the wrist. In Proceedings of the
1st international conference on Tangible and embedded
interaction. 2007.
[16] Reitmayr, G. and T. Drummond. Going out: Robust tracking for
outdoor augmented reality. In proceedings of ISMAR 2006. 2006.
[17] Rohs, M. Visual Code Widgets for Marker-based Interaction. In
IWSAWC'05: Proceedings of the 25th IEEE International
Conference on Distributed Computing Systems - Workshops
(ICDCS 2005 Workshops). 2005.
[18] Schall, G., E. Mendez, E. Kruijff, E. Veas, S. Junghanns, B.
52
[7]
1997.
[6]
[5]
[4]
[8]