Menu Presentation and Selection in Multimodal Immersive Environments

Namgyu Kim Gerard Jounghyun Kim Chanmo Park Inseok Lee*

Dept. of Comptuer Science and Engineering and * Dept . of Industrial Engineering Pohang University of Science and Technology (POSTECH) San 31, Hyoja-dong, Pohang, Kyungbuk, Korea 82-562-279-5664

ngkim@postech.ac.kr ABSTRACT
Usability has become one of the key ingredients in making virtual reality (VR) systems work, and a big part of a usable VR system is the design of effective interface/interaction schemes. During the past few years, several empirical studies have been conducted to evaluate the usability of generic interaction techniques (e.g. navigation, selection, and manipulation) in the virtual environment context [Bowman][Darken][Mine]. While some of the results are valuable, however, such efforts have not generalized into a mature interaction/interface design guideline yet. The reason may be that there are simply not enough studies to conclude on a definite set of guidelines due the many factors that must be considered such as the types of interaction/display devices, types of domain tasks, display resolution, etc. In this work, we investigate in the usability of various menu presentation and multimodal selection schemes in virtual environments. It is often required to make commands in a virtual environment, and when there are many number of available commands with a complex structure, a hierarchical menu has been one of the most popular choice. While the task of menu selection may be viewed as a composite task of manipulation and selection so that this work builds on the prior research in generic interaction, there are also enough idiosyncrasies that warrants a more in depth look as a generic task of its own. First of all, there is more freedom in the choice of menu display method in the 3D environment, and the display method may in turn affect the usability of a particular selection method. We have identified 5 major menu display methods in a virtual environment, and 13 interaction menu selection methods. In this paper, we explain how we came up with such a classification for the usability testing, describe how we ran the usability test, and report on some of the results we have obtained for two (out of 13) interaction methods across the 5 different menu display methods.

1.

INTRODUCTION

Usability has become one of the most important aspect of making virtual reality (VR) systems succeed for a give application, and a big part of a usable VR system is the design of effective interface/interaction methods. During the past few years, several empirical studies have been conducted to evaluate the usability of generic interaction techniques such as navigation, selection, and manipulation in the virtual environment context [Bowman][Darken][Mine][Pausch]. While some of the results are valuable in designing interaction methods for a given task, such collective efforts still fall short of forming a mature interaction/interface design guideline. It is not clear how these research results apply to designing interfaces for a particular task or application. The reason may be that there are simply not enough studies to conclude on a definite set of guidelines due the many factors that must be considered such as the types of interaction devices, types of display, types of domain tasks, frequency of the primitive tasks, display resolution, number of hands used, etc. In this work, we investigate in the usability of a menu system for virtual environments. It is often required to make frequent commands in a virtual environment, and when there are many number of available commands with a complex structure, a hierarchical menu has been one of the most popular choice both in 2D and 3D environments. A typical example is a computer-aided design systems. Not only there are many available commands with three to four hierarchical levels, the task of design require the user to continually use the menu system. Menu system is also probably one of the most familiar interface to the ever increasing computer users thanks to the wide spread window-based desktop interfaces. The task of menu selection may be viewed as a composite task of manipulation and selection. Therefore, we can certainly build on the prior research in the usability of generic interaction tasks. At the same time, for its its importance, frequent usage, and idiosyncrasies, menu interaction in the VE warrants a more in depth look as a generic task of its own.

Keywords
Multimodal Interaction, Menu Presentation, Menu Selection, Virtual Enviornments, Interaction Design, Interface Design.

3.1
First of all, there is more freedom in the choice of menu display method in the 3D environment than in the 2D desktop environment, and the display method may in turn affect the usability of a particular selection method. We have identified 5 major menu display methods in a virtual environment, and 13 interaction menu selection methods. The five display methods are identified based on their locations and whether they are fixed at that locations in the virtual environment. They are further classified according to their presentation styles that differ in the visibility of other virtual objects in the proximity, and showing of the command selection path. The 13 interface methods considers modalities of the two subtasks of a menu selection, i.e. positioning and making a command (3 for positioning and making a command respectively), and the number of hands used (0, 1, or 2). Out of the total 27 combinations, only 13 are chosen for their significance and feasibility. What are we trying to measure This paper is organized as follows: Assumption: frequent use HMD limited FOV and resolution In this paper, we explain how we came up with such a classification for the usability testing, describe how we ran the usability test, and report on some of the results we have obtained for two (out of 13) interaction methods across the 5 different menu display methods. Why multimodal 2D menu study = 2D not clear ? menu selection is quite primitive for other studies to be applied context is different still recognizability Frequent selection: why frequent need to quantify this figure ?

Location vs. View Direction

possible at the specification level.

3.2 Set

HMD Resolution vs. Large Command

of ASADAL in a very clean manner. We are considering to add new semantics for our implementation of the STATECHARTS to accommodate this feature.

3.3

Final Candidates for Experiment

As indicated before, implementing a VR system often requires cycles of performance tuning and the purpose of carrying out an LOD engineering beforehand is to reduce this cycles of work as much as possible at the specification level. While the primary purpose of specification simulation is to verify the logic and temporal properties of behavior specification, its expected performance can be estimated indirectly by collecting relative timing data. However, there is an inherent difficulty because the computational model of specification simulation can be quite different from the computational model of the platform where the actual VR implementation would be run (which can be several such as onesingle-loop model, distributed model, etc). Moreover, if it would be possible to specify conditions and actions for LOD switches, its overhead must be accounted for during simulation and its performance estimation. This calls for a specification simulation tool whose computation model is modeled after typical VR simulation models.

4. Mutlimodal Menu Interface: Positioning and Making a Command

In this section, we demonstrate our idea of LOD engineering for modeling and determining the appropriate number of instances of LODs of a vehicle object. There are 30 vehicles (with a fixed path) is to be populated in a small virtual town.

2.
2D stuff

Related Work

5.

The Experiment and Results

Virtual Menus examples Menu in 2D Fitts Law bolt has different context Menu in 3D used also but not much studeis on it in context of VR.

6.

Conclusion and Future Work

3.

Menu Presentation

In this paper, we have presented the concept of a structured LOD engineering processing using a top-down refinement strategy. We do not contend that this method is to replace the need for bottom-up approaches as design of systems often employs both paradigms. We have already proposed in a previous paper the use of structured specifications prior to code design for VR system development. Making use of the models from the intermediate refinement levels for LOD is a natural and intuitive method for performance conscious modeling. Before proceeding to the next modeling stage, we can decide whether certain part of the system could be refined or not on the basis of

75 D E

70

65

60 estimated time (ms)

55 B, C A

50

45

40

35

30 frames

the intermediate simulation results. Hopefully, the resulting virtual world is a conglomeration of geometric objects and their behaviors that are appropriately refined in different parts, and switching policies that are tailored for the user needs. ASADAL/PROTO, our specification making tool, is still short of supporting the requirements for VR software. We plan to, one by one, add specification tools and modify the computational model of the specification simulation of ASADAL/PROTO to make it a better planning tool for VR applications.

[6] Pahl, G. (Tr. by K. Wallace), Engineering Design, The Design Council, Sprinter Verlag, 1984 [7] Luebke, D. A Developer's Survey of Polygonal Simplification, IEEE VR Conference Tutorial 7 Course Notes, 1999 [8] Watson, B On Temporal Level of Detail: System Responsiveness, Feedback and User Performance, IEEE VR Conference Tutorial 7 Course Notes, 1999 [9] Musse, S Crowd Modeling in Collaborative Virtual Environments, Proceeding of the ACM VRST Conference, pp. 115-124, 1998 [10] Hoffman, W Database Design for Visual Simulation and Entertainment, SIGGRAPH 97 Course Notes on Designing Real Time Graphics for Entertainment, 1997 [11] Anderson, R.E. Social impacts of computing: Codes of professional ethics. Social Science Computing Review, 2 (Winter 1992), 453-469. [12] ACM SIG PROCEEDINGS template. http://www.acm.org/sigs/pubs/proceed/template.html. [13] Conger., S., and Loch, K.D. (eds.). Ethics and computer use. Commun. ACM 38, 12 (entire issue). [14] Mackay, W.E. Ethics, lies and videotape... in Proceedings of CHI '95 (Denver CO, May 1995), ACM Press, 138-145. [15] Schwartz, M., and Task Force on Bias-Free Language. Guidelines for Bias-Free Writing. Indiana University Press, Bloomington IN, 1995.

7.

ACKNOWLEDGMENTS

The work presented in this paper was supported in part by the Electronic and Telecommunication Research Institute of Korea and the Korean Research Foundation.

8.

REFERENCES
[1] G. Jounghyun Kim, K. C. Kang, H. Kim and J. Lee, Software Engineering of Virtual Worlds, Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pages 131-138, 1998 [2] Deborah A. Carlson and Jessica K. Hodgins, Simulation Levels of Detail for Real-time Animation, Proceedings of Graphics Interface, pages 1-8, 1997 [3] Hoppe, H. Progressive Meshes. Computer Graphics (SIGGRAPH 96 Proceedings), pages 99-108, 1996 [4] SE top down [5] Simon, H. The Sciences of the Artificial, Second Edition, MIT Press, 1981

75 D E

70

65

60 estimated time (ms)

55 B, C A

50

45

40

35

30 frames

Figure 6: Performance of different LOD combinations at the specifiction level.