Escolar Documentos
Profissional Documentos
Cultura Documentos
ABSTRACT
Existing immersed virtual reality and particularly augmented reality systems are usually based on dedicated and expensive hardware and I/O devices. Nevertheless they often provide limited support for the collaboration between multiple users and can hardly be used for arbitrary location independent application environments. Additionally, navigation and interaction within such virtual environments are not very natural or intuitive, making them difficult to use. This paper will introduce a new collaborative augmented reality environment - the Virtual Round Table. This environment is designed to support location-independent mixed reality applications, overcoming the limitations for collaboration and interaction of existing approaches. Moreover it extends the physical workplace of the users into the virtual environment, while preserving traditional verbal and non-verbal communication and cooperation mechanisms.
This simple example already showed most of the features the Virtual Round Table (VRT) environment is based on:
augmentation of the current (working) environment collaboration between multiple users intuitive interaction with 3D objects
The Virtual Round Table (VRT) is an interactive task-oriented cooperation environment based on augmented reality technology [8]. A prototype of the Virtual Round Table is currently developed and evaluated within the CAMELOT (Collaborative Augmented Multi-User Environment with Live Object Tracking) project. The Virtual Round Table environment enables participants of a work group to share a 3D application within their regular working environment. Dynamic communication processes are supported by the VRT environment in a task oriented approach. Beside common facial communication the system particularly encourages non-verbal communication, visual association, and sensorimotor abilities of the work group members. The basic idea of the Virtual Round Table is the perspectively correct 3D stereo visualization of a synthetic scene within the real world working environment of the user using see-through projection glasses (see figure 1). Augmented reality is used as a key technology to enhance the real world by virtual objects. In our approach this technology is exploited to facilitate intuitive multi-user interactions, which are realized by combining real and virtual objects into a conceptional unit. By superimposing physical items by virtual objects, they become symbolic representatives of their attached virtual counterparts. By this, manipulations can be directly mapped onto the appropriate virtual objects and hence enable users to interact with these objects in a straight forward manner. This concept of an intuitive tangible user interface provides the basis for the Virtual Round Table to become a collaborative work group environment. Complex structures represented by spatial arrangements of different objects can be manipulated arbitrarily by a group of users. By using the user hand as the primary s manipulator of objects, the Virtual Round Table system can take advantage of the human senses and prestidigitation in respect to virtual object manipulation. In contrast to other technical scenarios based on shutter glasses, such as the Responsive Workbench [19] or a CAVE [12], the Virtual Round Table is location independent and provides an individually adapted stereo view of the virtual world artifacts for each user. The use of conventional, non-sensor-attached items as placeholder objects enforces flexibility, expendability and local inde-
Keywords
Augmented reality, CSCW, collaborative virtual environments, multi-user.
1 INTRODUCTION
Imaging yourself sitting at a table in a restaurant giving your opponent directions to the railway station. Using salt and pepper or a glass to identify major buildings while drawing streets on the table cloth with your knife. Another person sitting at the neighbor table next to you has watched your explanation and spontaneously leans forward to correct you. This is a very intuitive as well as powerful way to provide complex information. Now, what if your opponent would not only listen to your explanations, but additionally see the appropriate virtual 3D objects (streets, buildings, the railway station) exactly where you put the placeholder objects. Thus he would obtain a much richer and better understanding of your explanation, but for you it would still be the same simple and intuitive interface, which allows you to interact with the subjects or even collaborate with other persons very spontaneously.
Permission to make digital or hard copies of part or all of this work or personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. CVE 2000, San Francisco, CA USA 2000 ACM 1-58113-303-0/00/09...$5.00
39
real table projected virtual objects light-weight camera superimposed real world objects light-weight stereo display
6 DOF tracker
Figure 1. Basic Virtual Round Table architecture pendence. In contrast to traditional pure virtual environments or video conferencing systems the familiar verbal and nonverbal communication is not restricted by the system. In fact our approach emphasizes the use of common collaboration and cooperation mechanisms used in regular meeting situations and extends them into the virtual environment. In this paper we will present the basic components of the Virtual Round Table environment and show how they are applied to our sample application scenario. In the second section we will introduce the Virtual Round Table architecture. In the third section we will present our current application scenario used for testing the approach with user groups. The fourth section gives a short overview on the results we received as well as on the problems we faced. The final section compares our approach to existing augmented reality systems.
view than augmentation based on video mixing. We currently use the Sony light-weight projection glasses LDI-D100 providing a resolution of 800x600 (SVGA) at 85 Hz. Due to the SmallTool rendering engine, 3D objects are based on the VRML ISO stan97 dard. The system requires a stereo-capable OpenGL compliant graphics accelerator.
2.1 Visualization
The visualization of 3D objects is based on the multi-user virtual reality toolkit SmallTool [10]. While this toolkit is currently available on IRIX, Solaris, Windows 9x/NT and Linux, we focus on a PC-based solution for Virtual Round Table applications in order to be able to use mobile laptops as soon as sophisticated 3D acceleration will become available. Additionally a PC-based solution is a condition for the aimed low-cost approach. We currently use a Windows NT based environment for testing and evaluation purposes. Augmentation is realized using semi-transparent stereo projection glasses (see figure 2). While the semi-transparent projection of virtual objects into the real scene does not allow for complete superimposition or covering of real object, it provides a more accurate
40
wireless operation sourceless operation (no stationary components) simple external initialization and re-calibration low-cost device (less than one tenth the cost compared to the
tracking system currently used) The MOVY six-degree-of-freedom tracker is based on three basically independent types of sensors (see figure 3):
camera) can see an object, its position can be calculated (obviously it is not necessary to detect object positions for objects not realized by any user). In addition to the real objects the round table itself is tracked by the camera. This provides the tracking system with additional information on the viewpoint of the user. The calculated viewpoint data can be used for external re-calibration of the viewpoint tracking system.
y inertial sensors
y gyroscopic sensors
y magnetic sensors
virtual artifact
Figure 3. MOVY sensor types In addition to these sensors an external initialization and re-calibration of the inertial sensors is required to prevent drifting. We currently evaluate different methods to address this problems. Possible solutions include temporarily fixed radio-based emitters or video-based approaches (see below).
interaction unit Figure 4. Interaction units as tangible interfaces superimposing physical items by virtual objects and using them as placeholder objects, they become a symbolic representative of its attached virtual counterpart. In this way, manipulations can be directly mapped onto the appropriate virtual object and hence enable users to interact with these objects in a straight forward manner.
3 APPLICATION SCENARIO
Application scenarios for the Virtual Round Table are based on three major components: collaboration, augmentation and interactivity. Potential applications include but are not limited to spatial or urban planning and architecture, education and simulation, catastrophe management as well as stage and fair planning. We currently exploit the possibilities of the Virtual Round Table environment in the area of building construction and urban planning. An augmented round table with 3D construction models on it is a powerful tool to fill the gap between an architectural design and the final physical 3D building. It serves as a virtual 3D scalemodel with mobility and high flexibility to use. Figure 5 shows a simple sample scenario (in this example a single fixed mounted camera is used rather than head-mounted cameras). The VRT is used as a major management tool for a construction project director and her team.
41
Figure 5. Simple Virtual Round Table scenario. Project preparation phases consume months of work for a medium size project and there are still many misunderstandings and confusions when the project is realized. Before the actual construction, the Virtual Round Table environment can be of great benefit for the phase of digestion of the design and the preparation for the large number of experts and teams to obtain a three dimensional perception of the buildings and their environment. In addition to the general urban planning tasks it can also be used for preparation of construction material, work planning, etc. for individual buildings. During the construction, it can be used for problem shooting, detail comprehension, progress control, and modification of the design due to new constraints. The combination of augmented reality and the unification of real and virtual objects to a manipulable entity emphasizes the communicative potential of an active modeling process by a detailed but still flexible representation of the objects of interest. Supposing a collaborative discussion process of different kinds of experts and project members, the Virtual Round Table environment enables all participants to easily develop a number of varying design concepts. In comparison to pure physical models, visually improved and changeable virtual objects advance the participant conceivability s of the construction. Additionally, abstract information such as stress and loading can be visualized and supplementary taken into account. Thus, optimizations of producibility and expense can be dynamically evaluated. Diagnostics of different constellations can easily be verified to ensure an optimal utilization of all available capacities.
42
ness and transparency are both realized by a lack of light issued from the projection glasses (see figure 6).
5 RELATED WORK
Augmented Reality (AR) as used for the Virtual Round Table approach is not a principally new technology, but was often restricted to expensive high performance hardware and by that limited to small set of application areas [13]. Existing approaches can be subdivided into those based on semi-transparent see-through systems and video-based systems to combine the virtual environment with the current view of the user [20]. The Bricks prototype [14] uses physical items for the manipulation of synthetic objects. These artifacts can be temporary attached to system internal entities and hence can act as specialized, spacemultiplexed input devices. The realized prototype application of the Graspable User Interface uses two Ascension Flock of Birds receivers as graspable bricks, operating on a rear projected desktop surface called Active Desk. The BUILD-IT [21] system is based upon the concept of a Natural User Interface, that is to empower computer interaction in a natural way using all of the users body parts. The BUILD-IT application supports engineers in designing assembly lines and building plants. The technical infrastructure is based upon a table top interaction area, enhanced by a projection of a 2D computer scene on the table top. Additionally, a video camera is used to track manipulations of a small, specialized brick, that can be used as an universal interaction handler. DigitalDesk [27] is one of the first augmented reality environment systems. Its main approach is to shift functionality from a workstation onto a desk instead of giving a workstation additional desktop properties as it is done in traditional graphical user interfaces. The DigitalDesk application is focused on direct computer-based interaction with selected regions of paper documents. The TransVision system [22] was developed at the Sony Computer Science Laboratory. In contrast to other augmented reality systems mixing of the real and virtual world it is not based on headmounted displays. The approach uses palmtop computer as display units. The user perceives the impression of a see-through screen at the palmtop computer. Actually a small camera is applied to each palmtop computer. The position and orientation of this camera is captured by a 6 DOF tracker. The grabbed video image is superimposed by the virtual 3D objects and displayed at the palmtop screen. The system allows an arrangement for collaborative work similar to the one used by the Virtual Round Table environment. User interactions however are very restricted due to the overall architecture. The Real Reality [11] approach introduced at the University of Bremen is focused on the synchronous modeling of physical and virtual systems. So-called twin-objects, consisting of a real artifact and a corresponding virtual representation, can be manipulated by the users at will. This kind of a complex construction kit enables users to create dynamic physical and virtual models in parallel. The Studierstube system [25] developed at the Technical University of Vienna uses light-weight head-mounted displays to project artificial 3D objects into the real world. In contrast to the Virtual Round Table environment, the system uses a stationary environment. The approach introduces the Personal Interaction Panel (PIP) [24] as a new input device for augmented reality. Virtual objects such as maps, button, or sliders are then projected onto the panel and can be selected by the pen. While the use of the PIP
Figure 6. Dark object areas appear transparent when displayed by see-through devices While the projection glasses currently used, already allows the user to adjust the level of transparency (from see-through to almost immersive) by an additional LCD layer, this layer currently covers the whole display. Providing such a layer on a random-accessbasis (one LCD for each pixel) would allow us to overcome most of the problems of current see-through devices. While this approach would require only minor changes to the current display technology, it would also require appropriate graphics accelerators providing not only a RGB but rather a RGBA video signal. We currently also evaluate video-based augmentation techniques, where the video image captured by a head-mounted camera is used as background image within the display, while the projection glasses operate in immersive (non-see-through) mode. However, due to the missing depth information within the video image, occlusion problems are even more problematic. While the quality of the six-degree-of-freedom trackers has been improved significantly, their repeating accuracy is still insufficient for augmented reality applications. Thus additional methods are required for a proper registration. Similar to other existing approaches we currently try to overcome this problem by an additional calibration based on the camera images. This approach however cannot easily be extended to arbitrary environments, where significant physical properties to be tracked by the camera recognition system are hardly to be know in advance. Thus the further improvement of the tracked six-degree-of-freedom data by redundant tracking components seems to be the only feasible solution. Spontaneous interaction with (arbitrary) real-world objects is very important. Circumstantial introduction processes and learning cycles for new place holder objects discourage participants from using them. In particular application areas -especially in the area of construction- fully immersed walk-throughs may be required in addition to a pure augmented reality environment.
43
shows, that AR can be used to realize new interaction mechanisms, the interaction techniques are basically the same as used within conventional immersed VR environments. The MIT Media Lab [17] introduces the vision of Tangible Bits [18], that allow users to grasp and manipulate digital information by coupling them to everyday physical objects and environments. The design of Tangible User Interfaces and ambient media is focused on foreground activities of the users as well as peripheral background information. Several prototype systems such as metaDESK, transBOARD and ambientROOM have been developed, that take the concept of a Tangible User Interface into account. Important application areas of augmented reality also include medical applications [9] and mechanical assembly line applications [23].
[12] Cruz-Neira, C., Sandin, D., and DeFanti, T., Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE, in Proceedings of SIGGRAPH ACM SIGGRAPH, pp. 13593, 142 (1993). [13] Feiner, S., MacIntyre, B., and Seligmann, D., Annotating the Real World with Knowledge-Based Graphics on a See-Through Head-Mounted-Display, in Proceedings of Graphics Interface 1992, pp. 78-95 (1992). [14] Fitzmaurice, G. W., Ishii, H., Buxton, W., Bricks: Laying the Foundations for Graspable User Interfaces, in Proceedings of CHI'95 Conference on Human Factors in Computing Systems, pp. 442-449, ACM, New York (1995). [www] http://www.dgp.toronto.edu/people/GeorgeFitzmaurice/papers/gwf_bdy.html [15] Henne, Peter, MOVY - Wireless Sensor for Gestures, Rotation, and Movement, ERCIM NEWS, No. 36, p. 42, (Jan. 1999). [16] Holographic displays. Musion GmbH. [www] http:// www.musion.de/ [17] Ishii, H.: Tangible Media Group Project List, 1996. [www] media.mit.edu [18] Ishii, H., Ullmer, B., Tangible Bits: Towards Seamless Interfaces between people, Bits and Atoms, CHI Atlanta, Georgia (1997). [www] http://tangi97, ble.www.media.mit.edu/groups/tangible/papers/ Tangible_Bits_CHI97/Tangible_Bits_CHI97.pdf [19] Krger, W., Bohn, C., Frhlich, B., Schth, H., Strau, W. and Wesche, G., The Responsive Workbench: A Virtual Work Environment, IEEE Computer 28 (7), pp. 42-48 (1995). [20] Milgram, P. and Kishino, F., Augmented Reality: A Class of Displays on the Reality-virtuality Continuum, SPIE Vol. 2351-34, Telemanipulator and Telepresence Technologies, pp. 282-292 (1994). [21] Rauterberg, M., Fjeld, M., Krueger, H., Bichsel, M., Leonhardt, U. & Meier, M., BUILD-IT: a video-based interaction technique of a planning tool for construction and design, in H. Miyamoto, S. Saito, M. Kajiyama & N. Koizumi (eds.) Proceedings of Work With Display Units--WWDU'97, pp. 175-176. Takorozawa: NORO Ergonomic Lab (1997). [www] http:// www.ifap.bepr.ethz.ch/~rauter/publications/ WWDU97paper.pdf [22] Rekimoto, J. and Nagao, K., The world though the computer: Computer augmented interaction with real world environments, in Proceedings of the UIST 95, ACM Symposium on User Interface Software and Technology, pp. 29-36 (November, 1995). [23] Sharma, R. and Molineros, J., Interactive Visualization and Augmentation of Mechanical Assembly Sequences, in Proceedings of Graphics Interface 96, pp. 230-237 (1996).
7 REFERENCES
[8] Azuma, R. A Survey of Augmented Reality, Presence, Vol. 6, No. 4, ed. W. Barfield, S. Feiner, T. Furness III, and M. Hirose, MIT Press, Cambridge, MA, pp. 355-385 (1997). [9] Berlage, T., Augmented reality for diagnosis based on ultrasound images, in: Troccaz, J.; Grimson, E.; Msges, R. (eds.): Proceedings CVRMed-MRCAS '97. (Lecture Notes in Computer Science Nr. 1205), pp. 253-262, Berlin: Springer (1997). [10] Broll, W., SmallTool - A Toolkit for Realizing Shared Virtual Environments on the Internet, Distributed Systems Engineering, Special Issue on Distributed Virtual Environments, No. 5, pp. 118-128 (1998). [11] Bruns, F. W., Integrated Real and Virtual Prototyping, in: Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society (IECON 98), Aachen, Germany (1998).
44
[24] Szalavri, Zs. and Gervautz, M., The Personal Interaction Panel - A Two-handed Interface for Augmented Reality, in Proceedings of EUROGRAPHICS 1997 (1997). [25] Szalavri, Zs., Schmalstieg, D., Fuhrmann, A., Gervautz, M., Studierstube - An Environment for Collab-
oration in Augmented Reality, Virtual Reality Journal, Springer (1998). [26] Virtual Retinal Displays, Microvision Inc. [www] http://www.mvis.com/ [27] Wellner, P., Interacting with Paper on the DigitalDesk, in Communications of the ACM, 36 (7), pp. 87-96 (1993).
45