Conducting a Virtual Orchestra: Multimedia Interaction in a Music Performance Simulator.

S. Schertenleib, M. Gutirrez, F. Vexo, D. Thalmann IEEE Multimedia, Special Issue on Multisensory Communication and Experience through Multimedia, July-September 2004, Vol. 11, Issue 3, pages. 40-49.

This paper presents a semi immersive VR system based on a large projection screen designed to create an entertaining multimedia experience to non-expert users. The participant plays the role of the conductor of a virtual orchestra performing in a 3D concert hall. Several autonomous virtual humans play musical pieces codified as MIDI files. The user conducts the performance by deciding on the music score to interpret and modifying in real-time the layout of the orchestra, the tempo and dynamics volumeof the performance. The conductor and the audience can view and listen to the music ensemble from any position in the 3D environment with dynamic acoustic effects. The user doesnt need to be a professional to conduct this orchestra since all the functionalities and options are available through one simple interface: a magnetically tracked handheld device.

Magic Wand and Enigma of the Sphinx.

T. Abaci, R. de Bondeli, J. Ciger, M. Clavien, F. Erol, M. Gutirrez, S. Noverraz, O. Renault, F. Vexo, D. Thalmann Computers & Graphics, Volume 28, Issue 4, August 2004, pages 477-484.

This paper presents an evaluation of the benefits and user acceptance of a multimodal interface in which the user interacts with a game-like interactive virtual reality application ``The Enigma of the Sphinx''. The interface consists of a large projection screen as the main display, a ``magic wand'', a stereo sound system and the user`s voice for ``casting spells''. We present our conclusions concerning ``friendliness'' and sense of presence, based on observations of more than 150 users in a public event (EPFL's Open Doors 2003).

Mediators: Virtual haptic interfaces for tele-operated robots.

M. Gutierrez, R. Ott, D. Thalmann, and F. Vexo. Proceedings of the 13th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2004)

This paper develops the concept of mediators: virtual interfaces with haptic feedback for teleoperation. Our approach is to replace physical operator interfaces by fully parameterizable adaptive virtual interfaces. Mediators open new possibilities for multimodal feedback for control interfaces. We apply mediators in the context of teleoperation of robots. The implemented prototype shows the feasibility of using virtual haptic interfaces to drive robots remotely.

Creating Cyberworlds: Experiences in Computer Science Education.

M. Gutierrez, D. Thalmann, and F. Vexo. Proceedings of International Conference on Cyberworlds 2004.

This article shows that the creation of Cyberworlds (interactive virtual environments) can be an excellent educational tool covering a wide range of computer science and engineering disciplines. We present the experiences gathered during more than one semester of work with computer science students in the framework of semester projects developed at the Virtual Reality lab of EPFL. Our objective is to share the lessons learnt by both the students and researchers (project advisors). We describe the projects proposed, the methods applied to help the students reach the objectives, the problems faced during the development work and how we managed to solve them.

Telerehabilitation: Controlling Haptic Virtual Environments through Handheld Interfaces.

M. Gutierrez, P. Lemoine, D. Thalmann, F. Vexo. Proceedings of ACM Symposium on Virtual Reality Software and Technology (VRST 2004).

This paper presents a telerehabilitation system for kinesthetic therapy (treatment of patients with arm motion coordination disorders). Patients can receive therapy while being immersed in a virtual environment (VE) with haptic feedback. Our system is based on a Haptic Workstation that provides force-feedback on the upper limbs. One of our main contributions is the use of a handheld device as the main interface for the therapist. The handheld allows for monitoring, adapting and designing exercises in realtime (dynamic VE). Visual contact with the patient is kept by means of a webcam.

Augmented CGI Films.

M. Gutierrez, D. Thalmann, F. Vexo, 2nd. International Conference on Intelligent Access of Multimedia Documents (MediaNet'04).

The contribution of this paper focuses on the definition of a multimedia application targeted at turning CGI films into a richer and more interactive experience. We combine highquality animated graphics to present a compelling story and enrich it by introducing a certain degree of interactivity: allowing the user to watch films using different camera paths. One of the main goals is to provide alternative ways to observe different facets of the same story, different versions according to user profiles and preferences. We

propose a generic coding scheme for high-quality 3D animation, and augment it with an XML content descriptor. Based on the MPEG-7 standard, the descriptor provides information such as preset camera paths that can be freely chosen by the user.