Acting in Virtual Reality

M. Slater, J. Howell, A.Steed, D-P. Pertaub, M. Gaurau
Department of Computer Science
University College London
Sharon Springel
Centre for Communications Systems Research
University of Cambridge

ABSTRACT manipulator for visualization. The one application in Professor
Brooks’ review that seemed to step outside of the design/engineer/
Three pairs of professional actors and a director each met in a
skill-rehearsal applications suite was psychotherapy for fear-of-
shared non-immersive virtual reality system over a two-week
flying and post-traumatic stress disorder. However, even in this
period to rehearse a short play. The actors and director never met
case, users engage with an external simulated environment in a
one another physically until a short time before a live rehearsal in
way that is consistent with the approach of behavioral exposure
front of an audience. The actors were represented by avatars which
therapy. Here a patient interacts with a psychotherapist using the
could be controlled to make a range of facial expressions, and
simulation and the experiences it evokes as a common point of
some body movements, including navigation through the space.
contact.
The study examined the extent to which virtual reality could be
A characteristic of each of these types of application is that
used by the actors and director to rehearse their later live perfor-
groups of people interact with one another through their shared
mance. Four indicators captured by questionnaires show that over
interest in some simulated objectified process: flying the aircraft,
the period of the four days their sense of presence in the virtual
viewing the design prototype, or examining the visualization. The
rehearsal space, their co-presence with the other actor, and their
virtual reality in each case provides a scenario, and the fundamen-
degree of cooperation all increased. Moreover their evaluation of
tal interaction is between each individual user and the environ-
the extent to which the virtual rehearsal was similar to a real
ment; interaction between people occurs with respect to their
rehearsal also increased. Debriefing sessions with the actors and
common interest in the scenario.
director are reported, which suggest that a performance level was
In this paper we explore a different virtual reality paradigm in
reached in the virtual rehearsal which formed the basis of a suc-
which the objective is emotionally significant interaction between
cessful live performance, one that could not have been achieved by
people, rather than between people and some other situation or
learning of lines or video conferencing.
thing. The virtual environment provides only the setting in which
Keywords person-to-person engagement takes place, it is the interaction
Virtual reality, multi-user, networked applications, facial anima- between the people, and the ‘emotional atmosphere’ which this
tion, virtual rehearsal, acting generates which is the major objective of the experience.
The scenario in which this is explored is that of dramatic act-
ing, in particular virtual rehearsal for live drama. Suppose actors
1. INTRODUCTION and their director never meet one another, except in a virtual real-
Practical applications of virtual reality are normally in the ity setting, until a short time before a live stage rehearsal. We con-
realms of engineering, product design, and skill rehearsal. Users sider the question: to what extent can they create sufficient acting
enter into a virtual environment in order to learn something new performance so that a live performance can take place with hardly
about the real situation to which the simulation corresponds, or to any live face-to-face rehearsal?
improve or learn a skill set. Professor Frederick Brooks Jnr The practical motivation for this was first suggested by the
recently carried out a survey of such applications [1] which head of research at the BBC: If it is possible for rehearsal to take
included vehicle (aircraft and boat) simulators, design and proto- place in a distributed virtual environment, then significant savings
typing systems, a NASA astronaut training system, and a nano- could be made in pre-performance production and planning.
Actors could stay situated wherever they may be in the world, con-
tinuing with their current commitments, while nevertheless devot-
ing some time to rehearsal of a new production via a shared VE.
The actors, director and others involved could get to know one
another. The technicians could work out camera angles and light-
ing in parallel with the performers working out their blocking (i.e.,
the spatial arrangements of the performance in synch with the
dynamics of the narrative).
Acting rehearsal is more than just learning of lines, which is
the least important part of the rehearsal process. Actors must act in

1

The avatar Repertory Theatre2. However. for explore a virtual world based on the animated movie [7]. pants. I would replace Fritz’s name with ued the same task. precisely-timed eyebrow changes. significant social interaction between participants.ibm. In two follow-up studies their emotional state through the normal everyday employment of [11][12] similar results were found.” bility of exhibiting facial expression. it was always found their facial expression. i. projects at regional theaters throughout the US and Europe. to the display of plicity. The participants were says. These punctuate ney’s Aladdin immersed and allowed participants to interactively the conversation as a secondary feedback channel. and making appropriate salutations. This simulated a tra. but that avatars ditional drama. reproducing an earlier result [10]. mance? It is important to distinguish different types of avatar expres- siveness.com The results of the series of three-person experiments in 2. This was an attempt to representing people in a collaborative virtual environment.e. ciency and smoothness of the resulting conversations. been described elsewhere in based on the current intentions of the user towards others. together with the results of [13][14] informed the 2 . By personalizing the situation. to our knowledge. Dis. of course.” Vujoshevich changed from the virtual to real experience. and type messages to The use of a VE for rehearsal of a later ‘real life’ performance others. The point was to see how their social interaction my fiancé’s. exhibiting such behavior were rated as more helpful and lifelike Viewers could navigate the scenario with a standard VRML compared with avatars that exhibited only emotional feedback. that designers involved. More example. only a constant neutral expression. unlike the situa. In [9] full range of human emotions in a way that produces the appropri. The exploit a collaborative virtual environment (CVE) as a medium in BodyChat system provides semi-autonomous avatars for partici- itself for the creation of a new form of entertainment. and vocal chords. For example. For example. The major rehearsal of a role in a Berchtold Brecht play [2]. and thereby avoiding the ‘dead’ actually share the same space together. The given the opportunity.htm [9][11][12].adding to their very high degree of ‘aliveness’and presence. findings. and unconsciously generated. Envelope behaviors are not consciously initiated by the A system which synthesizes a collaborative VE with live TV speaker. Envelope feed- ters with believable personalities which can perform. lying emotions such as happiness or puzzlement. The artists involved were employed on other then automatically generates the appropriate non-verbal behaviors. phy changes to be agreed by the performers. and interact back refers to non-verbal behaviors such as gaze timing and with human participants. Arm movements were only This describes a strategy used by actress Tania Vujoshevich in her possible for the one immersed participant in each group. an avatar may use of video-conferencing to rehearse an adaptation of the Diary of be put into a state of ‘availability’ with respect to another user. where the par. those neously edited and broadcast for live TV. she found she was able to convey represented by simple block-like humanoid avatars. The same must be plastered on the walls of a virtual room. as set by the literature. or browser. and expressions with the mouth. but the envelope communicative behavior is automated of a play has not. both con. also automatically carry out low level non-verbal behaviors such ticipants meet to carry out a shared task that they will later perform as blinking and breathing . body posture. Acting involves accessing and expressing the together in real life. sonal and social significance. Users may control their navigation. The major advantage of such an approach is allowing the avatar itself to take care of such automatic non-verbal that live video of the real performer is distributed. the exploitation of the medium for story-telling. and pretend I would never see him again. with no possi- the pain and sadness and fear the character was feeling. which was simulta. After meeting virtually for true during rehearsal: a while they then met in the corresponding real room and contin- “In the beginning. 1. director and other It is argued in [14] with supporting experimental evidence. tion in live theater or film. but also adds to the effi- The situation we are examining. Vaslav Nijinsky.vrmldream. to regulate turn-taking and hesitations in speech. To what that the avatars’lack of expressiveness (they were only able to turn extent can the impoverished substitute for emotional expression and move through the environment) impeded the development of available through today’s VE technology support acting perfor. Another goal is distribution. http://www. In this case members of the public is detrimental to interaction. notwithstanding their extreme sim- sciously chosen. except that it occurred entirely in a distributed VE. envelope behaviors was applied to embodied avatars. is from a technical point of view a CVE. Such situations have been rarely studied. important for regulating the conversational flow. that relate to this paper. However. this is behaviors not only frees the user from having to control every eye- at a cost of fragmentation of the total space: the performers never brow and mouth movement. play from the met first in a virtual environment to try to solve a set of riddles emotional expression delivered to one another. In [14] the principle of automated interacted with professional actors in a game. The recently the VRMLDream company created a performance with experiment showed that envelope behavior was not only more actors directly controlling virtual characters1. Emotional feedback employs specific Research into the exploitation of virtual environments for emblems such as a smile or scrunched eyebrows to express under- drama has concentrated mostly on the creation of synthetic charac. helping. for example [3][4][5][6]. were that the avatars took on per- In physical reality actors have complete access. moments while they are typing messages. and thus choose their own viewpoints. In [13] an experiment is reported that compares the effi- 2. Actors also. BACKGROUND cacy of emotional and envelope feedback for automated avatars in conversation with humans.order to rehearse. during rehearsals. virtual rehearsal for a live per. hand gestures. such as glancing towards the other avatar video-conferencing system was used in order to allow choreogra. head movements.com/sfasp/vr.. The closest to what is reported in this paper was the the user for their corresponding avatar. http://www. but their absence broadcast is described in [8]. developed by the New York based Gertrude Stein meaning that there is interest in having a conversation. an experiment was described where 10 groups of 3 people each ate affect in the audience. The avatars formance. not consciously noticed by the onlooker.

not involved in the planning of the study. The script was for a play of about five minutes. nal codification of the play that later transforms to their behavior on stage. so that complete vir- through the virtual rehearsal process the actors construct an inter. 8Gb RAM. for the first time. However. rehearsal actors were questioned by the audience. and mutual The virtual rehearsals. The scene was Directors Guild of Great Britain. but allow the actors to choose these as appro. The director used an Onyx2 system compris- how well the performance was working. The director’s view was also video-taped. They then had about 10 minutes to carry out some live rehearsal. and the last virtual some body gestures . This was followed by a debriefing their partners and in an audience. The actors were embodied as avatars who could see and talk to ence. tual rehearsals were recorded from this third position. S had a strong back. The scene was a virtual kitchen with an outside ante-room rep- tice with the system. On the fifth time the 3500 polygons and was rendered by DIVE using OpenGL calls. There were two directors. ing a Graphics Rack with 8 R12K processors. again with the scene shown on the full 19 directive. sessions were video-taped. The The actors and their director met four times virtually and then monitor was 23 inches and the full screen was used for the display.1 The Scenario A script was developed from a training program previously used at the UK’s National Film and Television School.design of our acting rehearsal system. S directed two of the groups. discussion. (B). Three pairs of actors took part throughout. The second ing a largely passive role . About 30 minutes of this time was spent in equivalent to where the audience would sit.the rest of the time was setup. for about one hour. During the virtual rehearsals the actors’ A second way in which this application differs from traditional screen views were video-taped with an inset of their real selves in a conferencing or chat rooms is that the virtual interaction is a smaller screen window. and two per week for two successive weeks.se/dive 3 . bodied voice with no visual representation. The kitchen and avatars consisted of about repaired.especially to be able to position the head. At the end of the live therefore have significant control over the behavior of their ava. The frame-rate was at least 20 then they met immediately afterwards physically for the first time frames per second on all machines. but on the other of about 20. 3. and An overview is shown in Figure 1. and answering a questionnaire. live performances. tak. Both actors and the director used desktop dis- ground in film and TV screenwriting and direction. THE VIRTUAL REHEARSAL STUDY 3. priate. frequently commenting on inch display monitor.sics. in the real rehearsal space. and all debriefing gaze-locking. disk and 4 IR2 Graphics pipes each with 2 Raster Managers. before the audience was let in and an 1. It was thought that actors must carried out by two of the investigators. http://www. actors again met their director for about 30 minutes in the VE. with a bonus after completion of the Figure 1 An Overview of the Rehearsal Space entire rehearsal process. The audience. On each of the first four occa. 180Gb tional. They were paid enough to cover their expenses for each meeting. but one of them had to withdraw due to an unex- pected acting engagement). (On the first day there was an addi- tional couple. The director was a disem- until the fifth and final occasion. whole focus is on how they say it. The four rehearsals took place actors had to have the capability for making facial expressions. This involved a man and a woman in a kitchen in the morning having an everyday conversa- tion about mundane matters while preparing to go out. were generated automatically. and for them to be able to find At the end of each of the first four rehearsal sessions the actors just that way of saying it that can evoke the appropriate response in completed a short questionnaire. such as blinking. suggesting individual moves. The actors were often sions the actors and their director met in the virtual environment reminded that the director was taking the ‘downstage’ position. So their ment. was a co-chair of the Infinite Reality Graphics and 64M main memory. The actors never met each other or the director physically one another in the shared virtual space. as is tradi. One (S) had been directly involved The virtual rehearsal system was implemented using the DIVE in the design of the study for several months. Accordingly their avatars should not automatically generate evening had a round table discussion where all pairs met together envelope behaviors. awaiting for system breakdowns to be resenting off-stage. and in the tars. It was decided that the observed rehearsal was carried out. on a fifth time to carry out the rehearsal for real before a live audi. leading to conflict with the actors.allowing greater freedom for the actors actor used an SGI High Impact system with 200Mhz R4400 and to establish their blocking and individual expression. low-level behaviors. One actor used an SGI Onyx with twin 196 MHz R10000. So another question is the extent to which The director also acted as a camera controller. were recruited by advertisement around the depart- hand they know in advance what they’re going to say. Actors pose a tough challenge: they must ately by the live rehearsal watched by an audience. and rehearsal was on the Monday of the third week followed immedi- make arm gestures. rehearsal for real life. B was more 64MB main memory. sometimes. 3. prac.3x platform [15]1. exhibit both envelope and emotional behavior. actual rehearsal . The second plays. shown on a 21 inch monitor covering the full screen.

3. to evoke a smile they would draw one. the expression that they were trying to induce. They could also use the arrow menu system.controlling the degree of For example: it was early morning and the night before one of happiness or sadness. In addition. An approximate hor- shown in Figure 2. The first is that 3. but was edited to improve the face and hair in particular. It would have been simple to provide a visual menu of facial expressions from which the actors could select by point-and-click. anger. The intensities were truncated to a maximum before being passed to the face plugin. hair and eye geometry was devised and fitted to the female avatar and various of the body parts were scaled and adjusted. watching the other actor. head was facing. For the mouth a the accompanying animation algorithms were rewritten for DIVE. Thus in principle the head and body direction itively. and also the an intensity for any one of the four major expressions and also to degree of symmetry. Some examples are with used for surprise and for anger. either one of the previ- ous ones that they had rehearsed or a new one. this was not employed for two reasons.controlling the degree of anger or surprise. and A very simple recognition algorithm was used. This is shown in the inset below. The second is that we wanted the actors to be had the face controller at the top. sadness and a neutral the gradients on each side together with the area of the triangle expression. Or. At first this movement was completely free.net/vrml/h-anim/h-anim-examples. thus allowing a wide variety of facial phone call that would give the results of a biopsy. so that after a party they had slept together for the first time .4 Other Controls actors would need to continually scan the set of displayed faces in order to choose the one that they wanted . thus changing their viewpoint. A substantially the same throughout. The face was texture-mapped to help with the required masculine or feminine appearance. a grimace or a neutral expression . created by 3Name3D1. The sizes of be displayed: happiness. The code was designed to enable the actor to specify were used to determine the intensity of the expression. Figure 2 Some Examples of Facial Expressions tion was based around a muscle model originally proposed in [16]. After a relatively short degree of practice (specifically 3.giving half. Moreover the intensity knew if the other really meant it. triangle was determined from the stroke action. Some very basic animation capabilities were provided that allowed the avatar to sit. A stroke such as The facial animation interface allowed five major expressions to would be interpreted as ‘sad’and as ‘happy’. http://www. A modified subset of the geometry was used since the original avatar had no facial animation. This sub-text was the one to be rehearsed before the live audience. An overlaid smaller window the environment. izontal line would result in a neutral expression in each case. anger/surprise) were orthogonal. smile. the actors could move making the actions that they carry out somewhat consistent with their avatar’s head in any direction. Or the woman was awaiting a of each could be controlled. The main screen- speaking their lines. but last night sets (happiness/sadness. the body was texture-mapped for clothing. them the feeling that it was somewhat under their control. A stroke on the top half would affect the them had crashed their brand new car. rehearsal they chose a sub-text themselves. Although the script was emotionally neutral.but neither combinations of the two could be produced. stand up and move its arms either singly or together. However. surprise. and other controls in the lower actively engaged in determining their facial expression . but after the first session the actors complained that this freedom was too confusing (their heads could be twisted back-to-front reminis- cent of The Exorcist) so the system was adjusted such that when 1.determining a were told by the director to act according to a different sub-text. The body of the female avatar was an adapted version of an H- Anim compliant avatar called Nancy. On the fourth expressions. and remained The abstract face was divided into a top and bottom half. were independent. A new head. So instead of a while leaving the body stationary.2 The Avatar and Facial Model The male avatar originated with the DIVE system. they had been sharing eyebrows . the actors controlled their facial expression by keys to move the body forwards or backwards in the direction the means of mouse strokes on an abstract ‘smiley’ type face.html 4 . Facial anima. each time the actors met they stroke on the bottom half would affect the mouth . These two the same house together for some time as friends. and moving through sized window showed the scenario. Intu. The face itself was obtained from a program supplied in [17].3 The Controller for Facial Expressions by the second rehearsal session) all of the actors but one were using facial expressions with ease. and also By using the mouse in the main window.this at the same time as The screen was divided into two windows.ballreich. Both the male and female avatars were fitted with a new face to enable facial animation. A similar method was used for the eyebrows allow for asymmetric facial expressions.

5 . Co-presence point. and for all as a group. was at first extremely ill-at. One of the six found great difficulty even in navi. you would give a score of 1). • To what extent were there times during the rehearsal when the and similarly for the ability to cre. they requested a ‘face Presence mirror’ that could be switched on This is the extent to which the individual is able to suspend and off. by their 7 disembodiment. 1 2 3 4 The results are reported through the questionnaire responses.1 Questionnaire Responses body was swivelled around automatically. To what extent where they could see their own did you feel that the rehearsal you have just experienced was similar to those rehearsals? avatars. Although they could see the This is the extent to which the computer becomes transparent facial expression of their partner and there is a sense of being with the other people in the VE. or a second Similarity to real rehearsals person disembodied point of view • Think about your recent rehearsals in real life. RESULTS each actor individually. This was displayed as a disbelief and have a sense of being in the virtual space. It was elic- small rendition of their own ava. since it is impossible to objectively quantify the degree of acting perfor. by one or more questions on a 7-point scale with 1 indicating a low It was possible for the actors to degree and 7 a high degree of the corresponding attribute. Figure 3 Questionnaire Results mance achieved. This concentrated on four issues. The ence was really taking place? actors complained in the earlier Cooperation sessions that they were not sure when the other actor was ‘looking’ • To what extent. However if you were asked this question about whether you were sitting in sity control could be switched on a room at home now. and you almost for- ate asymmetric expressions (the got about the real world of the lab in which the whole experi- ‘lopsided’ switch in the inset). mance rehearsal. if at all. the 4. so that eyes momentarily with your acting partner? locked when the avatars faced each other. Session debriefing comments including those of the director (B).and exo-centric viewpoints. rehearsal space became the reality for you. All the actors always used the first person ego-centric view. All six of the actors saw 6 the process through to the end.the head turned around more than 60 degrees from the front. Hence a gaze lock was installed. and caused a break • When you think back about your experience. but voluntarily suggested attending for ‘out of hours’ additional practice with the interface. After each of the first four sessions a brief questionnaire was ered independently of one another issued to the actors. each elicited using sliders. Five of the six had no difficulties Mean Response 5 similar with using the system after the first session. and so 2 attended 30 minutes earlier on the next occasion. do you remember in their sense of presence to shift this as more like just interacting with a computer or with other people? between ego. It was they were very concerned to elicited by two questions: ensure that their own expression • In the rehearsal you have just experienced to what extent did was as they intended. room you are in now. cooperating 3 ease with the whole process. if you were asked this question about the in Figure 2). and most important of all direct observations of the process. ment as shown by the scatter-plots of mean responses in Figure 3. By the end of the 1 process she was capable of using the system expressively. The positive results achieved were therefore surprising. move between a first person. The expression inten. and experimented with presence its possibilities. 4 copresence gating through the environment at all. did you have a sense of cooperating at them. The results on all four indicators are clear (significance tests are not appropriate for such small samples). It was believed that the actors would feel alien- ated by their inability to express themselves. Since it was you feel that the other actor was in the space with you? inconvenient. ego- centric point of view. ited by two questions: tar’s face at the top left corner of • To what extent did you have the sense of being in the rehearsal the main window (as can be seen space? (For example. there was an The authors of this paper were sceptical about the possibility improvement in all indicators over the four sessions of the experi- of this virtual rehearsal system offering the possibility of perfor. you would give a score of 7. For 4. The arms could be raised or low. The questionnaires were used as approximate indicators of what was taking place. and would quickly walk away. or off (it was left ‘on’ by default).

Spatial Organization aged to talk about their experience. the distance The actor who had the greatest difficulty with the system com. and now that that horrible beginning is kind of were beginning to use their head movements in character rather than finished . it’s the recurring nightmare - little anchor points. and if you can’t get a “Yes. and you simply looking about or trying to find each other..is to use your imagination. The last point is particularly significant. replace real rehearsal (though that is not the point)..... Their actions were more . Here are The fact that this occurred is evident on the video recordings. one made an important point. you’re on a set. with each set. to reflect the current emotional Not all actors agreed that this type of rehearsal could ever demands of the script is shown in Figure 5. It is not meant to completely replace the live rehearsal process as a whole ..you just do have an advantage here over a normal live rehearsal room. that the rehearsal system could at rehearsal system could be used for some level of acting rehearsal. part of the normal experience . between them. I couldn’t really enjoy that space. This in itself could be a initial session the actors were doubtful. where possible. One in particu.a little more - “It’s kind of becoming .. You don’t ever think about walking into a room . positive comments about the virtual rehearsal process. it looks to me like this is vocally. that again strongly fits the intended goals of the scenes (shown on the video recordings) where the man is seated at virtual rehearsal system: the table.” of the actors separately after the sessions. you’re stuck how like I imagined it’ [breaks into laughter]. “I was using it more today.. In the virtual .. after all.2 Debriefings phone conversation isn’t a real conversation. and the focus was on the simi.” An example of a such a head-move ‘in character’ is shown in Figure 4. do you actors. it’s like any actor’s worst nightmare. At this stage I think that still mented: they don’t quite trust it. we’ve only sophisticated towards the end of the exercise.. whether he thought that stances are such that meeting for real is not possible. I think it would be useful for establishing blocking.then I’m not rehearsal the actors share the same space. some of the comments: where actors are seen to be adopting the same positions and orien- Session 1 tations in the virtual and real rehearsals. you could see that it then becomes quite absorbing. However. as lar strongly missed the sense of ‘body heat’that is generated when shown in Figure 6. It is. Each one was able to do this exercise of storing up what I will be doing when I’m in real life. I was still trying to move around “. looking down a lot . you can com- municate on a telephone very effectively but using a different skill A free-format discussion was held.” sion to imagine that the rehearsal space was embedded in the “As it is now.. Director B was asked. I can smile and walk and talk like this [points to self] This is a crucial area where the virtual rehearsal system would but until I can connect the virtual reality aspects to the way that I work differ from. say.and I find myself tending to go through this the rehearsal space would be located.or not with the way that I work.. then you go on the set and you go ‘It’s nothing like you’re on stage. you’re having problems . I was going over to the window moodily The actors clearly did construct an internal mental model of and things like that.. As one actor said: “.. in the same way that a tele- 6 .one of them remarking that in even a modest they were clearly all very much at ease with the system. after Session 2. However. and you can’t move.. that’s a real experience.. Each one was asked during the debriefing ses- quite good yeh.. The actors were encour. What is impossible to show with static pictures is that sion that followed the live rehearsal.” themselves in that space in relationship to the script and the other “It’s so completely alien to the way that you usually rehearse.. Different alternatives can also be tested. I think that they were beginning to achieve it physically as well. They clearly made use of emotional facial people are near to one another. As mentioned above one of the actors commented. there are generally agreed. “I felt clumsy.. you never have your body twisted one way and your the space and objects within it are usually marked out by tape and head another way. was that after the arrangements of the actors during the play..in Session 4 an emotional sense they were varying their distance. and you know. they seemed to be physically downcast. An example of the actors choosing an appropriate dis- Post rehearsal discussion tance between themselves. and the extent to which the virtual this was a general agreement. and moving away and things like that .3 Observations doing in real life.” From a computer graphics point of view.” kind of enter into it. on-line audio or video conferencing. get these .” with complete success. For example. but debriefing room itself. where it. working out of the spatial The general impression. in the round table discus. and what’s more I think that they achieved it not only group of people together in the same room. as was frequently know what I mean? because you’re thinking about such a different set observed during the virtual rehearsal process. for the actors to collectively work out together with their director what they will be 4.but to lead into it where circum.” chalk-marks on the floor. expressions.. least be used for establishing blocking.” there.having its own kind of reality . Moreover it may of things.so that was the rehearsal space. the actors had achieved the sub-text of the session (that the woman tor B said in an interview: was waiting for a telephone call about the results of a biopsy): “We’re talking about remote rehearsal here. yes. and are able to position actually going to be rehearsing. quite truthfully I felt . modeling the set is the Session 2 simplest part of the process.4.. with the way that I am .... The actors made use of pointing gestures. and again larity to a real rehearsal. as would be expected. It expresses one of the major goals of the virtual rehearsal. As the direc. and glancing towards and away from the woman as their “Virtual rehearsing isn’t actual rehearsing.” radio play. for example. and re the remarks that Bill was making that this being like a on the right lines. and had production just the blocking itself could take a full week to agree. By the end of the process significant saving . you could see that they done it four times now. and then point to where the various items in not very much more . they also generated envelope feedback..

and the use of the space is the floor’]?”. it was impossible in a virtual rehearsal to reach out and touch. As one actress repeatedly commented. There is another scene where they are virtual performance is helpful in a transfer to a real-life perfor- standing together and each is making head nods and shakes punc. most applications of VEs the nature and style of interaction but what can be seen are transitions from virtual to real in the use between the participants is to help them achieve some other effect . it raises user expectations past what can be currently met.. where the woman actually sees nothing useful on her display (the viewpoint is too close to the texture map) the whole point of this was to create an effect in the other actor (and ultimately for the audience). and mostly to use it well. all the detail went in just whole point of the VE. In mance that was observed. of the space (Figure 7). This took the investigators by surprise. The possibility of using the head control for this effect was never told to the actors.. In a question and answer session after their such as having a discussion. mance. when an agent is so successfully human-like in its communication style . one replied: “Yes. or understanding live performance actors generally agreed that the virtual rehearsal some complex visual structure. when asked action itself which is the purpose of the application. expectations were rela- tively low: no actor ever commented on the fact that there was no lip synch for speech. On the other hand as avatars improve there is a point at which they raise expectations to such an extent that the illusion breaks. professionals. learning a new skill. once again this is subjective. Moreover. clearly wanted to Figure 4 The man is telling the woman not to worry after she impress. and really feel this in an emotionally significant way. precisely because the virtual reality was so far from reality. Just as in Figure 5. Although the purpose was rehearsal for a later real-life performance. CONCLUSIONS rehearsal? The answer is almost certainly yes: the directors agreed that what was done in the ten minutes of live rehearsal before the This paper has described a system for virtual rehearsal and an audience entered could not possibly have produced the perfor- in-depth study of its use by professional actors and directors. absolutely . This was noted in [13] where some expected behavior did not occur because: ‘.. However. Of course what was feasible is impoverished compared to what is possible in real life. together with the question of whether the in that 10 minutes we spent [‘on-the-floor’]. gestures. most of them quickly realized that they could use the mouse in order to make rapid small head moves. The actors. Figure 6 The use of pointing. However.conversation progresses.’The avatars used in the virtual rehearsal sys- Figure 5 Varying the emotional distance between themselves. in a remarkably short amount of time. The addition of the ability to make some facial gestures and to create envelope feedback compared to previous CVEs was exploited to the full by the participants. Finally. tuating their conversation in a very realistic manner. the actors learned to use it. and used everything in the armory given to them in order admits opening his letter by mistake. was there transfer from virtual rehearsal to the real-life 5. Emotional and “Was it a fruitful transition [the transfer from VR to ‘on-the- envelope expression. It was detail that I’d 7 . In virtual rehearsal. it was the impression of observers (admittedly an entirely subjective point of view) that the director’s viewpoint of the virtual rehearsal showed significant act- ing performance. had made their performance possible. Of course... tem seem to fall between these two extremes. We hypothesize that there is a type of avatar that is so poor that no useful interaction can take place between people. in spite of the relative complexity of the interface and the paucity of what was possible. to do this. For example.. it is the inter.

(1999) The power of a nod exception was a young woman who. 16-27. 8 . her aid in the VR kitchen and did his best to help her through her [14] Cassell. (1996) Disney’s Aladdin: First Steps Toward Storytell- ing in Virtual Reality. Her acting partner rushed.. Sadagic. 205-216. O’Brien. guidance of [the director]. Proceedings of the AISB 1999 Symposium on AI and Creativity in Entertainment and Visual Art.. (1999) Creating a Live Broadcast from a Virtual Environment. A. J. N. 519-539. E.. and Stenius. June 1997. Edinburgh. Computer Graphics Proceedings (SIGGRAPH). succinct control-booth Project. and Thorisson. 13 (4-5)..htm. (1996) Talk and The whole notion of ‘performance’is. K. A. J. [3] Mateas. J. (1997) An Oz-Centric Review of Interactive Drama and Believable Agents. rehearsals later wrote [18]: [11] Tromp. Pycock.edu/rps/jan99/act- ing.and Blumberg. Presence: Teleoperators and Virtual “Yes I agree that we rehearsed in that virtual reality space. conversational avatars: making communicative behaviors ring. Taylor. E.” [10] Bowers. A. tification. “It was remarkable watching the actual actors on separate monitors as Slater. (1998) Small Group Behavior in the COVEN they rehearsed on-screen under the gentle. A. [15] Frécon. M.. Wyver. Steed... ceedings (SIGGRAPH). 45-64. 20(1) January. (1996) Improv: A System for Scripting Interactive Actors in Virtual Worlds. J. PA. Frecon. 193-203. Carnegie Mellon University. M. J. rehearsal-like. M.P (1999) What’s Real about Virtual Reality? School of Drama. [9] Slater. 18(6).... C. This dimensional facial expressions.” autonomous. R. on-screen via his avatar. beyond quan. 5(3). 2. M. H. personal communica- IEEE Computer Graphics and Applications. Bowers. We would like to thank Ivor Benjamin for his help during the Distributed Systems Engineering Journal.html.. Pittsburgh. Sadagic.planned [during the virtual rehearsals]. seven [12] Steed. R. and Vilhjalmsson. Technical Report CMU- CS-97-156. Of the four couples that I witnessed. by her own admission. (1999) Fully embodied dilemma. When present.. emotional exchange was occur. http://www2. Computer Graphics Proceedings (SIGGRAPH). IEEE Virtual Reality. An off-script. [17] Waters K. and Parke F. Houston. (1999) Acting the Part. C. 91-100. School of Computer Science. was com. Sadagic. rehearsal experiments. Tromp. Research/Penn State. that’s where all the stuff came from. S. to gence. 375-384. Schroeder. (1999) The Art and Science of Synthetic Character Design.. Slater. December. (1987) A muscle model for animating three- for his post-production work on the documentary video..acm. A K Peters Ltd. J. J. vocally and gesturally. (1997) Interacting with Personality-Rich Characters Stanford Knowledge Systems Laboratory Report KSL-97-06. B. References [18] David Chambers. [6] Perlin. (1998) DIVE: A scaleable net- Acknowledgments work architecture for distributed virtual environments. (1996) Computer Facial Anima- tion. [8] Greenhalgh. and Hayes-Roth.” His partner concurred: A Comparative Study. A. of the actors clearly reached some level of emotional engagement with (1999) Leadership and collaboration in virtual environ- their virtual partner at some point. Bullock. Watson. Autonomous Agents and Multi-Agent Sys- tems. emotionally expressive and responsive.psu. J. A. [2] Reiland.. 37-51. 58-63. emotional feedback in ani- puter illiterate and completely befuddled by the keyboard/touchpad/ mated conversational agents.R. http://www. and Environments 9(1). Snoddy. [7] Pausch. March 1999. Bullock.. Professor of Acting and Directing. the emotional content was clearly perceivable.. K. November/ tion. The lone [13] Cassell.. B. K and Goldberg. Usoh. one or two were almost consistently ments. Applied Artificial Intelli- monitor set-up. [5] Kline. E. Computer Graphics Pro- project is funded by the Digital VCE. Yale [1] Brooks. (2000) Small Group Behavior in a Virtual and Real Environment: Figure 7 Virtual and Real at the same moment in the play. 21(4):17-24. Yale University. Walker.org/sig- judge this. G. J.. so the authors can only rely on the directors and actors to CHI’96 Electronic Proceedings. D. Computer Graphics Proceedings (SIGGRAPH). F.deasy. therefore. Scotland. An independent visitor during the first day only of chi/chi96/proceedings/papers/Bowers/jb_txt. Embodiment in Collaborative Virtual Environments. M.. you know. A. We would also like to thank Mitch Mitchel [16] Waters. M. J. ISBN: 1568810148. and a glance: Envelope vs. July 1999. Haseltine. of course. R. [4] Rousseau. A. IEEE CG&A. 53-63.