Acting in Virtual Reality

M. Slater, J. Howell, A.Steed, D-P. Pertaub, M. Gaurau
Department of Computer Science
University College London
Sharon Springel
Centre for Communications Systems Research
University of Cambridge

ABSTRACT manipulator for visualization. The one application in Professor
Brooks’ review that seemed to step outside of the design/engineer/
Three pairs of professional actors and a director each met in a
skill-rehearsal applications suite was psychotherapy for fear-of-
shared non-immersive virtual reality system over a two-week
flying and post-traumatic stress disorder. However, even in this
period to rehearse a short play. The actors and director never met
case, users engage with an external simulated environment in a
one another physically until a short time before a live rehearsal in
way that is consistent with the approach of behavioral exposure
front of an audience. The actors were represented by avatars which
therapy. Here a patient interacts with a psychotherapist using the
could be controlled to make a range of facial expressions, and
simulation and the experiences it evokes as a common point of
some body movements, including navigation through the space.
The study examined the extent to which virtual reality could be
A characteristic of each of these types of application is that
used by the actors and director to rehearse their later live perfor-
groups of people interact with one another through their shared
mance. Four indicators captured by questionnaires show that over
interest in some simulated objectified process: flying the aircraft,
the period of the four days their sense of presence in the virtual
viewing the design prototype, or examining the visualization. The
rehearsal space, their co-presence with the other actor, and their
virtual reality in each case provides a scenario, and the fundamen-
degree of cooperation all increased. Moreover their evaluation of
tal interaction is between each individual user and the environ-
the extent to which the virtual rehearsal was similar to a real
ment; interaction between people occurs with respect to their
rehearsal also increased. Debriefing sessions with the actors and
common interest in the scenario.
director are reported, which suggest that a performance level was
In this paper we explore a different virtual reality paradigm in
reached in the virtual rehearsal which formed the basis of a suc-
which the objective is emotionally significant interaction between
cessful live performance, one that could not have been achieved by
people, rather than between people and some other situation or
learning of lines or video conferencing.
thing. The virtual environment provides only the setting in which
Keywords person-to-person engagement takes place, it is the interaction
Virtual reality, multi-user, networked applications, facial anima- between the people, and the ‘emotional atmosphere’ which this
tion, virtual rehearsal, acting generates which is the major objective of the experience.
The scenario in which this is explored is that of dramatic act-
ing, in particular virtual rehearsal for live drama. Suppose actors
1. INTRODUCTION and their director never meet one another, except in a virtual real-
Practical applications of virtual reality are normally in the ity setting, until a short time before a live stage rehearsal. We con-
realms of engineering, product design, and skill rehearsal. Users sider the question: to what extent can they create sufficient acting
enter into a virtual environment in order to learn something new performance so that a live performance can take place with hardly
about the real situation to which the simulation corresponds, or to any live face-to-face rehearsal?
improve or learn a skill set. Professor Frederick Brooks Jnr The practical motivation for this was first suggested by the
recently carried out a survey of such applications [1] which head of research at the BBC: If it is possible for rehearsal to take
included vehicle (aircraft and boat) simulators, design and proto- place in a distributed virtual environment, then significant savings
typing systems, a NASA astronaut training system, and a nano- could be made in pre-performance production and planning.
Actors could stay situated wherever they may be in the world, con-
tinuing with their current commitments, while nevertheless devot-
ing some time to rehearsal of a new production via a shared VE.
The actors, director and others involved could get to know one
another. The technicians could work out camera angles and light-
ing in parallel with the performers working out their blocking (i.e.,
the spatial arrangements of the performance in synch with the
dynamics of the narrative).
Acting rehearsal is more than just learning of lines, which is
the least important part of the rehearsal process. Actors must act in


For example. The major advantage of such an approach is allowing the avatar itself to take care of such automatic non-verbal that live video of the real performer is distributed.. In [14] the principle of automated interacted with professional actors in a game. Vaslav Nijinsky. that relate to this paper. but that avatars ditional drama. However. together with the results of [13][14] informed the 2 . and unconsciously generated.” Vujoshevich changed from the virtual to real experience. The recently the VRMLDream company created a performance with experiment showed that envelope behavior was not only more actors directly controlling virtual characters1. virtual rehearsal for a live per. and expressions with the mouth. ciency and smoothness of the resulting conversations. i. mance? It is important to distinguish different types of avatar expres- siveness. for explore a virtual world based on the animated movie [7]. which was simulta. but also adds to the effi- The situation we are examining. Emotional feedback employs specific Research into the exploitation of virtual environments for emblems such as a smile or scrunched eyebrows to express under- drama has concentrated mostly on the creation of synthetic charac.htm [9][11][12].com The results of the series of three-person experiments in 2. http://www. I would replace Fritz’s name with ued the same task. The avatars formance. Users may control their navigation. body posture.” bility of exhibiting facial expression. significant social interaction between participants. projects at regional theaters throughout the US and Europe. Arm movements were only This describes a strategy used by actress Tania Vujoshevich in her possible for the one immersed participant in each group. Acting involves accessing and expressing the together in real life. After meeting virtually for true during rehearsal: a while they then met in the corresponding real room and contin- “In the beginning. phy changes to be agreed by the performers. reproducing an earlier result [10]. The closest to what is reported in this paper was the the user for their corresponding avatar. http://www. except that it occurred entirely in a distributed VE. BACKGROUND cacy of emotional and envelope feedback for automated avatars in conversation with humans. but the envelope communicative behavior is automated of a play has not. an experiment was described where 10 groups of 3 people each ate affect in the audience. lying emotions such as happiness or puzzlement.adding to their very high degree of ‘aliveness’and presence. and type messages to The use of a VE for rehearsal of a later ‘real life’ performance others. for example [3][4][5][6]. director and other It is argued in [14] with supporting experimental evidence. By personalizing the situation. this is behaviors not only frees the user from having to control every eye- at a cost of fragmentation of the total space: the performers never brow and mouth movement.vrmldream. The given the opportunity. To what that the avatars’lack of expressiveness (they were only able to turn extent can the impoverished substitute for emotional expression and move through the environment) impeded the development of available through today’s VE technology support acting perfor. Envelope behaviors are not consciously initiated by the A system which synthesizes a collaborative VE with live TV speaker. unlike the This was an attempt to representing people in a collaborative virtual environment. findings. These punctuate ney’s Aladdin immersed and allowed participants to interactively the conversation as a secondary feedback channel. and vocal chords. hand gestures. The participants were says. not consciously noticed by the onlooker. to regulate turn-taking and hesitations in speech. The same must be plastered on the walls of a virtual room. The avatar Repertory Theatre2. In this case members of the public is detrimental to interaction. In [13] an experiment is reported that compares the effi- 2. developed by the New York based Gertrude Stein meaning that there is interest in having a conversation. and making appropriate salutations. or browser. moments while they are typing messages. Dis. and interact back refers to non-verbal behaviors such as gaze timing and with human participants. and thus choose their own viewpoints. those neously edited and broadcast for live TV.order to rehearse. In [9] full range of human emotions in a way that produces the appropri. More example. only a constant neutral expression. precisely-timed eyebrow changes. This simulated a tra. The major rehearsal of a role in a Berchtold Brecht play [2]. important for regulating the conversational flow. Such situations have been rarely studied. head movements. 1. and pretend I would never see him again.e. an avatar may use of video-conferencing to rehearse an adaptation of the Diary of be put into a state of ‘availability’ with respect to another user. sonal and social significance. been described elsewhere in based on the current intentions of the user towards others. the exploitation of the medium for story-telling. Envelope feed- ters with believable personalities which can However. of course. tion in live theater or film. to our knowledge. envelope behaviors was applied to embodied avatars. where the par. pants. notwithstanding their extreme sim- sciously chosen. both con. The point was to see how their social interaction my fiancé’s. exhibiting such behavior were rated as more helpful and lifelike Viewers could navigate the scenario with a standard VRML compared with avatars that exhibited only emotional feedback. For example. it was always found their facial expression. also automatically carry out low level non-verbal behaviors such ticipants meet to carry out a shared task that they will later perform as blinking and breathing . but their absence broadcast is described in [8]. Actors also. during rehearsals. is from a technical point of view a CVE. In two follow-up studies their emotional state through the normal everyday employment of [11][12] similar results were found. to the display of plicity. that designers involved. The artists involved were employed on other then automatically generates the appropriate non-verbal behaviors. and thereby avoiding the ‘dead’ actually share the same space together. Another goal is distribution. helping. were that the avatars took on per- In physical reality actors have complete access. The exploit a collaborative virtual environment (CVE) as a medium in BodyChat system provides semi-autonomous avatars for partici- itself for the creation of a new form of entertainment. play from the met first in a virtual environment to try to solve a set of riddles emotional expression delivered to one another. as set by the literature. she found she was able to convey represented by simple block-like humanoid avatars. such as glancing towards the other avatar video-conferencing system was used in order to allow choreogra. with no possi- the pain and sadness and fear the character was feeling.

the rest of the time was setup. The director was a disem- until the fifth and final occasion. The kitchen and avatars consisted of about repaired. Three pairs of actors took part throughout. The actors were often sions the actors and their director met in the virtual environment reminded that the director was taking the ‘downstage’ position. About 30 minutes of this time was spent in equivalent to where the audience would sit. with a bonus after completion of the Figure 1 An Overview of the Rehearsal Space entire rehearsal process. The audience. 8Gb RAM. but on the other of about 20. (B). B was more 64MB main memory. but allow the actors to choose these as appro. on a fifth time to carry out the rehearsal for real before a live audi. (On the first day there was an addi- tional couple. One (S) had been directly involved The virtual rehearsal system was implemented using the DIVE in the design of the study for several months. before the audience was let in and an 1. prac. However.especially to be able to position the head. for about one hour. nal codification of the play that later transforms to their behavior on stage. This was followed by a debriefing their partners and in an audience. disk and 4 IR2 Graphics pipes each with 2 Raster Managers. low-level behaviors.1 The Scenario A script was developed from a training program previously used at the UK’s National Film and Television School. leading to conflict with the actors. ing a Graphics Rack with 8 R12K processors. The second ing a largely passive role . On each of the first four occa. So their ment.allowing greater freedom for the actors actor used an SGI High Impact system with 200Mhz R4400 and to establish their blocking and individual expression. frequently commenting on inch display monitor. but one of them had to withdraw due to an unex- pected acting engagement). rehearsal actors were questioned by the audience. discussion. and mutual The virtual rehearsals. Accordingly their avatars should not automatically generate evening had a round table discussion where all pairs met together envelope behaviors. were generated automatically. 3. and for them to be able to find At the end of each of the first four rehearsal sessions the actors just that way of saying it that can evoke the appropriate response in completed a short questionnaire. During the virtual rehearsals the actors’ A second way in which this application differs from traditional screen views were video-taped with an inset of their real selves in a conferencing or chat rooms is that the virtual interaction is a smaller screen window. Both actors and the director used desktop dis- ground in film and TV screenwriting and direction. The scene was a virtual kitchen with an outside ante-room rep- tice with the system. sometimes. The actors were embodied as avatars who could see and talk to ence. and two per week for two successive weeks. It was decided that the observed rehearsal was carried out. again with the scene shown on the full 19 directive. awaiting for system breakdowns to be resenting off-stage. rehearsal for real life. So another question is the extent to which The director also acted as a camera controller. not involved in the planning of the study. They then had about 10 minutes to carry out some live rehearsal. and in the tars. whole focus is on how they say of our acting rehearsal system. S directed two of the groups. priate. sessions were video-taped. as is tradi. and the last virtual some body gestures .se/dive 3 . The The actors and their director met four times virtually and then monitor was 23 inches and the full screen was used for the display. The second plays. bodied voice with no visual representation. The director used an Onyx2 system compris- how well the performance was working. THE VIRTUAL REHEARSAL STUDY 3. and answering a questionnaire. shown on a 21 inch monitor covering the full screen. and all debriefing gaze-locking. tak. such as blinking.sics. and rehearsal was on the Monday of the third week followed immedi- make arm gestures. was a co-chair of the Infinite Reality Graphics and 64M main memory. S had a strong back. The director’s view was also video-taped. The script was for a play of about five minutes. were recruited by advertisement around the depart- hand they know in advance what they’re going to say. On the fifth time the 3500 polygons and was rendered by DIVE using OpenGL calls. suggesting individual moves.3x platform [15]1. Actors pose a tough challenge: they must ately by the live rehearsal watched by an audience. 180Gb tional. in the real rehearsal space. This involved a man and a woman in a kitchen in the morning having an everyday conversa- tion about mundane matters while preparing to go out. The actors never met each other or the director physically one another in the shared virtual space. http://www. The frame-rate was at least 20 then they met immediately afterwards physically for the first time frames per second on all machines. One actor used an SGI Onyx with twin 196 MHz R10000. live performances. so that complete vir- through the virtual rehearsal process the actors construct an inter. actual rehearsal . exhibit both envelope and emotional behavior. 3. It was thought that actors must carried out by two of the investigators. tual rehearsals were recorded from this third position. actors again met their director for about 30 minutes in the VE. for the first time. There were two directors. At the end of the live therefore have significant control over the behavior of their ava. They were paid enough to cover their expenses for each meeting. The scene was Directors Guild of Great Britain. and An overview is shown in Figure 1. The four rehearsals took place actors had to have the capability for making facial expressions.

Intu. and A very simple recognition algorithm was used. this was not employed for two reasons. The main screen- speaking their lines. After a relatively short degree of practice (specifically 3. thus allowing a wide variety of facial phone call that would give the results of a biopsy. anger/surprise) were orthogonal. the body was texture-mapped for clothing. It would have been simple to provide a visual menu of facial expressions from which the actors could select by point-and-click. This is shown in the inset below. the actors could move making the actions that they carry out somewhat consistent with their avatar’s head in any direction. Or. Moreover the intensity knew if the other really meant it. head was facing. created by 3Name3D1. The first is that 3. A modified subset of the geometry was used since the original avatar had no facial animation. Some very basic animation capabilities were provided that allowed the avatar to sit. Thus in principle the head and body direction itively. This sub-text was the one to be rehearsed before the live audience.4 Other Controls actors would need to continually scan the set of displayed faces in order to choose the one that they wanted . So instead of a while leaving the body stationary. thus changing their viewpoint. hair and eye geometry was devised and fitted to the female avatar and various of the body parts were scaled and adjusted.html 4 . The face itself was obtained from a program supplied in [17]. they had been sharing eyebrows .determining a were told by the director to act according to a different sub-text. and moving through sized window showed the scenario. The body of the female avatar was an adapted version of an H- Anim compliant avatar called Nancy. rehearsal they chose a sub-text themselves. The sizes of be displayed: happiness. The face was texture-mapped to help with the required masculine or feminine appearance. izontal line would result in a neutral expression in each case. a grimace or a neutral expression . Some examples are with used for surprise and for anger. them the feeling that it was somewhat under their control. and also By using the mouse in the main window. An overlaid smaller window the environment. sadness and a neutral the gradients on each side together with the area of the triangle expression. anger. but was edited to improve the face and hair in particular. At first this movement was completely free. and also the an intensity for any one of the four major expressions and also to degree of symmetry. and other controls in the lower actively engaged in determining their facial expression . the expression that they were trying to induce. the actors controlled their facial expression by keys to move the body forwards or backwards in the direction the means of mouse strokes on an abstract ‘smiley’ type face. In addition. and remained The abstract face was divided into a top and bottom half. stand up and move its arms either singly or together.ballreich. A stroke on the top half would affect the them had crashed their brand new car. to evoke a smile they would draw one.2 The Avatar and Facial Model The male avatar originated with the DIVE system.3 The Controller for Facial Expressions by the second rehearsal session) all of the actors but one were using facial expressions with ease. triangle was determined from the stroke action. A new head. They could also use the arrow menu system. Although the script was emotionally neutral. Figure 2 Some Examples of Facial Expressions tion was based around a muscle model originally proposed in [16]. For the mouth a the accompanying animation algorithms were rewritten for DIVE. An approximate hor- shown in Figure 2. surprise. smile. each time the actors met they stroke on the bottom half would affect the mouth . The second is that we wanted the actors to be had the face controller at the top. Both the male and female avatars were fitted with a new face to enable facial animation. A stroke such as The facial animation interface allowed five major expressions to would be interpreted as ‘sad’and as ‘happy’. watching the other actor.controlling the degree of anger or surprise. The intensities were truncated to a maximum before being passed to the face half. but after the first session the actors complained that this freedom was too confusing (their heads could be twisted back-to-front reminis- cent of The Exorcist) so the system was adjusted such that when 1. so that after a party they had slept together for the first time . http://www. 3. On the fourth expressions. but last night sets (happiness/sadness. A similar method was used for the eyebrows allow for asymmetric facial expressions. were independent. Facial either one of the previ- ous ones that they had rehearsed or a new one. The code was designed to enable the actor to specify were used to determine the intensity of the expression. However. A substantially the same throughout.this at the same time as The screen was divided into two windows. These two the same house together for some time as friends.controlling the degree of For example: it was early morning and the night before one of happiness or sadness.but neither combinations of the two could be produced. Or the woman was awaiting a of each could be controlled.

cooperating 3 ease with the whole process. if you were asked this question about the in Figure 2). since it is impossible to objectively quantify the degree of acting perfor. 4 copresence gating through the environment at all. or a second Similarity to real rehearsals person disembodied point of view • Think about your recent rehearsals in real life. the 4. by their 7 disembodiment. RESULTS each actor individually. ment as shown by the scatter-plots of mean responses in Figure 3. move between a first person. and would quickly walk away. and so 2 attended 30 minutes earlier on the next occasion. 5 . By the end of the 1 process she was capable of using the system expressively. you would give a score of 1). The questionnaires were used as approximate indicators of what was taking place. room you are in now. The results on all four indicators are clear (significance tests are not appropriate for such small samples). but voluntarily suggested attending for ‘out of hours’ additional practice with the interface. 1 2 3 4 The results are reported through the questionnaire responses. For 4. It was they were very concerned to elicited by two questions: ensure that their own expression • In the rehearsal you have just experienced to what extent did was as they intended. and most important of all direct observations of the process. did you have a sense of cooperating at them. Although they could see the This is the extent to which the computer becomes transparent facial expression of their partner and there is a sense of being with the other people in the VE. The arms could be raised or low. they requested a ‘face Presence mirror’ that could be switched on This is the extent to which the individual is able to suspend and off. ego- centric point of view. Hence a gaze lock was installed. Figure 3 Questionnaire Results mance achieved.the head turned around more than 60 degrees from the front. rehearsal space became the reality for you. However if you were asked this question about whether you were sitting in sity control could be switched on a room at home now. so that eyes momentarily with your acting partner? locked when the avatars faced each other.and exo-centric viewpoints. Co-presence point.1 Questionnaire Responses body was swivelled around automatically. and for all as a group. All six of the actors saw 6 the process through to the end. All the actors always used the first person ego-centric view. was at first extremely ill-at. and experimented with presence its possibilities. This concentrated on four issues. The positive results achieved were therefore surprising. One of the six found great difficulty even in navi. do you remember in their sense of presence to shift this as more like just interacting with a computer or with other people? between ego. Session debriefing comments including those of the director (B). if at all. mance rehearsal. This was displayed as a disbelief and have a sense of being in the virtual space. The expression inten. by one or more questions on a 7-point scale with 1 indicating a low It was possible for the actors to degree and 7 a high degree of the corresponding attribute. The ence was really taking place? actors complained in the earlier Cooperation sessions that they were not sure when the other actor was ‘looking’ • To what extent. After each of the first four sessions a brief questionnaire was ered independently of one another issued to the actors. you would give a score of 7. It was elic- small rendition of their own ava. and caused a break • When you think back about your experience. • To what extent were there times during the rehearsal when the and similarly for the ability to cre. there was an The authors of this paper were sceptical about the possibility improvement in all indicators over the four sessions of the experi- of this virtual rehearsal system offering the possibility of perfor. ited by two questions: tar’s face at the top left corner of • To what extent did you have the sense of being in the rehearsal the main window (as can be seen space? (For example. and you almost for- ate asymmetric expressions (the got about the real world of the lab in which the whole experi- ‘lopsided’ switch in the inset). each elicited using sliders. It was believed that the actors would feel alien- ated by their inability to express themselves. Since it was you feel that the other actor was in the space with you? inconvenient. Five of the six had no difficulties Mean Response 5 similar with using the system after the first session. or off (it was left ‘on’ by default). To what extent where they could see their own did you feel that the rehearsal you have just experienced was similar to those rehearsals? avatars.

it’s the recurring nightmare - little anchor points.. quite truthfully I felt .” with complete success. I can smile and walk and talk like this [points to self] This is a crucial area where the virtual rehearsal system would but until I can connect the virtual reality aspects to the way that I work differ from.” radio play. where Session 4 an emotional sense they were varying their distance. Each one was asked during the debriefing ses- quite good yeh. I was still trying to move around “.and I find myself tending to go through this the rehearsal space would be located. between them.” there. you’re stuck how like I imagined it’ [breaks into laughter].one of them remarking that in even a modest they were clearly all very much at ease with the system.a little more - “It’s kind of becoming .” kind of enter into it... after all. and the focus was on the simi. modeling the set is the Session 2 simplest part of the process. However. replace real rehearsal (though that is not the point).. part of the normal experience . was that after the arrangements of the actors during the play. it’s like any actor’s worst nightmare. and if you can’t get a “Yes. and re the remarks that Bill was making that this being like a on the right lines. “I felt clumsy.. I couldn’t really enjoy that space. they seemed to be physically downcast.” sion to imagine that the rehearsal space was embedded in the “As it is now. and then point to where the various items in not very much more . as lar strongly missed the sense of ‘body heat’that is generated when shown in Figure 6. one made an important point. some of the comments: where actors are seen to be adopting the same positions and orien- Session 1 tations in the virtual and real rehearsals.. It expresses one of the major goals of the virtual rehearsal. that again strongly fits the intended goals of the scenes (shown on the video recordings) where the man is seated at virtual rehearsal system: the table. The actors were encour. there are generally agreed. and again larity to a real rehearsal. One in particu.. It is not meant to completely replace the live rehearsal process as a whole . and you can’t move. get these . on-line audio or video conferencing.. I think it would be useful for establishing blocking. Each one was able to do this exercise of storing up what I will be doing when I’m in real life. expressions. with the way that I am . It is.2 Debriefings phone conversation isn’t a real conversation. This in itself could be a initial session the actors were doubtful. In the virtual . for example. However. Spatial Organization aged to talk about their experience..then I’m not rehearsal the actors share the same space. Their actions were more .. then you go on the set and you go ‘It’s nothing like you’re on stage. “I was using it more today. they also generated envelope feedback. and glancing towards and away from the woman as their “Virtual rehearsing isn’t actual rehearsing.3 Observations doing in real life. I think that they were beginning to achieve it physically as well. the actors had achieved the sub-text of the session (that the woman tor B said in an interview: was waiting for a telephone call about the results of a biopsy): “We’re talking about remote rehearsal here. in the same way that a tele- 6 .. you never have your body twisted one way and your the space and objects within it are usually marked out by tape and head another way. As one actor said: “. yes. An example of the actors choosing an appropriate dis- Post rehearsal discussion tance between themselves... in the round table discus.. you’re on a set. say. working out of the spatial The general impression. where it. I was going over to the window moodily The actors clearly did construct an internal mental model of and things like that. Director B was asked.. Here are The fact that this occurred is evident on the video recordings.. as was frequently know what I mean? because you’re thinking about such a different set observed during the virtual rehearsal process. and moving away and things like that . and you simply looking about or trying to find each other.” themselves in that space in relationship to the script and the other “It’s so completely alien to the way that you usually to use your imagination. You don’t ever think about walking into a room .. to reflect the current emotional Not all actors agreed that this type of rehearsal could ever demands of the script is shown in Figure 5. for the actors to collectively work out together with their director what they will be 4. As mentioned above one of the actors commented. and what’s more I think that they achieved it not only group of people together in the same that was the rehearsal space.. and now that that horrible beginning is kind of were beginning to use their head movements in character rather than finished .” of the actors separately after the sessions.” From a computer graphics point of view.having its own kind of reality .” An example of a such a head-move ‘in character’ is shown in Figure 4. you could see that they done it four times now. you’re having problems . looking down a lot . and the extent to which the virtual this was a general agreement. least be used for establishing blocking. we’ve only sophisticated towards the end of the exercise. that the rehearsal system could at rehearsal system could be used for some level of acting just do have an advantage here over a normal live rehearsal room. it looks to me like this is vocally. The last point is particularly significant.” chalk-marks on the floor.. whether he thought that stances are such that meeting for real is not possible. the distance The actor who had the greatest difficulty with the system com. but debriefing room itself. The actors made use of pointing gestures. as would be expected.. Different alternatives can also be tested.. that’s a real experience. and you know..but to lead into it where circum. By the end of the process significant saving .. do you actors. What is impossible to show with static pictures is that sion that followed the live rehearsal. positive comments about the virtual rehearsal process. you can com- municate on a telephone very effectively but using a different skill A free-format discussion was held. and are able to position actually going to be rehearsing.4.or not with the way that I work.. you could see that it then becomes quite absorbing. They clearly made use of emotional facial people are near to one another. after Session 2. For example. and had production just the blocking itself could take a full week to agree.. As the direc. At this stage I think that still mented: they don’t quite trust it. Moreover it may of things. with each set..

. where the woman actually sees nothing useful on her display (the viewpoint is too close to the texture map) the whole point of this was to create an effect in the other actor (and ultimately for the audience). most of them quickly realized that they could use the mouse in order to make rapid small head moves. it raises user expectations past what can be currently met. As one actress repeatedly commented. had made their performance possible. and used everything in the armory given to them in order admits opening his letter by mistake. Although the purpose was rehearsal for a later real-life performance. This was noted in [13] where some expected behavior did not occur because: ‘.. expectations were rela- tively low: no actor ever commented on the fact that there was no lip synch for speech. The actors. There is another scene where they are virtual performance is helpful in a transfer to a real-life perfor- standing together and each is making head nods and shakes punc. Figure 6 The use of pointing. Finally. and mostly to use it well. The possibility of using the head control for this effect was never told to the actors.. learning a new skill. However. Emotional and “Was it a fruitful transition [the transfer from VR to ‘on-the- envelope expression. precisely because the virtual reality was so far from reality. Of course. and really feel this in an emotionally significant way.. In virtual rehearsal.. all the detail went in just whole point of the VE. For example. Just as in Figure 5. It was detail that I’d 7 . and the use of the space is the floor’]?”. when an agent is so successfully human-like in its communication style . most applications of VEs the nature and style of interaction but what can be seen are transitions from virtual to real in the use between the participants is to help them achieve some other effect . in spite of the relative complexity of the interface and the paucity of what was possible. tem seem to fall between these two extremes. professionals. Of course what was feasible is impoverished compared to what is possible in real life.conversation progresses. one replied: “Yes. of the space (Figure 7). This took the investigators by surprise. However. to do this. it was impossible in a virtual rehearsal to reach out and touch. the actors learned to use it. We hypothesize that there is a type of avatar that is so poor that no useful interaction can take place between people. mance. in a remarkably short amount of time.’The avatars used in the virtual rehearsal sys- Figure 5 Varying the emotional distance between themselves. clearly wanted to Figure 4 The man is telling the woman not to worry after she impress. gestures. together with the question of whether the in that 10 minutes we spent [‘on-the-floor’].. it was the impression of observers (admittedly an entirely subjective point of view) that the director’s viewpoint of the virtual rehearsal showed significant act- ing performance. tuating their conversation in a very realistic manner. or understanding live performance actors generally agreed that the virtual rehearsal some complex visual structure. it is the inter. In a question and answer session after their such as having a discussion. In mance that was observed. absolutely . On the other hand as avatars improve there is a point at which they raise expectations to such an extent that the illusion breaks. Moreover. CONCLUSIONS rehearsal? The answer is almost certainly yes: the directors agreed that what was done in the ten minutes of live rehearsal before the This paper has described a system for virtual rehearsal and an audience entered could not possibly have produced the perfor- in-depth study of its use by professional actors and directors. when asked action itself which is the purpose of the application. The addition of the ability to make some facial gestures and to create envelope feedback compared to previous CVEs was exploited to the full by the participants. once again this is subjective. was there transfer from virtual rehearsal to the real-life 5.

personal communica- IEEE Computer Graphics and Applications. Taylor. [17] Waters K. rehearsal experiments. Pittsburgh. rehearsals later wrote [18]: [11] Tromp. Of the four couples that I witnessed. A. N. (1997) An Oz-Centric Review of Interactive Drama and Believable Agents. Pycock. Haseltine. G. the emotional content was clearly perceivable.planned [during the virtual rehearsals]. J. on-screen via his avatar. A. of the actors clearly reached some level of emotional engagement with (1999) Leadership and collaboration in virtual environ- their virtual partner at some point. (2000) Small Group Behavior in a Virtual and Real Environment: Figure 7 Virtual and Real at the same moment in the play. seven [12] Steed. [7] Pausch. [8] Greenhalgh. succinct control-booth Project. and Environments 9(1). We would like to thank Ivor Benjamin for his help during the Distributed Systems Engineering Journal.acm. (1999) Creating a Live Broadcast from a Virtual Environment. Usoh. J. Presence: Teleoperators and Virtual “Yes I agree that we rehearsed in that virtual reality space. ceedings (SIGGRAPH). Bowers. Technical Report CMU- CS-97-156. Applied Artificial Intelli- monitor set-up. emotionally expressive and responsive. March 1999. and Stenius.deasy. J. (1996) Disney’s Aladdin: First Steps Toward Storytell- ing in Virtual Reality. and Thorisson. Autonomous Agents and Multi-Agent Sys- tems... 91-100. C. Carnegie Mellon University. therefore. Watson. 18(6). A. Computer Graphics Pro- project is funded by the Digital VCE. [4] Rousseau. 5(3)..P (1999) What’s Real about Virtual Reality? School of Drama. (1996) Improv: A System for Scripting Interactive Actors in Virtual Worlds. emotional feedback in ani- puter illiterate and completely befuddled by the keyboard/touchpad/ mated conversational agents. Slater. June 1997. December.” His partner concurred: A Comparative Study. B. R. conversational avatars: making communicative behaviors ring. of course. July 1999. D. (1999) Acting the Part. http://www. Proceedings of the AISB 1999 Symposium on AI and Creativity in Entertainment and Visual Art. http://www2. B. 8 . J. R.. was com. Computer Graphics Proceedings (SIGGRAPH).. J. E. Yale [1] Brooks. and Vilhjalmsson. Her acting partner rushed. J. An independent visitor during the first day only of chi/chi96/proceedings/papers/Bowers/jb_txt. guidance of [the director]. K and Goldberg.. Tromp. Professor of Acting and Directing. An off-script. (1999) The Art and Science of Synthetic Character Design... (1996) Talk and The whole notion of ‘performance’is. her aid in the VR kitchen and did his best to help her through her [14] Cassell. vocally and gesturally. [3] Mateas...” autonomous. J. M. 193-203. 205-216. beyond quan. 58-63. J. This dimensional facial expressions. IEEE Virtual Reality. E. (1998) Small Group Behavior in the COVEN they rehearsed on-screen under the gentle. [2] Reiland. ISBN: 1568810148. O’Brien. F.and Blumberg.. [5] Kline. C. M.. judge this.. M.. 519-539. A. A. S. 13 (4-5). that’s where all the stuff came from. (1998) DIVE: A scaleable net- Acknowledgments work architecture for distributed virtual environments. to gence. Sadagic.R. 21(4):17-24. 16-27. [15] Frécon. J.. We would also like to thank Mitch Mitchel [16] Waters. rehearsal-like. PA.psu. E. M. one or two were almost consistently ments. H. Wyver. M. Sadagic. A K Peters Ltd. and Hayes-Roth. Houston. 375-384. and Parke F. K.html. Frecon. Steed. so the authors can only rely on the directors and actors to CHI’96 Electronic Proceedings. (1999) Fully embodied dilemma. The lone [13] ing. 20(1) January. by her own admission. Scotland. Sadagic. “It was remarkable watching the actual actors on separate monitors as Slater. [6] Perlin. Schroeder.” [10] Bowers. School of Computer Science. and a glance: Envelope vs. Snoddy. Walker.htm. Edinburgh. Bullock. (1996) Computer Facial Anima- tion. A.. Research/Penn State. 2. J. References [18] David Chambers. emotional exchange was occur. tification. 53-63. Embodiment in Collaborative Virtual Environments.. Bullock. M. 37-51. K. Yale University. November/ tion. Computer Graphics Proceedings (SIGGRAPH).. (1997) Interacting with Personality-Rich Characters Stanford Knowledge Systems Laboratory Report KSL-97-06.. A. When present.. IEEE CG&A. (1987) A muscle model for animating three- for his post-production work on the documentary video. Computer Graphics Proceedings (SIGGRAPH). you know. A. [9] Slater. R. (1999) The power of a nod exception was a young woman who.

Sign up to vote on this title
UsefulNot useful