Acting in Virtual Reality

M. Slater, J. Howell, A.Steed, D-P. Pertaub, M. Gaurau
Department of Computer Science
University College London
Sharon Springel
Centre for Communications Systems Research
University of Cambridge

ABSTRACT manipulator for visualization. The one application in Professor
Brooks’ review that seemed to step outside of the design/engineer/
Three pairs of professional actors and a director each met in a
skill-rehearsal applications suite was psychotherapy for fear-of-
shared non-immersive virtual reality system over a two-week
flying and post-traumatic stress disorder. However, even in this
period to rehearse a short play. The actors and director never met
case, users engage with an external simulated environment in a
one another physically until a short time before a live rehearsal in
way that is consistent with the approach of behavioral exposure
front of an audience. The actors were represented by avatars which
therapy. Here a patient interacts with a psychotherapist using the
could be controlled to make a range of facial expressions, and
simulation and the experiences it evokes as a common point of
some body movements, including navigation through the space.
contact.
The study examined the extent to which virtual reality could be
A characteristic of each of these types of application is that
used by the actors and director to rehearse their later live perfor-
groups of people interact with one another through their shared
mance. Four indicators captured by questionnaires show that over
interest in some simulated objectified process: flying the aircraft,
the period of the four days their sense of presence in the virtual
viewing the design prototype, or examining the visualization. The
rehearsal space, their co-presence with the other actor, and their
virtual reality in each case provides a scenario, and the fundamen-
degree of cooperation all increased. Moreover their evaluation of
tal interaction is between each individual user and the environ-
the extent to which the virtual rehearsal was similar to a real
ment; interaction between people occurs with respect to their
rehearsal also increased. Debriefing sessions with the actors and
common interest in the scenario.
director are reported, which suggest that a performance level was
In this paper we explore a different virtual reality paradigm in
reached in the virtual rehearsal which formed the basis of a suc-
which the objective is emotionally significant interaction between
cessful live performance, one that could not have been achieved by
people, rather than between people and some other situation or
learning of lines or video conferencing.
thing. The virtual environment provides only the setting in which
Keywords person-to-person engagement takes place, it is the interaction
Virtual reality, multi-user, networked applications, facial anima- between the people, and the ‘emotional atmosphere’ which this
tion, virtual rehearsal, acting generates which is the major objective of the experience.
The scenario in which this is explored is that of dramatic act-
ing, in particular virtual rehearsal for live drama. Suppose actors
1. INTRODUCTION and their director never meet one another, except in a virtual real-
Practical applications of virtual reality are normally in the ity setting, until a short time before a live stage rehearsal. We con-
realms of engineering, product design, and skill rehearsal. Users sider the question: to what extent can they create sufficient acting
enter into a virtual environment in order to learn something new performance so that a live performance can take place with hardly
about the real situation to which the simulation corresponds, or to any live face-to-face rehearsal?
improve or learn a skill set. Professor Frederick Brooks Jnr The practical motivation for this was first suggested by the
recently carried out a survey of such applications [1] which head of research at the BBC: If it is possible for rehearsal to take
included vehicle (aircraft and boat) simulators, design and proto- place in a distributed virtual environment, then significant savings
typing systems, a NASA astronaut training system, and a nano- could be made in pre-performance production and planning.
Actors could stay situated wherever they may be in the world, con-
tinuing with their current commitments, while nevertheless devot-
ing some time to rehearsal of a new production via a shared VE.
The actors, director and others involved could get to know one
another. The technicians could work out camera angles and light-
ing in parallel with the performers working out their blocking (i.e.,
the spatial arrangements of the performance in synch with the
dynamics of the narrative).
Acting rehearsal is more than just learning of lines, which is
the least important part of the rehearsal process. Actors must act in

1

play from the met first in a virtual environment to try to solve a set of riddles emotional expression delivered to one another. i. Actors also. 1. This simulated a tra. both con. I would replace Fritz’s name with ued the same task. developed by the New York based Gertrude Stein meaning that there is interest in having a conversation. The recently the VRMLDream company created a performance with experiment showed that envelope behavior was not only more actors directly controlling virtual characters1. important for regulating the conversational flow. Arm movements were only This describes a strategy used by actress Tania Vujoshevich in her possible for the one immersed participant in each group. ciency and smoothness of the resulting conversations. In [9] full range of human emotions in a way that produces the appropri. envelope behaviors was applied to embodied avatars. and making appropriate salutations. To what that the avatars’lack of expressiveness (they were only able to turn extent can the impoverished substitute for emotional expression and move through the environment) impeded the development of available through today’s VE technology support acting perfor. as set by the literature. More example. However. For example. projects at regional theaters throughout the US and Europe. findings. not consciously noticed by the onlooker. is from a technical point of view a CVE. the exploitation of the medium for story-telling. The given the opportunity. been described elsewhere in based on the current intentions of the user towards others. This was an attempt to representing people in a collaborative virtual environment.com The results of the series of three-person experiments in 2. body posture. head movements. In this case members of the public is detrimental to interaction. The point was to see how their social interaction my fiancé’s. with no possi- the pain and sadness and fear the character was feeling. The major advantage of such an approach is allowing the avatar itself to take care of such automatic non-verbal that live video of the real performer is distributed. phy changes to be agreed by the performers. Envelope feed- ters with believable personalities which can perform. tion in live theater or film. hand gestures. but the envelope communicative behavior is automated of a play has not. for example [3][4][5][6]. and thereby avoiding the ‘dead’ actually share the same space together. moments while they are typing messages. The participants were says. http://www. precisely-timed eyebrow changes. These punctuate ney’s Aladdin immersed and allowed participants to interactively the conversation as a secondary feedback channel. she found she was able to convey represented by simple block-like humanoid avatars. and interact back refers to non-verbal behaviors such as gaze timing and with human participants. The same must be plastered on the walls of a virtual room.ibm. but also adds to the effi- The situation we are examining. mance? It is important to distinguish different types of avatar expres- siveness. The avatar Repertory Theatre2. virtual rehearsal for a live per. for explore a virtual world based on the animated movie [7]. Such situations have been rarely studied. that designers involved. For example. Dis. and expressions with the mouth.htm [9][11][12]. http://www. such as glancing towards the other avatar video-conferencing system was used in order to allow choreogra. that relate to this paper. only a constant neutral expression. The avatars formance. pants. reproducing an earlier result [10]. Another goal is distribution. exhibiting such behavior were rated as more helpful and lifelike Viewers could navigate the scenario with a standard VRML compared with avatars that exhibited only emotional feedback. those neously edited and broadcast for live TV. except that it occurred entirely in a distributed VE. and pretend I would never see him again. were that the avatars took on per- In physical reality actors have complete access. The exploit a collaborative virtual environment (CVE) as a medium in BodyChat system provides semi-autonomous avatars for partici- itself for the creation of a new form of entertainment.” bility of exhibiting facial expression. also automatically carry out low level non-verbal behaviors such ticipants meet to carry out a shared task that they will later perform as blinking and breathing . director and other It is argued in [14] with supporting experimental evidence. and thus choose their own viewpoints. The artists involved were employed on other then automatically generates the appropriate non-verbal behaviors. and type messages to The use of a VE for rehearsal of a later ‘real life’ performance others. unlike the situa. In two follow-up studies their emotional state through the normal everyday employment of [11][12] similar results were found. lying emotions such as happiness or puzzlement. to our knowledge. significant social interaction between participants. BACKGROUND cacy of emotional and envelope feedback for automated avatars in conversation with humans.” Vujoshevich changed from the virtual to real experience. to regulate turn-taking and hesitations in speech. this is behaviors not only frees the user from having to control every eye- at a cost of fragmentation of the total space: the performers never brow and mouth movement. Acting involves accessing and expressing the together in real life. helping. Users may control their navigation. but that avatars ditional drama. which was simulta.adding to their very high degree of ‘aliveness’and presence. and unconsciously generated. Emotional feedback employs specific Research into the exploitation of virtual environments for emblems such as a smile or scrunched eyebrows to express under- drama has concentrated mostly on the creation of synthetic charac.. In [13] an experiment is reported that compares the effi- 2.e. In [14] the principle of automated interacted with professional actors in a game. or browser.com/sfasp/vr. However. notwithstanding their extreme sim- sciously chosen.order to rehearse. it was always found their facial expression. The closest to what is reported in this paper was the the user for their corresponding avatar.vrmldream. Envelope behaviors are not consciously initiated by the A system which synthesizes a collaborative VE with live TV speaker. The major rehearsal of a role in a Berchtold Brecht play [2]. to the display of plicity. of course. After meeting virtually for true during rehearsal: a while they then met in the corresponding real room and contin- “In the beginning. during rehearsals. but their absence broadcast is described in [8]. an experiment was described where 10 groups of 3 people each ate affect in the audience. By personalizing the situation. an avatar may use of video-conferencing to rehearse an adaptation of the Diary of be put into a state of ‘availability’ with respect to another user. Vaslav Nijinsky. together with the results of [13][14] informed the 2 . and vocal chords. sonal and social significance. where the par.

It was thought that actors must carried out by two of the investigators. awaiting for system breakdowns to be resenting off-stage. Actors pose a tough challenge: they must ately by the live rehearsal watched by an audience. frequently commenting on inch display monitor.allowing greater freedom for the actors actor used an SGI High Impact system with 200Mhz R4400 and to establish their blocking and individual expression. whole focus is on how they say it. On each of the first four occa. They then had about 10 minutes to carry out some live rehearsal. There were two directors. discussion. not involved in the planning of the study. rehearsal actors were questioned by the audience. were recruited by advertisement around the depart- hand they know in advance what they’re going to say. (B). http://www. and for them to be able to find At the end of each of the first four rehearsal sessions the actors just that way of saying it that can evoke the appropriate response in completed a short questionnaire. One actor used an SGI Onyx with twin 196 MHz R10000. 180Gb tional. THE VIRTUAL REHEARSAL STUDY 3. The actors never met each other or the director physically one another in the shared virtual space.design of our acting rehearsal system. Three pairs of actors took part throughout. The second ing a largely passive role .sics. The director was a disem- until the fifth and final occasion. and answering a questionnaire. The actors were often sions the actors and their director met in the virtual environment reminded that the director was taking the ‘downstage’ position.3x platform [15]1. on a fifth time to carry out the rehearsal for real before a live audi. S directed two of the groups. S had a strong back. live performances. for about one hour. and two per week for two successive weeks. again with the scene shown on the full 19 directive. and in the tars. The second plays. At the end of the live therefore have significant control over the behavior of their ava. The audience. and mutual The virtual rehearsals. such as blinking. sessions were video-taped. One (S) had been directly involved The virtual rehearsal system was implemented using the DIVE in the design of the study for several months. leading to conflict with the actors. It was decided that the observed rehearsal was carried out. The actors were embodied as avatars who could see and talk to ence. nal codification of the play that later transforms to their behavior on stage. was a co-chair of the Infinite Reality Graphics and 64M main memory. sometimes. with a bonus after completion of the Figure 1 An Overview of the Rehearsal Space entire rehearsal process. in the real rehearsal space. The four rehearsals took place actors had to have the capability for making facial expressions. and rehearsal was on the Monday of the third week followed immedi- make arm gestures.the rest of the time was setup. prac. low-level behaviors. before the audience was let in and an 1.1 The Scenario A script was developed from a training program previously used at the UK’s National Film and Television School. but one of them had to withdraw due to an unex- pected acting engagement).se/dive 3 . disk and 4 IR2 Graphics pipes each with 2 Raster Managers. This involved a man and a woman in a kitchen in the morning having an everyday conversa- tion about mundane matters while preparing to go out. The director used an Onyx2 system compris- how well the performance was working. as is tradi. were generated automatically. The script was for a play of about five minutes. Both actors and the director used desktop dis- ground in film and TV screenwriting and direction. They were paid enough to cover their expenses for each meeting. priate. About 30 minutes of this time was spent in equivalent to where the audience would sit. suggesting individual moves. However. but on the other of about 20. for the first time. The director’s view was also video-taped. but allow the actors to choose these as appro. B was more 64MB main memory. During the virtual rehearsals the actors’ A second way in which this application differs from traditional screen views were video-taped with an inset of their real selves in a conferencing or chat rooms is that the virtual interaction is a smaller screen window. ing a Graphics Rack with 8 R12K processors. so that complete vir- through the virtual rehearsal process the actors construct an inter. actors again met their director for about 30 minutes in the VE. The scene was a virtual kitchen with an outside ante-room rep- tice with the system. On the fifth time the 3500 polygons and was rendered by DIVE using OpenGL calls. This was followed by a debriefing their partners and in an audience. rehearsal for real life. exhibit both envelope and emotional behavior. and An overview is shown in Figure 1. So another question is the extent to which The director also acted as a camera controller. shown on a 21 inch monitor covering the full screen. The kitchen and avatars consisted of about repaired. So their ment. 3. (On the first day there was an addi- tional couple.especially to be able to position the head. 3. tak. and the last virtual some body gestures . tual rehearsals were recorded from this third position. The frame-rate was at least 20 then they met immediately afterwards physically for the first time frames per second on all machines. and all debriefing gaze-locking. The scene was Directors Guild of Great Britain. actual rehearsal . The The actors and their director met four times virtually and then monitor was 23 inches and the full screen was used for the display. bodied voice with no visual representation. Accordingly their avatars should not automatically generate evening had a round table discussion where all pairs met together envelope behaviors. 8Gb RAM.

and also the an intensity for any one of the four major expressions and also to degree of symmetry. rehearsal they chose a sub-text themselves. sadness and a neutral the gradients on each side together with the area of the triangle expression. stand up and move its arms either singly or together. Moreover the intensity knew if the other really meant it. anger. 3. them the feeling that it was somewhat under their control. created by 3Name3D1. triangle was determined from the stroke action. The code was designed to enable the actor to specify were used to determine the intensity of the expression. a grimace or a neutral expression . and also By using the mouse in the main window. The intensities were truncated to a maximum before being passed to the face plugin. thus changing their viewpoint. At first this movement was completely free. A stroke on the top half would affect the them had crashed their brand new car. they had been sharing eyebrows . but last night sets (happiness/sadness. but after the first session the actors complained that this freedom was too confusing (their heads could be twisted back-to-front reminis- cent of The Exorcist) so the system was adjusted such that when 1. and A very simple recognition algorithm was used. A new head. So instead of a while leaving the body stationary.4 Other Controls actors would need to continually scan the set of displayed faces in order to choose the one that they wanted . Both the male and female avatars were fitted with a new face to enable facial animation. For the mouth a the accompanying animation algorithms were rewritten for DIVE. so that after a party they had slept together for the first time . and moving through sized window showed the scenario. http://www.giving half. The second is that we wanted the actors to be had the face controller at the top. Although the script was emotionally neutral. head was facing. and remained The abstract face was divided into a top and bottom half. After a relatively short degree of practice (specifically 3. A modified subset of the geometry was used since the original avatar had no facial animation. Or the woman was awaiting a of each could be controlled. smile. watching the other actor. and other controls in the lower actively engaged in determining their facial expression .but neither combinations of the two could be produced. but was edited to improve the face and hair in particular.determining a were told by the director to act according to a different sub-text. anger/surprise) were orthogonal. Or. the body was texture-mapped for clothing. An overlaid smaller window the environment. This sub-text was the one to be rehearsed before the live audience. The body of the female avatar was an adapted version of an H- Anim compliant avatar called Nancy. thus allowing a wide variety of facial phone call that would give the results of a biopsy. A similar method was used for the eyebrows allow for asymmetric facial expressions. hair and eye geometry was devised and fitted to the female avatar and various of the body parts were scaled and adjusted. The main screen- speaking their lines. were independent. On the fourth expressions. They could also use the arrow menu system. The first is that 3. These two the same house together for some time as friends. This is shown in the inset below.3 The Controller for Facial Expressions by the second rehearsal session) all of the actors but one were using facial expressions with ease.controlling the degree of For example: it was early morning and the night before one of happiness or sadness. The face was texture-mapped to help with the required masculine or feminine appearance. Intu. However. It would have been simple to provide a visual menu of facial expressions from which the actors could select by point-and-click. An approximate hor- shown in Figure 2.controlling the degree of anger or surprise. Facial anima. the actors could move making the actions that they carry out somewhat consistent with their avatar’s head in any direction. The sizes of be displayed: happiness. A stroke such as The facial animation interface allowed five major expressions to would be interpreted as ‘sad’and as ‘happy’. izontal line would result in a neutral expression in each case.ballreich. Some very basic animation capabilities were provided that allowed the avatar to sit. surprise.html 4 . In addition.net/vrml/h-anim/h-anim-examples.this at the same time as The screen was divided into two windows. Some examples are with used for surprise and for anger. Thus in principle the head and body direction itively. either one of the previ- ous ones that they had rehearsed or a new one. Figure 2 Some Examples of Facial Expressions tion was based around a muscle model originally proposed in [16]. to evoke a smile they would draw one. each time the actors met they stroke on the bottom half would affect the mouth . the actors controlled their facial expression by keys to move the body forwards or backwards in the direction the means of mouse strokes on an abstract ‘smiley’ type face.2 The Avatar and Facial Model The male avatar originated with the DIVE system. A substantially the same throughout. The face itself was obtained from a program supplied in [17]. this was not employed for two reasons. the expression that they were trying to induce.

rehearsal space became the reality for you. Five of the six had no difficulties Mean Response 5 similar with using the system after the first session. and you almost for- ate asymmetric expressions (the got about the real world of the lab in which the whole experi- ‘lopsided’ switch in the inset). you would give a score of 7. For 4. It was believed that the actors would feel alien- ated by their inability to express themselves. By the end of the 1 process she was capable of using the system expressively. After each of the first four sessions a brief questionnaire was ered independently of one another issued to the actors. each elicited using sliders. This concentrated on four issues. and caused a break • When you think back about your experience. The positive results achieved were therefore surprising.1 Questionnaire Responses body was swivelled around automatically. Hence a gaze lock was installed. Although they could see the This is the extent to which the computer becomes transparent facial expression of their partner and there is a sense of being with the other people in the VE. 5 . and so 2 attended 30 minutes earlier on the next occasion. Session debriefing comments including those of the director (B). It was they were very concerned to elicited by two questions: ensure that their own expression • In the rehearsal you have just experienced to what extent did was as they intended. did you have a sense of cooperating at them. It was elic- small rendition of their own ava. room you are in now. ego- centric point of view. To what extent where they could see their own did you feel that the rehearsal you have just experienced was similar to those rehearsals? avatars. so that eyes momentarily with your acting partner? locked when the avatars faced each other. if you were asked this question about the in Figure 2).the head turned around more than 60 degrees from the front. • To what extent were there times during the rehearsal when the and similarly for the ability to cre. they requested a ‘face Presence mirror’ that could be switched on This is the extent to which the individual is able to suspend and off. and experimented with presence its possibilities. by their 7 disembodiment. since it is impossible to objectively quantify the degree of acting perfor. 4 copresence gating through the environment at all. One of the six found great difficulty even in navi. or off (it was left ‘on’ by default). there was an The authors of this paper were sceptical about the possibility improvement in all indicators over the four sessions of the experi- of this virtual rehearsal system offering the possibility of perfor. 1 2 3 4 The results are reported through the questionnaire responses. ited by two questions: tar’s face at the top left corner of • To what extent did you have the sense of being in the rehearsal the main window (as can be seen space? (For example. The results on all four indicators are clear (significance tests are not appropriate for such small samples). However if you were asked this question about whether you were sitting in sity control could be switched on a room at home now. was at first extremely ill-at. The arms could be raised or low. RESULTS each actor individually. or a second Similarity to real rehearsals person disembodied point of view • Think about your recent rehearsals in real life. do you remember in their sense of presence to shift this as more like just interacting with a computer or with other people? between ego. All the actors always used the first person ego-centric view. move between a first person. and for all as a group. Figure 3 Questionnaire Results mance achieved. but voluntarily suggested attending for ‘out of hours’ additional practice with the interface. mance rehearsal. by one or more questions on a 7-point scale with 1 indicating a low It was possible for the actors to degree and 7 a high degree of the corresponding attribute. and most important of all direct observations of the process. All six of the actors saw 6 the process through to the end. The ence was really taking place? actors complained in the earlier Cooperation sessions that they were not sure when the other actor was ‘looking’ • To what extent. if at all. ment as shown by the scatter-plots of mean responses in Figure 3. the 4. cooperating 3 ease with the whole process.and exo-centric viewpoints. The expression inten. you would give a score of 1). The questionnaires were used as approximate indicators of what was taking place. This was displayed as a disbelief and have a sense of being in the virtual space. Since it was you feel that the other actor was in the space with you? inconvenient. Co-presence point. and would quickly walk away.

you never have your body twisted one way and your the space and objects within it are usually marked out by tape and head another way. they seemed to be physically downcast. It expresses one of the major goals of the virtual rehearsal. Their actions were more .is to use your imagination.” of the actors separately after the sessions. get these . replace real rehearsal (though that is not the point). and again larity to a real rehearsal. Here are The fact that this occurred is evident on the video recordings.” An example of a such a head-move ‘in character’ is shown in Figure 4. The last point is particularly significant. The actors made use of pointing gestures. it looks to me like this is vocally. and what’s more I think that they achieved it not only group of people together in the same room.. quite truthfully I felt . Director B was asked. as was frequently know what I mean? because you’re thinking about such a different set observed during the virtual rehearsal process.. the actors had achieved the sub-text of the session (that the woman tor B said in an interview: was waiting for a telephone call about the results of a biopsy): “We’re talking about remote rehearsal here..” From a computer graphics point of view. Different alternatives can also be tested. as would be expected.. and now that that horrible beginning is kind of were beginning to use their head movements in character rather than finished .one of them remarking that in even a modest they were clearly all very much at ease with the system. expressions.. modeling the set is the Session 2 simplest part of the process.so that was the rehearsal space.. looking down a lot .. For example. As one actor said: “. and re the remarks that Bill was making that this being like a on the right lines. you could see that it then becomes quite absorbing. between them. However... An example of the actors choosing an appropriate dis- Post rehearsal discussion tance between themselves. to reflect the current emotional Not all actors agreed that this type of rehearsal could ever demands of the script is shown in Figure 5. whether he thought that stances are such that meeting for real is not possible. Moreover it may of things. do you actors..in Session 4 an emotional sense they were varying their distance.a little more - “It’s kind of becoming .. there are generally agreed. with the way that I am .. “I felt clumsy. “I was using it more today.you just do have an advantage here over a normal live rehearsal room.. the distance The actor who had the greatest difficulty with the system com. As the direc. was that after the arrangements of the actors during the play. One in particu. they also generated envelope feedback.” kind of enter into it.. I was going over to the window moodily The actors clearly did construct an internal mental model of and things like that. As mentioned above one of the actors commented. and you know. yes. but debriefing room itself. and the focus was on the simi. and you can’t move. where it. you could see that they done it four times now..then I’m not rehearsal the actors share the same space. for the actors to collectively work out together with their director what they will be 4. as lar strongly missed the sense of ‘body heat’that is generated when shown in Figure 6. some of the comments: where actors are seen to be adopting the same positions and orien- Session 1 tations in the virtual and real rehearsals.2 Debriefings phone conversation isn’t a real conversation. I can smile and walk and talk like this [points to self] This is a crucial area where the virtual rehearsal system would but until I can connect the virtual reality aspects to the way that I work differ from. and had production just the blocking itself could take a full week to agree. It is not meant to completely replace the live rehearsal process as a whole .. By the end of the process significant saving . in the same way that a tele- 6 .” themselves in that space in relationship to the script and the other “It’s so completely alien to the way that you usually rehearse. least be used for establishing blocking.having its own kind of reality . part of the normal experience . and are able to position actually going to be rehearsing. Each one was asked during the debriefing ses- quite good yeh. with each set... Spatial Organization aged to talk about their experience.... I think it would be useful for establishing blocking. and you simply looking about or trying to find each other. that the rehearsal system could at rehearsal system could be used for some level of acting rehearsal. you can com- municate on a telephone very effectively but using a different skill A free-format discussion was held. Each one was able to do this exercise of storing up what I will be doing when I’m in real life. positive comments about the virtual rehearsal process.” sion to imagine that the rehearsal space was embedded in the “As it is now. one made an important point. I was still trying to move around “. then you go on the set and you go ‘It’s nothing like you’re on stage.” with complete success. I think that they were beginning to achieve it physically as well. and if you can’t get a “Yes. The actors were encour.3 Observations doing in real life. you’re having problems . It is. However. that again strongly fits the intended goals of the scenes (shown on the video recordings) where the man is seated at virtual rehearsal system: the table. In the virtual ..and I find myself tending to go through this the rehearsal space would be located.. working out of the spatial The general impression. and then point to where the various items in not very much more . I couldn’t really enjoy that space.but to lead into it where circum. At this stage I think that still mented: they don’t quite trust it.” radio play.or not with the way that I work. and the extent to which the virtual this was a general agreement. and moving away and things like that . you’re on a set. after all. that’s a real experience. in the round table discus... You don’t ever think about walking into a room . What is impossible to show with static pictures is that sion that followed the live rehearsal. where possible. it’s the recurring nightmare - little anchor points.4. after Session 2.” chalk-marks on the floor.. on-line audio or video conferencing. and glancing towards and away from the woman as their “Virtual rehearsing isn’t actual rehearsing. we’ve only sophisticated towards the end of the exercise. it’s like any actor’s worst nightmare. They clearly made use of emotional facial people are near to one another. you’re stuck how like I imagined it’ [breaks into laughter]. for example.” there. say. This in itself could be a initial session the actors were doubtful.

tem seem to fall between these two extremes. CONCLUSIONS rehearsal? The answer is almost certainly yes: the directors agreed that what was done in the ten minutes of live rehearsal before the This paper has described a system for virtual rehearsal and an audience entered could not possibly have produced the perfor- in-depth study of its use by professional actors and directors. most applications of VEs the nature and style of interaction but what can be seen are transitions from virtual to real in the use between the participants is to help them achieve some other effect . Moreover. the actors learned to use it. Emotional and “Was it a fruitful transition [the transfer from VR to ‘on-the- envelope expression. and the use of the space is the floor’]?”. professionals. one replied: “Yes. in spite of the relative complexity of the interface and the paucity of what was possible. learning a new skill. For example. and really feel this in an emotionally significant way. However. when asked action itself which is the purpose of the application. to do this.. tuating their conversation in a very realistic manner.. Of course what was feasible is impoverished compared to what is possible in real life.. gestures. precisely because the virtual reality was so far from reality. of the space (Figure 7). The possibility of using the head control for this effect was never told to the actors. In virtual rehearsal. This was noted in [13] where some expected behavior did not occur because: ‘. it is the inter. Just as in Figure 5. or understanding live performance actors generally agreed that the virtual rehearsal some complex visual structure. together with the question of whether the in that 10 minutes we spent [‘on-the-floor’]. in a remarkably short amount of time. It was detail that I’d 7 . had made their performance possible. As one actress repeatedly commented. This took the investigators by surprise. and used everything in the armory given to them in order admits opening his letter by mistake.’The avatars used in the virtual rehearsal sys- Figure 5 Varying the emotional distance between themselves. On the other hand as avatars improve there is a point at which they raise expectations to such an extent that the illusion breaks. mance. We hypothesize that there is a type of avatar that is so poor that no useful interaction can take place between people. absolutely . There is another scene where they are virtual performance is helpful in a transfer to a real-life perfor- standing together and each is making head nods and shakes punc. In mance that was observed. clearly wanted to Figure 4 The man is telling the woman not to worry after she impress. Finally. most of them quickly realized that they could use the mouse in order to make rapid small head moves. In a question and answer session after their such as having a discussion.conversation progresses.. The addition of the ability to make some facial gestures and to create envelope feedback compared to previous CVEs was exploited to the full by the participants. it raises user expectations past what can be currently met. when an agent is so successfully human-like in its communication style . Although the purpose was rehearsal for a later real-life performance. was there transfer from virtual rehearsal to the real-life 5. and mostly to use it well. all the detail went in just whole point of the VE. The actors. once again this is subjective. Figure 6 The use of pointing. expectations were rela- tively low: no actor ever commented on the fact that there was no lip synch for speech. Of course. it was the impression of observers (admittedly an entirely subjective point of view) that the director’s viewpoint of the virtual rehearsal showed significant act- ing performance. it was impossible in a virtual rehearsal to reach out and touch. where the woman actually sees nothing useful on her display (the viewpoint is too close to the texture map) the whole point of this was to create an effect in the other actor (and ultimately for the audience). However...

J. succinct control-booth Project. (1999) The Art and Science of Synthetic Character Design. K. Computer Graphics Proceedings (SIGGRAPH). Bowers. [7] Pausch. (1987) A muscle model for animating three- for his post-production work on the documentary video.. (2000) Small Group Behavior in a Virtual and Real Environment: Figure 7 Virtual and Real at the same moment in the play. was com. Bullock. November/ tion. 13 (4-5). that’s where all the stuff came from. rehearsal experiments. Usoh. emotionally expressive and responsive. so the authors can only rely on the directors and actors to CHI’96 Electronic Proceedings. Yale University. Yale [1] Brooks. J.acm. (1999) Fully embodied dilemma. Technical Report CMU- CS-97-156. Autonomous Agents and Multi-Agent Sys- tems. (1999) Creating a Live Broadcast from a Virtual Environment. [6] Perlin. M.. E. Research/Penn State. seven [12] Steed. [15] Frécon. IEEE CG&A. S. (1999) The power of a nod exception was a young woman who. (1999) Acting the Part. [5] Kline. 58-63. We would also like to thank Mitch Mitchel [16] Waters. 18(6).” [10] Bowers. her aid in the VR kitchen and did his best to help her through her [14] Cassell. rehearsal-like. ceedings (SIGGRAPH). A. J.. References [18] David Chambers. J.” His partner concurred: A Comparative Study. A K Peters Ltd. R. Professor of Acting and Directing. (1996) Disney’s Aladdin: First Steps Toward Storytell- ing in Virtual Reality. 45-64. Applied Artificial Intelli- monitor set-up. (1998) Small Group Behavior in the COVEN they rehearsed on-screen under the gentle. M. and Stenius. of course.” autonomous. 37-51. M. to gence. ISBN: 1568810148. Scotland. vocally and gesturally. A.. Bullock. and Hayes-Roth. J. (1996) Talk and The whole notion of ‘performance’is. A. Sadagic. 53-63. Frecon. Haseltine.P (1999) What’s Real about Virtual Reality? School of Drama. E. personal communica- IEEE Computer Graphics and Applications. J. Sadagic.R. Walker. 205-216. C. of the actors clearly reached some level of emotional engagement with (1999) Leadership and collaboration in virtual environ- their virtual partner at some point. F. 20(1) January.. and Environments 9(1). emotional exchange was occur. A. (1998) DIVE: A scaleable net- Acknowledgments work architecture for distributed virtual environments. and a glance: Envelope vs.. conversational avatars: making communicative behaviors ring. This dimensional facial expressions. A. A. A. tification. When present. July 1999. J. Of the four couples that I witnessed. K and Goldberg. one or two were almost consistently ments. B. http://www2. therefore. Proceedings of the AISB 1999 Symposium on AI and Creativity in Entertainment and Visual Art.psu. B. M. Pittsburgh. Steed. O’Brien. Computer Graphics Pro- project is funded by the Digital VCE. IEEE Virtual Reality. [8] Greenhalgh. Embodiment in Collaborative Virtual Environments.. An off-script. http://www. A. 375-384. Schroeder. and Vilhjalmsson.. Slater.. (1996) Improv: A System for Scripting Interactive Actors in Virtual Worlds. guidance of [the director]. June 1997. Sadagic. emotional feedback in ani- puter illiterate and completely befuddled by the keyboard/touchpad/ mated conversational agents. “It was remarkable watching the actual actors on separate monitors as Slater. We would like to thank Ivor Benjamin for his help during the Distributed Systems Engineering Journal. J. N. 5(3). and Parke F. on-screen via his avatar.. Pycock. [2] Reiland. Taylor.and Blumberg.. H..org/sig- judge this. [17] Waters K. the emotional content was clearly perceivable. Her acting partner rushed. PA. C. Presence: Teleoperators and Virtual “Yes I agree that we rehearsed in that virtual reality space. Tromp. by her own admission.. rehearsals later wrote [18]: [11] Tromp. R.. 2. Edinburgh. Watson. Computer Graphics Proceedings (SIGGRAPH). March 1999. An independent visitor during the first day only of chi/chi96/proceedings/papers/Bowers/jb_txt. J.html.. Computer Graphics Proceedings (SIGGRAPH).. School of Computer Science. M. The lone [13] Cassell. Carnegie Mellon University. December. R. Snoddy. E. you know. (1997) An Oz-Centric Review of Interactive Drama and Believable Agents. K. D.. 8 . [4] Rousseau. 91-100. Wyver. 193-203. (1996) Computer Facial Anima- tion.edu/rps/jan99/act- ing. M. Houston. [9] Slater. 519-539. J. beyond quan.planned [during the virtual rehearsals].deasy. [3] Mateas.. 21(4):17-24. and Thorisson..htm. (1997) Interacting with Personality-Rich Characters Stanford Knowledge Systems Laboratory Report KSL-97-06. 16-27.. G.

Sign up to vote on this title
UsefulNot useful