Acting in Virtual Reality

M. Slater, J. Howell, A.Steed, D-P. Pertaub, M. Gaurau
Department of Computer Science
University College London
Sharon Springel
Centre for Communications Systems Research
University of Cambridge

ABSTRACT manipulator for visualization. The one application in Professor
Brooks’ review that seemed to step outside of the design/engineer/
Three pairs of professional actors and a director each met in a
skill-rehearsal applications suite was psychotherapy for fear-of-
shared non-immersive virtual reality system over a two-week
flying and post-traumatic stress disorder. However, even in this
period to rehearse a short play. The actors and director never met
case, users engage with an external simulated environment in a
one another physically until a short time before a live rehearsal in
way that is consistent with the approach of behavioral exposure
front of an audience. The actors were represented by avatars which
therapy. Here a patient interacts with a psychotherapist using the
could be controlled to make a range of facial expressions, and
simulation and the experiences it evokes as a common point of
some body movements, including navigation through the space.
contact.
The study examined the extent to which virtual reality could be
A characteristic of each of these types of application is that
used by the actors and director to rehearse their later live perfor-
groups of people interact with one another through their shared
mance. Four indicators captured by questionnaires show that over
interest in some simulated objectified process: flying the aircraft,
the period of the four days their sense of presence in the virtual
viewing the design prototype, or examining the visualization. The
rehearsal space, their co-presence with the other actor, and their
virtual reality in each case provides a scenario, and the fundamen-
degree of cooperation all increased. Moreover their evaluation of
tal interaction is between each individual user and the environ-
the extent to which the virtual rehearsal was similar to a real
ment; interaction between people occurs with respect to their
rehearsal also increased. Debriefing sessions with the actors and
common interest in the scenario.
director are reported, which suggest that a performance level was
In this paper we explore a different virtual reality paradigm in
reached in the virtual rehearsal which formed the basis of a suc-
which the objective is emotionally significant interaction between
cessful live performance, one that could not have been achieved by
people, rather than between people and some other situation or
learning of lines or video conferencing.
thing. The virtual environment provides only the setting in which
Keywords person-to-person engagement takes place, it is the interaction
Virtual reality, multi-user, networked applications, facial anima- between the people, and the ‘emotional atmosphere’ which this
tion, virtual rehearsal, acting generates which is the major objective of the experience.
The scenario in which this is explored is that of dramatic act-
ing, in particular virtual rehearsal for live drama. Suppose actors
1. INTRODUCTION and their director never meet one another, except in a virtual real-
Practical applications of virtual reality are normally in the ity setting, until a short time before a live stage rehearsal. We con-
realms of engineering, product design, and skill rehearsal. Users sider the question: to what extent can they create sufficient acting
enter into a virtual environment in order to learn something new performance so that a live performance can take place with hardly
about the real situation to which the simulation corresponds, or to any live face-to-face rehearsal?
improve or learn a skill set. Professor Frederick Brooks Jnr The practical motivation for this was first suggested by the
recently carried out a survey of such applications [1] which head of research at the BBC: If it is possible for rehearsal to take
included vehicle (aircraft and boat) simulators, design and proto- place in a distributed virtual environment, then significant savings
typing systems, a NASA astronaut training system, and a nano- could be made in pre-performance production and planning.
Actors could stay situated wherever they may be in the world, con-
tinuing with their current commitments, while nevertheless devot-
ing some time to rehearsal of a new production via a shared VE.
The actors, director and others involved could get to know one
another. The technicians could work out camera angles and light-
ing in parallel with the performers working out their blocking (i.e.,
the spatial arrangements of the performance in synch with the
dynamics of the narrative).
Acting rehearsal is more than just learning of lines, which is
the least important part of the rehearsal process. Actors must act in

1

pants. she found she was able to convey represented by simple block-like humanoid avatars. Arm movements were only This describes a strategy used by actress Tania Vujoshevich in her possible for the one immersed participant in each group. to the display of plicity.” Vujoshevich changed from the virtual to real experience. developed by the New York based Gertrude Stein meaning that there is interest in having a conversation. those neously edited and broadcast for live TV. or browser. but the envelope communicative behavior is automated of a play has not. Envelope behaviors are not consciously initiated by the A system which synthesizes a collaborative VE with live TV speaker. For example. The avatars formance. Vaslav Nijinsky. The major rehearsal of a role in a Berchtold Brecht play [2]. The point was to see how their social interaction my fiancé’s. such as glancing towards the other avatar video-conferencing system was used in order to allow choreogra. findings. Such situations have been rarely studied. and unconsciously generated. Acting involves accessing and expressing the together in real life. BACKGROUND cacy of emotional and envelope feedback for automated avatars in conversation with humans. The exploit a collaborative virtual environment (CVE) as a medium in BodyChat system provides semi-autonomous avatars for partici- itself for the creation of a new form of entertainment. reproducing an earlier result [10]. projects at regional theaters throughout the US and Europe. where the par. In [14] the principle of automated interacted with professional actors in a game. as set by the literature. but also adds to the effi- The situation we are examining. unlike the situa. play from the met first in a virtual environment to try to solve a set of riddles emotional expression delivered to one another. for explore a virtual world based on the animated movie [7]. The closest to what is reported in this paper was the the user for their corresponding avatar. precisely-timed eyebrow changes. and type messages to The use of a VE for rehearsal of a later ‘real life’ performance others. been described elsewhere in based on the current intentions of the user towards others. to our knowledge. virtual rehearsal for a live per. and interact back refers to non-verbal behaviors such as gaze timing and with human participants. and pretend I would never see him again. and vocal chords. with no possi- the pain and sadness and fear the character was feeling. Users may control their navigation. for example [3][4][5][6]. This was an attempt to representing people in a collaborative virtual environment. body posture. However. is from a technical point of view a CVE.order to rehearse. which was simulta.htm [9][11][12]. that designers involved. also automatically carry out low level non-verbal behaviors such ticipants meet to carry out a shared task that they will later perform as blinking and breathing . envelope behaviors was applied to embodied avatars. In [9] full range of human emotions in a way that produces the appropri.” bility of exhibiting facial expression. this is behaviors not only frees the user from having to control every eye- at a cost of fragmentation of the total space: the performers never brow and mouth movement. the exploitation of the medium for story-telling. After meeting virtually for true during rehearsal: a while they then met in the corresponding real room and contin- “In the beginning. Emotional feedback employs specific Research into the exploitation of virtual environments for emblems such as a smile or scrunched eyebrows to express under- drama has concentrated mostly on the creation of synthetic charac. The avatar Repertory Theatre2. More example. In two follow-up studies their emotional state through the normal everyday employment of [11][12] similar results were found. The major advantage of such an approach is allowing the avatar itself to take care of such automatic non-verbal that live video of the real performer is distributed. of course. head movements.com The results of the series of three-person experiments in 2. The given the opportunity. These punctuate ney’s Aladdin immersed and allowed participants to interactively the conversation as a secondary feedback channel. 1. In this case members of the public is detrimental to interaction. to regulate turn-taking and hesitations in speech. Actors also. i. http://www. helping.. Another goal is distribution. The participants were says. but their absence broadcast is described in [8]. ciency and smoothness of the resulting conversations. and making appropriate salutations. To what that the avatars’lack of expressiveness (they were only able to turn extent can the impoverished substitute for emotional expression and move through the environment) impeded the development of available through today’s VE technology support acting perfor. during rehearsals. director and other It is argued in [14] with supporting experimental evidence. only a constant neutral expression. In [13] an experiment is reported that compares the effi- 2. mance? It is important to distinguish different types of avatar expres- siveness. both con. but that avatars ditional drama. lying emotions such as happiness or puzzlement. that relate to this paper.com/sfasp/vr. important for regulating the conversational flow. sonal and social significance. phy changes to be agreed by the performers. and thereby avoiding the ‘dead’ actually share the same space together. an avatar may use of video-conferencing to rehearse an adaptation of the Diary of be put into a state of ‘availability’ with respect to another user. tion in live theater or film. and expressions with the mouth. The recently the VRMLDream company created a performance with experiment showed that envelope behavior was not only more actors directly controlling virtual characters1. it was always found their facial expression. hand gestures. notwithstanding their extreme sim- sciously chosen. However. This simulated a tra. an experiment was described where 10 groups of 3 people each ate affect in the audience.e. I would replace Fritz’s name with ued the same task. Dis. The same must be plastered on the walls of a virtual room. except that it occurred entirely in a distributed VE. and thus choose their own viewpoints. not consciously noticed by the onlooker.vrmldream. were that the avatars took on per- In physical reality actors have complete access. together with the results of [13][14] informed the 2 .ibm. Envelope feed- ters with believable personalities which can perform.adding to their very high degree of ‘aliveness’and presence. For example. http://www. By personalizing the situation. exhibiting such behavior were rated as more helpful and lifelike Viewers could navigate the scenario with a standard VRML compared with avatars that exhibited only emotional feedback. The artists involved were employed on other then automatically generates the appropriate non-verbal behaviors. moments while they are typing messages. significant social interaction between participants.

live performances. The scene was Directors Guild of Great Britain. They were paid enough to cover their expenses for each meeting. was a co-chair of the Infinite Reality Graphics and 64M main memory. were recruited by advertisement around the depart- hand they know in advance what they’re going to say. were generated automatically. S directed two of the groups. on a fifth time to carry out the rehearsal for real before a live audi. leading to conflict with the actors. This was followed by a debriefing their partners and in an audience. and mutual The virtual rehearsals. The frame-rate was at least 20 then they met immediately afterwards physically for the first time frames per second on all machines. The kitchen and avatars consisted of about repaired. So their ment. Both actors and the director used desktop dis- ground in film and TV screenwriting and direction. sometimes. B was more 64MB main memory. discussion. shown on a 21 inch monitor covering the full screen. One (S) had been directly involved The virtual rehearsal system was implemented using the DIVE in the design of the study for several months. but one of them had to withdraw due to an unex- pected acting engagement). The audience. and An overview is shown in Figure 1. S had a strong back. in the real rehearsal space. whole focus is on how they say it. http://www. actors again met their director for about 30 minutes in the VE. Three pairs of actors took part throughout. and answering a questionnaire. The director was a disem- until the fifth and final occasion. frequently commenting on inch display monitor. suggesting individual moves. but on the other of about 20. actual rehearsal . This involved a man and a woman in a kitchen in the morning having an everyday conversa- tion about mundane matters while preparing to go out. On the fifth time the 3500 polygons and was rendered by DIVE using OpenGL calls. disk and 4 IR2 Graphics pipes each with 2 Raster Managers. Actors pose a tough challenge: they must ately by the live rehearsal watched by an audience. On each of the first four occa. so that complete vir- through the virtual rehearsal process the actors construct an inter. During the virtual rehearsals the actors’ A second way in which this application differs from traditional screen views were video-taped with an inset of their real selves in a conferencing or chat rooms is that the virtual interaction is a smaller screen window. There were two directors. The actors were embodied as avatars who could see and talk to ence. It was thought that actors must carried out by two of the investigators. 3. bodied voice with no visual representation. rehearsal for real life. and all debriefing gaze-locking. However.3x platform [15]1. low-level behaviors. About 30 minutes of this time was spent in equivalent to where the audience would sit. (On the first day there was an addi- tional couple. exhibit both envelope and emotional behavior. nal codification of the play that later transforms to their behavior on stage. sessions were video-taped. with a bonus after completion of the Figure 1 An Overview of the Rehearsal Space entire rehearsal process. 180Gb tional. for about one hour. The script was for a play of about five minutes. 3. and the last virtual some body gestures . It was decided that the observed rehearsal was carried out. They then had about 10 minutes to carry out some live rehearsal. and rehearsal was on the Monday of the third week followed immedi- make arm gestures. before the audience was let in and an 1. but allow the actors to choose these as appro. and in the tars.allowing greater freedom for the actors actor used an SGI High Impact system with 200Mhz R4400 and to establish their blocking and individual expression. THE VIRTUAL REHEARSAL STUDY 3. tak. such as blinking. The The actors and their director met four times virtually and then monitor was 23 inches and the full screen was used for the display.design of our acting rehearsal system.especially to be able to position the head. and two per week for two successive weeks. not involved in the planning of the study. tual rehearsals were recorded from this third position. The director’s view was also video-taped. The director used an Onyx2 system compris- how well the performance was working. and for them to be able to find At the end of each of the first four rehearsal sessions the actors just that way of saying it that can evoke the appropriate response in completed a short questionnaire. again with the scene shown on the full 19 directive. The scene was a virtual kitchen with an outside ante-room rep- tice with the system. for the first time.1 The Scenario A script was developed from a training program previously used at the UK’s National Film and Television School. as is tradi. So another question is the extent to which The director also acted as a camera controller. At the end of the live therefore have significant control over the behavior of their ava. awaiting for system breakdowns to be resenting off-stage. (B). One actor used an SGI Onyx with twin 196 MHz R10000. The actors never met each other or the director physically one another in the shared virtual space. The four rehearsals took place actors had to have the capability for making facial expressions. The second ing a largely passive role .se/dive 3 .sics. The second plays. ing a Graphics Rack with 8 R12K processors. 8Gb RAM. The actors were often sions the actors and their director met in the virtual environment reminded that the director was taking the ‘downstage’ position.the rest of the time was setup. rehearsal actors were questioned by the audience. priate. Accordingly their avatars should not automatically generate evening had a round table discussion where all pairs met together envelope behaviors. prac.

3 The Controller for Facial Expressions by the second rehearsal session) all of the actors but one were using facial expressions with ease. A new head. the expression that they were trying to induce.net/vrml/h-anim/h-anim-examples. The face itself was obtained from a program supplied in [17]. anger/surprise) were orthogonal. sadness and a neutral the gradients on each side together with the area of the triangle expression. either one of the previ- ous ones that they had rehearsed or a new one. So instead of a while leaving the body stationary. They could also use the arrow menu system. The sizes of be displayed: happiness. The body of the female avatar was an adapted version of an H- Anim compliant avatar called Nancy. so that after a party they had slept together for the first time . smile. surprise. 3. A substantially the same throughout. created by 3Name3D1.giving half.but neither combinations of the two could be produced. and also By using the mouse in the main window. watching the other actor.html 4 . These two the same house together for some time as friends.controlling the degree of anger or surprise. but was edited to improve the face and hair in particular. This sub-text was the one to be rehearsed before the live audience. A similar method was used for the eyebrows allow for asymmetric facial expressions. each time the actors met they stroke on the bottom half would affect the mouth . After a relatively short degree of practice (specifically 3. izontal line would result in a neutral expression in each case. head was facing. For the mouth a the accompanying animation algorithms were rewritten for DIVE.ballreich. The intensities were truncated to a maximum before being passed to the face plugin. thus changing their viewpoint. A modified subset of the geometry was used since the original avatar had no facial animation. Although the script was emotionally neutral. This is shown in the inset below.2 The Avatar and Facial Model The male avatar originated with the DIVE system. Figure 2 Some Examples of Facial Expressions tion was based around a muscle model originally proposed in [16]. An approximate hor- shown in Figure 2. Moreover the intensity knew if the other really meant it. Or the woman was awaiting a of each could be controlled. them the feeling that it was somewhat under their control. the actors controlled their facial expression by keys to move the body forwards or backwards in the direction the means of mouse strokes on an abstract ‘smiley’ type face. the actors could move making the actions that they carry out somewhat consistent with their avatar’s head in any direction. rehearsal they chose a sub-text themselves.4 Other Controls actors would need to continually scan the set of displayed faces in order to choose the one that they wanted . The first is that 3. In addition. thus allowing a wide variety of facial phone call that would give the results of a biopsy. they had been sharing eyebrows . The main screen- speaking their lines. and other controls in the lower actively engaged in determining their facial expression . triangle was determined from the stroke action. and remained The abstract face was divided into a top and bottom half. However.this at the same time as The screen was divided into two windows. Both the male and female avatars were fitted with a new face to enable facial animation. were independent. hair and eye geometry was devised and fitted to the female avatar and various of the body parts were scaled and adjusted. The code was designed to enable the actor to specify were used to determine the intensity of the expression. anger. and A very simple recognition algorithm was used. but last night sets (happiness/sadness. Some examples are with used for surprise and for anger. On the fourth expressions. The second is that we wanted the actors to be had the face controller at the top. At first this movement was completely free. Thus in principle the head and body direction itively. the body was texture-mapped for clothing. It would have been simple to provide a visual menu of facial expressions from which the actors could select by point-and-click. Intu. and also the an intensity for any one of the four major expressions and also to degree of symmetry. Or. A stroke on the top half would affect the them had crashed their brand new car. stand up and move its arms either singly or together. but after the first session the actors complained that this freedom was too confusing (their heads could be twisted back-to-front reminis- cent of The Exorcist) so the system was adjusted such that when 1. and moving through sized window showed the scenario.controlling the degree of For example: it was early morning and the night before one of happiness or sadness. a grimace or a neutral expression . http://www.determining a were told by the director to act according to a different sub-text. An overlaid smaller window the environment. this was not employed for two reasons. The face was texture-mapped to help with the required masculine or feminine appearance. Some very basic animation capabilities were provided that allowed the avatar to sit. Facial anima. A stroke such as The facial animation interface allowed five major expressions to would be interpreted as ‘sad’and as ‘happy’. to evoke a smile they would draw one.

cooperating 3 ease with the whole process. ited by two questions: tar’s face at the top left corner of • To what extent did you have the sense of being in the rehearsal the main window (as can be seen space? (For example.the head turned around more than 60 degrees from the front. ego- centric point of view. if at all. It was elic- small rendition of their own ava. they requested a ‘face Presence mirror’ that could be switched on This is the extent to which the individual is able to suspend and off. each elicited using sliders. 5 . there was an The authors of this paper were sceptical about the possibility improvement in all indicators over the four sessions of the experi- of this virtual rehearsal system offering the possibility of perfor. After each of the first four sessions a brief questionnaire was ered independently of one another issued to the actors. Co-presence point. or off (it was left ‘on’ by default). Since it was you feel that the other actor was in the space with you? inconvenient. if you were asked this question about the in Figure 2). and for all as a group. All six of the actors saw 6 the process through to the end. The results on all four indicators are clear (significance tests are not appropriate for such small samples). do you remember in their sense of presence to shift this as more like just interacting with a computer or with other people? between ego.1 Questionnaire Responses body was swivelled around automatically. was at first extremely ill-at. but voluntarily suggested attending for ‘out of hours’ additional practice with the interface. By the end of the 1 process she was capable of using the system expressively. by their 7 disembodiment. ment as shown by the scatter-plots of mean responses in Figure 3. The expression inten. did you have a sense of cooperating at them. since it is impossible to objectively quantify the degree of acting perfor. and experimented with presence its possibilities. The questionnaires were used as approximate indicators of what was taking place. and would quickly walk away. mance rehearsal. Hence a gaze lock was installed. However if you were asked this question about whether you were sitting in sity control could be switched on a room at home now. This concentrated on four issues. room you are in now. you would give a score of 7. Session debriefing comments including those of the director (B). by one or more questions on a 7-point scale with 1 indicating a low It was possible for the actors to degree and 7 a high degree of the corresponding attribute. move between a first person. rehearsal space became the reality for you. • To what extent were there times during the rehearsal when the and similarly for the ability to cre. Although they could see the This is the extent to which the computer becomes transparent facial expression of their partner and there is a sense of being with the other people in the VE. For 4. All the actors always used the first person ego-centric view. 1 2 3 4 The results are reported through the questionnaire responses. and caused a break • When you think back about your experience. The arms could be raised or low. It was they were very concerned to elicited by two questions: ensure that their own expression • In the rehearsal you have just experienced to what extent did was as they intended. The ence was really taking place? actors complained in the earlier Cooperation sessions that they were not sure when the other actor was ‘looking’ • To what extent. and so 2 attended 30 minutes earlier on the next occasion. 4 copresence gating through the environment at all. so that eyes momentarily with your acting partner? locked when the avatars faced each other. To what extent where they could see their own did you feel that the rehearsal you have just experienced was similar to those rehearsals? avatars. and you almost for- ate asymmetric expressions (the got about the real world of the lab in which the whole experi- ‘lopsided’ switch in the inset). It was believed that the actors would feel alien- ated by their inability to express themselves. or a second Similarity to real rehearsals person disembodied point of view • Think about your recent rehearsals in real life. the 4. RESULTS each actor individually. This was displayed as a disbelief and have a sense of being in the virtual space. Five of the six had no difficulties Mean Response 5 similar with using the system after the first session. Figure 3 Questionnaire Results mance achieved. you would give a score of 1).and exo-centric viewpoints. The positive results achieved were therefore surprising. and most important of all direct observations of the process. One of the six found great difficulty even in navi.

” there...4. and glancing towards and away from the woman as their “Virtual rehearsing isn’t actual rehearsing. The last point is particularly significant. “I was using it more today. get these . Here are The fact that this occurred is evident on the video recordings. and the extent to which the virtual this was a general agreement..in Session 4 an emotional sense they were varying their distance. Moreover it may of things. to reflect the current emotional Not all actors agreed that this type of rehearsal could ever demands of the script is shown in Figure 5. there are generally agreed. For example. The actors made use of pointing gestures. It is not meant to completely replace the live rehearsal process as a whole . that again strongly fits the intended goals of the scenes (shown on the video recordings) where the man is seated at virtual rehearsal system: the table.3 Observations doing in real life.. and moving away and things like that . replace real rehearsal (though that is not the point). I can smile and walk and talk like this [points to self] This is a crucial area where the virtual rehearsal system would but until I can connect the virtual reality aspects to the way that I work differ from.but to lead into it where circum. looking down a lot . with each set.” From a computer graphics point of view.. you’re having problems . However...you just do have an advantage here over a normal live rehearsal room. and you know. was that after the arrangements of the actors during the play.. and what’s more I think that they achieved it not only group of people together in the same room... it looks to me like this is vocally.one of them remarking that in even a modest they were clearly all very much at ease with the system. do you actors. working out of the spatial The general impression.” An example of a such a head-move ‘in character’ is shown in Figure 4. As one actor said: “.. in the round table discus. after Session 2. that’s a real experience.. One in particu. quite truthfully I felt .a little more - “It’s kind of becoming . and now that that horrible beginning is kind of were beginning to use their head movements in character rather than finished ..” chalk-marks on the floor. An example of the actors choosing an appropriate dis- Post rehearsal discussion tance between themselves. and you can’t move. In the virtual . Director B was asked.” with complete success..so that was the rehearsal space. I think that they were beginning to achieve it physically as well.. it’s the recurring nightmare - little anchor points. Each one was able to do this exercise of storing up what I will be doing when I’m in real life.. “I felt clumsy. as was frequently know what I mean? because you’re thinking about such a different set observed during the virtual rehearsal process.. where possible. This in itself could be a initial session the actors were doubtful. you can com- municate on a telephone very effectively but using a different skill A free-format discussion was held. I couldn’t really enjoy that space. with the way that I am .having its own kind of reality .and I find myself tending to go through this the rehearsal space would be located. At this stage I think that still mented: they don’t quite trust it.” themselves in that space in relationship to the script and the other “It’s so completely alien to the way that you usually rehearse. that the rehearsal system could at rehearsal system could be used for some level of acting rehearsal. but debriefing room itself. as would be expected. it’s like any actor’s worst nightmare. we’ve only sophisticated towards the end of the exercise.. I was going over to the window moodily The actors clearly did construct an internal mental model of and things like that. say.is to use your imagination. It is. modeling the set is the Session 2 simplest part of the process. Their actions were more . for example. Spatial Organization aged to talk about their experience. you could see that it then becomes quite absorbing. in the same way that a tele- 6 . You don’t ever think about walking into a room . after all. the actors had achieved the sub-text of the session (that the woman tor B said in an interview: was waiting for a telephone call about the results of a biopsy): “We’re talking about remote rehearsal here.2 Debriefings phone conversation isn’t a real conversation. on-line audio or video conferencing. and had production just the blocking itself could take a full week to agree. By the end of the process significant saving . they seemed to be physically downcast.. However.” of the actors separately after the sessions. least be used for establishing blocking.then I’m not rehearsal the actors share the same space. they also generated envelope feedback.or not with the way that I work. They clearly made use of emotional facial people are near to one another. part of the normal experience . As mentioned above one of the actors commented.. I was still trying to move around “.” radio play. you’re stuck how like I imagined it’ [breaks into laughter]. one made an important point. and again larity to a real rehearsal.” sion to imagine that the rehearsal space was embedded in the “As it is now. and if you can’t get a “Yes. and the focus was on the simi. As the direc.. Different alternatives can also be tested. as lar strongly missed the sense of ‘body heat’that is generated when shown in Figure 6. positive comments about the virtual rehearsal process.. and are able to position actually going to be rehearsing. between them. whether he thought that stances are such that meeting for real is not possible. you could see that they done it four times now. you’re on a set. where it.. and then point to where the various items in not very much more . and re the remarks that Bill was making that this being like a on the right lines. expressions.” kind of enter into it. you never have your body twisted one way and your the space and objects within it are usually marked out by tape and head another way. What is impossible to show with static pictures is that sion that followed the live rehearsal. The actors were encour.. and you simply looking about or trying to find each other.. Each one was asked during the debriefing ses- quite good yeh.. then you go on the set and you go ‘It’s nothing like you’re on stage. I think it would be useful for establishing blocking. the distance The actor who had the greatest difficulty with the system com. It expresses one of the major goals of the virtual rehearsal. some of the comments: where actors are seen to be adopting the same positions and orien- Session 1 tations in the virtual and real rehearsals. yes. for the actors to collectively work out together with their director what they will be 4.

Finally. and really feel this in an emotionally significant way. it is the inter. It was detail that I’d 7 . Just as in Figure 5. In a question and answer session after their such as having a discussion. when asked action itself which is the purpose of the application. professionals. The possibility of using the head control for this effect was never told to the actors. to do this. This was noted in [13] where some expected behavior did not occur because: ‘. As one actress repeatedly commented. in a remarkably short amount of time. most of them quickly realized that they could use the mouse in order to make rapid small head moves. once again this is subjective.. absolutely . This took the investigators by surprise. one replied: “Yes. We hypothesize that there is a type of avatar that is so poor that no useful interaction can take place between people. had made their performance possible. in spite of the relative complexity of the interface and the paucity of what was possible.. and the use of the space is the floor’]?”. In virtual rehearsal. mance. CONCLUSIONS rehearsal? The answer is almost certainly yes: the directors agreed that what was done in the ten minutes of live rehearsal before the This paper has described a system for virtual rehearsal and an audience entered could not possibly have produced the perfor- in-depth study of its use by professional actors and directors. The actors. when an agent is so successfully human-like in its communication style . it raises user expectations past what can be currently met. Although the purpose was rehearsal for a later real-life performance.. of the space (Figure 7).’The avatars used in the virtual rehearsal sys- Figure 5 Varying the emotional distance between themselves. In mance that was observed. tem seem to fall between these two extremes. learning a new skill. gestures. Of course what was feasible is impoverished compared to what is possible in real life. it was impossible in a virtual rehearsal to reach out and touch.. clearly wanted to Figure 4 The man is telling the woman not to worry after she impress. was there transfer from virtual rehearsal to the real-life 5. expectations were rela- tively low: no actor ever commented on the fact that there was no lip synch for speech. or understanding live performance actors generally agreed that the virtual rehearsal some complex visual structure. However. and mostly to use it well. There is another scene where they are virtual performance is helpful in a transfer to a real-life perfor- standing together and each is making head nods and shakes punc. all the detail went in just whole point of the VE. tuating their conversation in a very realistic manner. The addition of the ability to make some facial gestures and to create envelope feedback compared to previous CVEs was exploited to the full by the participants.. For example. On the other hand as avatars improve there is a point at which they raise expectations to such an extent that the illusion breaks. However. it was the impression of observers (admittedly an entirely subjective point of view) that the director’s viewpoint of the virtual rehearsal showed significant act- ing performance. together with the question of whether the in that 10 minutes we spent [‘on-the-floor’].. Figure 6 The use of pointing. Moreover. and used everything in the armory given to them in order admits opening his letter by mistake. Emotional and “Was it a fruitful transition [the transfer from VR to ‘on-the- envelope expression. most applications of VEs the nature and style of interaction but what can be seen are transitions from virtual to real in the use between the participants is to help them achieve some other effect . Of course.conversation progresses. where the woman actually sees nothing useful on her display (the viewpoint is too close to the texture map) the whole point of this was to create an effect in the other actor (and ultimately for the audience). the actors learned to use it. precisely because the virtual reality was so far from reality.

J. succinct control-booth Project. Bowers. you know. (1996) Improv: A System for Scripting Interactive Actors in Virtual Worlds. Computer Graphics Proceedings (SIGGRAPH). When present. J. tification. PA. E. M.. personal communica- IEEE Computer Graphics and Applications. therefore. Computer Graphics Proceedings (SIGGRAPH).planned [during the virtual rehearsals]. R. (1999) The Art and Science of Synthetic Character Design. J. IEEE Virtual Reality. (1996) Disney’s Aladdin: First Steps Toward Storytell- ing in Virtual Reality. emotional exchange was occur.. [9] Slater. Research/Penn State. http://www. Technical Report CMU- CS-97-156. 18(6). seven [12] Steed.. guidance of [the director]. Haseltine. Applied Artificial Intelli- monitor set-up.deasy. (1987) A muscle model for animating three- for his post-production work on the documentary video. This dimensional facial expressions. by her own admission. Steed. IEEE CG&A. Tromp. J.. 5(3). so the authors can only rely on the directors and actors to CHI’96 Electronic Proceedings. (1997) Interacting with Personality-Rich Characters Stanford Knowledge Systems Laboratory Report KSL-97-06. Pittsburgh. http://www2. Bullock. Computer Graphics Proceedings (SIGGRAPH). A.” autonomous. and Parke F. A K Peters Ltd. Snoddy. emotionally expressive and responsive. A. J. Carnegie Mellon University. M. R.. We would like to thank Ivor Benjamin for his help during the Distributed Systems Engineering Journal. Watson. June 1997. (1999) Creating a Live Broadcast from a Virtual Environment. ceedings (SIGGRAPH). K. Slater.psu.. [6] Perlin. [7] Pausch.html.” His partner concurred: A Comparative Study. Scotland. H. (1998) Small Group Behavior in the COVEN they rehearsed on-screen under the gentle. B. B. that’s where all the stuff came from. C. References [18] David Chambers. conversational avatars: making communicative behaviors ring.acm. Autonomous Agents and Multi-Agent Sys- tems.. G. Sadagic. F. Usoh. 375-384. and Environments 9(1). K. J. We would also like to thank Mitch Mitchel [16] Waters. Of the four couples that I witnessed. (1996) Talk and The whole notion of ‘performance’is. Bullock.. S. Presence: Teleoperators and Virtual “Yes I agree that we rehearsed in that virtual reality space.htm. A. rehearsal-like.. November/ tion.. 20(1) January. K and Goldberg. Schroeder. her aid in the VR kitchen and did his best to help her through her [14] Cassell. [17] Waters K. 37-51. 45-64. vocally and gesturally. Houston. emotional feedback in ani- puter illiterate and completely befuddled by the keyboard/touchpad/ mated conversational agents. 53-63.. M. R. 13 (4-5).P (1999) What’s Real about Virtual Reality? School of Drama. A. Yale University. Wyver. M. (1999) Acting the Part. 21(4):17-24. 519-539. of course. and Thorisson. 8 . and a glance: Envelope vs. [8] Greenhalgh. and Hayes-Roth. 58-63. Embodiment in Collaborative Virtual Environments. An independent visitor during the first day only of chi/chi96/proceedings/papers/Bowers/jb_txt. M. the emotional content was clearly perceivable. “It was remarkable watching the actual actors on separate monitors as Slater. to gence. (1998) DIVE: A scaleable net- Acknowledgments work architecture for distributed virtual environments. Computer Graphics Pro- project is funded by the Digital VCE. Proceedings of the AISB 1999 Symposium on AI and Creativity in Entertainment and Visual Art. December. and Vilhjalmsson. 205-216. Her acting partner rushed. and Stenius. Walker. Sadagic. Professor of Acting and Directing.edu/rps/jan99/act- ing.. ISBN: 1568810148. N. (1999) Fully embodied dilemma. Frecon. (1996) Computer Facial Anima- tion. [5] Kline. J. [15] Frécon. Sadagic. The lone [13] Cassell. Edinburgh. C. An off-script. E.. A.. Yale [1] Brooks. rehearsal experiments. O’Brien. Taylor. A.” [10] Bowers. 16-27. 193-203. J. 91-100.R. was com. rehearsals later wrote [18]: [11] Tromp.. [2] Reiland. beyond quan. 2.. March 1999. A.. (1997) An Oz-Centric Review of Interactive Drama and Believable Agents. A.. M. E.org/sig- judge this. School of Computer Science. one or two were almost consistently ments. Pycock. July 1999. [3] Mateas. J.and Blumberg. of the actors clearly reached some level of emotional engagement with (1999) Leadership and collaboration in virtual environ- their virtual partner at some point. J.. on-screen via his avatar. (2000) Small Group Behavior in a Virtual and Real Environment: Figure 7 Virtual and Real at the same moment in the play. [4] Rousseau. D. (1999) The power of a nod exception was a young woman who..

Sign up to vote on this title
UsefulNot useful