CS490z Final Report Auto-Dynamic Difficulty In Video Games Hybrid software and research project

Christine Bailey Student Number: 250057994 Email: cdbailey@gaul.csd.uwo.ca

Supervisor: Dr. Michael Katchabaw

Signature of Supervisor:_________________________

Table of Contents
1
1.1 1.2

IntIntroduction...............................Introduction..........................................................

Justification..........................Justification......................................................................................Justific

Proposal:Proposal: Auto-Dynamic Difficulty...Proposal: Auto-Dynamic Difficulty....Proposal: Aut

2
2.1

BackgrounBackground.....................................................................................Backgr
2.1.1

DynDynamicDynamic User Factors..............................................................................Dynamic User Fact

Skill,Skill, chaSkill, challeSkill, challenge, and optimal experience...........................................

2.1.1.1 TheThe rolThe role of learning in games................................................................................3 2.1.1.2 LearningLearning through death in the classroom...........................................Learning through

2.1.1.3 FacilitatingFacilitating learningFacilitating learning throFacilitating learning through reflecti 2.1.1.4 TutorialsTutorials and avatar monologue.............................................................................7 2.1.2 2.2 EmotionalEmotional needs.....................................................................................................7

StaticStatic User Attributes: The Influence Of PStatic User Attributes: The Influence Of PlaySta

3
3.1 3.2 3.3

MethodsMethods Of Measurement.................Methods Of Measurement...........

FactorsFactors To Be Considered...............................................................................................9

MeasurinMeasuringMeasuring Skill Level.....................................................................Measuring Skill L

MeasuringMeasuring Measuring Player-Type........................Measuring Player-Type............................. 3.3.1 3.3.2

ArousalArousal and emotion.........Arousal and emotion..........................................................Arou

ExploExplorExplorersExplorers versus achievers.......................................................................

3.4

UserUser Testing.........................................................................................User Testing............................... 3.4.1 3.4.2

CasualCasual versus Core.............................................................................Casual versus Core....

PotentialPotential issues in user testing....................................................Potential issues in u

3.5

AdditionalAdditional Ideas................................................................................................................13

4
4.1 4.2 4.3 4.4

DifficultyDifficulty Difficulty Adjustment...................................................Difficulty Ad

DifficultyDifficulty Adjustment Through Deception...................................................................15 ProactiveProactive Versus Retroactive Measurements and Corrections...................................16 DifficultyDifficulty Through Ranking And Scores FDifficulty Through Ranking And Scores For

FactorsFactors To Factors To BFactors To Be Considered....................................................................

5
5.1 5.2 5.3 5.4

ImplementationImplementation Of A Test Environm ent.................................Imple

InitialInitial Tools - The Quake II Engine.............................................................................18 TheThe Issues With Quake II............................................................................................................18 TheThe Solution: Unreal Tournament 2004.................................................................................19

LearningLearning How To Use UnrealEd................................................................................................ 5.4.1 5.4.2 5.4.3 5.4.4 JumpLevel1JumpLevel1 and JumpLevel2............................................................................22

TimedLevel ..........................................................................................................................................

TurretLevTurretLevel ....................................................................................................................... BotLevel1 ..............................................................................................................24

6
6.1

ProjectProject EvalProject EvaluProject Evaluation and Conclusion...................

FutureFuture Work.....................................................................................................................26

7 8

Glossary......Glossary..........................................................Glossary............................. References.....References...........................................................................References.

List of Figures
Figure 1 - A screenshot of UnrealEd with an early prototype level as the currently opened project. Figure 2 - An overhead view of JumpLevel1. Figure 3 - Screenshot of UnrealEd with TimedLevel as the currently opened project. Figure 4 - Screenshot of UnrealEd with TurretLevel as the currently opened project. Figure 5 - Screenshot of UnrealEd with BotLevel as the currently opened project.

Introduction
The topic of this project addresses difficulty in video games. Users of such games vary in

many ways, including reflex speed, tolerated frustration levels, and ultimate motivations for playing the game. This project tries to address the question of how to implement dynamic difficulty levels to provide a satisfying gameplay experience to a wide variety of users. 1.1 Justification The goal of producing a game is to provide many things, including entertainment, challenge, and an experience of altered state, but ultimately a game must be fun. One major source of polarization among players on this issue is the level of difficulty in a game. There is a great degree of variation in players with respect to skill levels, reflex speeds, hand-eye coordination, tolerance for frustration, and motivations. These differences ultimately cause problems when attempting to implement appropriate difficulty levels in games. Several approaches have been used in the past to provide appropriate difficulty levels in a variety of different ways, such as having a single static difficulty level (chosen either by the designer or through play-testing of the target audience), having several different static difficulty levels to choose from at the start of the game, or providing cheat-codes. However, each method has its drawbacks and limitations, and ultimately cannot provide an appropriate difficulty level to all users. While many would not see this as a problem serious enough to address (indeed, several of the techniques above are satisfying to the majority of gamers), the issue of difficulty in games does have an impact on the breadth of a game s audience, which ultimately impacts on the revenue generated by the game. If the game can be enjoyed by a wider variety of users, more people will buy the game, which will result in increased sales. However, if the game cannot provide a satisfying experience to most people, dissatisfied users will give the game bad reviews (many reviews of games are available in magazines, on television programs, and even for free online, both by professional reviewers and by regular users), and fewer people will buy the game. Dissatisfied users may also sell the game through secondhand stores, resulting in reduced sales of new games, and decreased revenue for the game publisher. 1

1.2

Proposal: Auto-Dynamic Difficulty Some game genres currently use a technique called Auto-Dynamic Difficulty (ADD) to

create more play balance in competitive games, such as racing games. For example, most racing games detect if a player is far behind the other players (i.e. if they crashed near the start of the game) and will increase their acceleration until they manage to catch up to the other players. This allows players to have at least some chance of still winning the game at any particular moment in the game. By monitoring the events occurring in the game, the game is able to use that information to make the game more fun. This paper proposes that ADD can be used in other game genres to monitor the player s skill level, and to use that information to dynamically alter the game s difficulty to best suit the needs of the player. There will be several steps in solving this challenge: 1. Do a background literature search to identify the variables involved, and review previous research into this topic. 2. 3. 4. 5. Determine what quantitative measures can be recorded from the player s in-game actions. Implement an environment to implement and test an auto-dynamic difficulty system. Implement the auto-dynamic difficulty system. Discover what the in-game measures indicate about the player s skill level, motivations, and emotional states through user testing. 6. Finally, determined how that gained information can be applied to alter the gameplay experience accordingly and in a subtle way.

Due to time constraints, only the first three steps will be completed for this project, with implementation of the ADD system and full user group research being handled in a separate study.

Background
There are several user factors that can influence a user s enjoyment of a game. Two main

categories involve dynamic factors of the user and the static attributes of the user. Dynamic factors of the user to be addressed include the skill level of the user, various learning issues (i.e. how is teaching and learning new skills handled in the game, and how quickly can the player 2

learn from that), and the emotional needs that games satisfy. An example of a static user attribute to be addressed is player type , or the way in which the user interacts with the game. 2.1 Dynamic User Factors 2.1.1 Skill, challenge, and optimal experience

Csikszentmihalyi (1996) described what he called flow to refer to an individual s optimal experience . In a state of flow, the individual experiences intrinsic enjoyment from undertaking a task that feels almost effortless and natural, while also causing the individual to feel focussed and challenged. One facet of flow is that there is a balance between the challenge presented by the task and the skill of the individual. This closely resembles Vygotsky s concept of a zone of proximal development, in which a balance between skill and challenge is needed in educational settings in order for learning to take place (as cited in Woolfolk, Winne, & Perry, 2003). Vygotsky pointed out that this zone of proximal development was different for each student, and that tasks considered easy by some may be too difficult for others. Assigning difficulty levels in games must address this problem so as to hit the optimal experience for as many players as possible, each of which may have very different zones of proximal development. 2.1.1.1 The role of learning in games The previously noted application of educational theory to games draws a parallel between games and their role as teaching tools. While many skills learned in games may not be considered conventionally useful (i.e. gaining increased hand-eye coordination (Beth Isreal, as cited by Dobnik, 2004), adaptability to alternate keyboard controls, accuracy in aiming a virtual rocket launcher at virtual aliens, etc.), games are fundamentally entrenched in learning theory and application. Game designers knowingly attempt to teach in video games: games are often designed with tutorial levels, built-in hints if the player is having difficulty, and with rewards and punishments for behaviours that have or have not been learnt properly. Operant conditioning (or the application of pleasant or unpleasant consequences to behaviours) applies to games, as when player error is noted by emission of an unpleasant beep , by the screen flashing, or by the player s avatar crying out in pain. Token economies can be seen in rewards of virtual goods (such as virtual money, coins, or gems). Operant conditioning is also in effect when the player is rewarded for playing with graphical special effects, cutscenes revealing more of the game 3

storyline, or higher scores. Addiction to videogames, particularly as noted in multiplayer games, can also be observed to be rooted in operant conditioning: by playing, the user is rewarded with real-life reinforcement (such as escape from the stressors of everyday life, reduced boredom, and entertainment), as well as virtual rewards (such as virtual money and goods) that are essentially token economies (a method that, incidentally, is often used in institutional settings to reinforce desired behaviours in individuals). The reason levels in a game become more difficult as they progress is directly due to learning: As the player progresses, they learn how to perform the ingame tasks much more efficiently, whether it be a motor task (such as navigating through the world, aiming a virtual weapon, or fighting opponents), or a cognitive task (such as puzzlesolving, strategizing, and planning). If the levels did not become more difficult to match the individual s degree of learning, they would quickly become bored. Finally, even classical conditioning (behaviour control through involuntary reactions to paired stimuli) has been observed in the development of Shock-controllers which give a player a shock in response to in-game events (Rose, 2003). One could view the enjoyment derived from games as being due to the enjoyment derived from successfully learning a new task, or improving a skill and getting an associated reinforcement for doing so. Due to this strong parallel between learning and playing games, it is necessary to consider learning theory in addressing difficulty in games. Learning in the context of auto-dynamic difficulty is particularly important, for, as previously mentioned, player optimal experience will not occur if the game presents challenges beyond the individual user s zone of proximal development. If the game is too difficult, the player will become frustrated, as certain teaching techniques are often misused in games. One that will be addressed is learning through death , or simply having to re-attempt a failed task (and receiving punishment for each failure) until success is achieved. Often in games, if a player fails at a task, they are simply asked to do it again. This can come in the form of causing player death (i.e. you were not successful in jumping across the chasm, therefore you fall into a pit of lava and die) and forcing them to retry, or it can have less severe, yet equally redundant or tedious consequences (i.e. you were not successful in jumping across the chasm, therefore you fall into a lake at a bottom of a very deep pit, and must climb all the way back to the top of the chasm via a series of tedious navigations and a coordinated 4

sequence of button/key combinations). While one may say that practice makes perfect , we must consider whether or not it applies in this context. Vygotsky (as cited in Woolfolk, Winne, & Perry, 2003) argued that, within the zone of proximal development for a specific material to be learned, a student is incapable of learning without a mentor to guide their efforts or provide scaffolding. Above the zone of proximal development, the student is incapable of learning the material, regardless of whether or not they have a mentor, due to a lack of prerequisite knowledge or a faulty schema or mental set. Below the zone of proximal development, the student is capable of learning and performing a task without a mentor. If the task given in a video game is within or below the zone of proximal development (assuming the game provides enough scaffolding/hints and structure to assume the role of a substitute teacher ) having them repeat the task may very well permit them to achieve the task. However, if the task is too difficult, and therefore above the zone of proximal development, repeating the effort (the term effort is used as opposed to task , since the user may not be performing the appropriate steps, using an appropriate strategy, etc.) will not aid the player in progressing in the game, or learning the strategy the game requires them to know before progressing. While most games use learning through death to force the player to retry their attempts at learning, this effort is misguided. 2.1.1.2 Learning through death in the classroom Consider a case where learning through death is actually implemented in a classroom (in a figurative way, of course). Consider, for example, a student with no prior knowledge of calculus taking a learning through death course in advanced calculus and failing the midterm. In this case, the student does not have the prerequisite knowledge and so the topic of advanced calculus is above their zone of proximal development. In the context of games, it is quite likely that someone will purchase a game where the game tasks are above their zone of proximal development. In this example, it can be assumed that tutoring or extra-help is not available, as games often will not provide this specialized need-based help. Learning through death would force the student to take the exact same midterm again and again, until they finally manage to pass it. If the student failed the entire course, likewise, they would have to repeat the entire course without any change in the way the material was presented, nor any feedback from the 5

previous attempt, other than You failed written on their report card. As the educational system in North America currently is, if a student failed a course, when they take it again, the teacher may present it in a different way, or the student can ask pertinent questions in class, or see a tutor to remedy their lack of knowledge in the prerequisite material. As we can see, learning through death would not be effective in the classroom, and so it should not be expected that it would be an effective teaching method in video games. 2.1.1.3 Facilitating learning through reflection and Soul Reaver In the previously discussed learning through death method, usually users are placed immediately back into the situation that caused failure without feedback for improvement or a chance to reflect on a better solution. Norman (as cited in Preece, Rogers, & Sharp, 2002) pointed to the importance of reflection in learning, and in particular to its role in creative thinking. One game where such reflection is permitted, and indeed plays a large role in progress through the game, is Legacy of Kain: Soul Reaver. In this game, death does not cause one to have to restart from a previously saved game. The avatar, a character named Raziel, is essentially a ghost that can take on a material form (through which one interacts with objects in the environment) or a spectral form. When death occurs in the material realm , Raziel is shifted to the spectral version of that location in the world. Meanwhile, none of the progress is lost in the material realm (i.e. previously defeated opponents stay defeated, etc.). However, the player must navigate Raziel to find a portal from the spectral realm to the material realm that will allow him back to the material realm. The time taken to travel back to a portal is not perceived as particularly tedious, for traversal through the spectral realm does not present a great challenge or complex tasks (except spatial navigation, but many landmarks are provided to simplify navigation through the environment). However, the time it takes to make the traversal permits the user time for reflection on whatever difficulty it was that sent Raziel to the spectral realm, and how it may be overcome. In attempting to provide the player pause after failure however, one must be cautious not to bore the player, break their immersion, or otherwise incite them to quit the game. One does not want to force the player to watch a drawn-out cutscene on failure. In some situations the player may even want to immediately try again. Methods that provide the player opportunity for 6

reflection may be more appropriate in instances where creative thinking or a change of strategy might result in success. 2.1.1.4 Tutorials and avatar monologue Many games attempt to circumvent problems with user learning by providing tutorials to help users learn game controls and basic strategies. However there are issues with a static tutorial approach to teaching in games. Static, pre-set tutorials do not assess the player s previous experience, understanding, or skill level, and thus may present material at an inappropriate level. ADD can be used to create a dynamic tutorial that is receptive and responsive to perceived player needs. Some claim that explicit tutorials and hints are intrusive and may interfere with player immersion (Adams, 2004). An alternative method could be implemented in certain types of games where such tutorials or hints are built into the game storyline, or are presented as an internal monologue from the avatar (i.e. the avatar s voice states, I found that I was having difficulty in trying to scale the cliff face directly. Maybe there was another way, while a cutscene shows a scan of the environment, resting on a more promising location, and hopefully prompting the user to change their strategy). ADD could prompt these types of hints at appropriate times (for example, after n failed attempts to scale the aforementioned cliff face). 2.1.2 Emotional needs

"How dare that Triad blow up my brand new Yakuza Stinger while I was listening to my favourite talk radio show!! I'll show them, I'll show ALL of them!!" Christine Bailey, email correspondence on Grand Theft Auto III, September 15, 2004

Another dynamic factor that may influence the user s enjoyment of a game is the user s current emotional motivations in playing the game. Lazzaro (2004) pointed out that play types could be related to the emotional motivations users have for playing games. She proposed four keys that players seek: Hard Fun (emotions from meaningful challenges), Easy Fun/Immersion (emotions from immersion and curiosity), Altered States/Internal Experience (emotions from excitement and relief), and The People Factor (emotions from experiences with other people). The fact that motivations can be influenced by 7

emotions, and not simply by the user s personality type (or player type) suggests that play styles can change dynamically within a game, and one player may engage in several different play styles. These emotional motivations would affect difficulty levels in cases where difficulty levels interfere with desired emotions. For example, a user may begin play seeking Easy Fun/Immersion, thus wishing an easy difficulty level, but may switch to seeking Hard Fun and temporarily wishing difficult challenges. 2.2 Static User Attributes: The Influence Of Player Types Bartle (1996) discussed different player types in the context of multiplayer games. He identified Achievers (oriented towards acting on the game world), Explorers (oriented towards interacting with the game world), Killers (oriented towards acting on other players), and Socializers (oriented towards interacting with other players). His work illustrated that different players have different interests, and therefore may require different experiences to achieve optimal experience. Since this project will be studied in the context of a single-player game, the focus of this work will be on the world-oriented Achievers and Explorers. Achievers seem to be users who are temperamentally Hard Fun seekers, while Explorers may typically seek Easy Fun/Immersion or Altered States/Internal Experience . Adams (2000) discussed the differences between what he identified as core gamers and casual gamers. Adams core gamer was described as being achievement-oriented, enjoying overcoming challenge, and being far more tolerant of frustration. This seems to correlate with Bartle s Achiever. Adams also addressed the casual gamer, whom he described as being motivated to play a game for the sheer enjoyment of playing the game (Adams, 2000, para. 11). Adams suggested that the casual gamer is far less tolerant of frustration. The casual gamer would seem to identify with Bartle s Explorer.

Methods Of Measurement
In order to determine the player s optimal game difficulty level, the ADD system will

have to be able to determine the player s frustration tolerance from measures taken of the user s actions in-game. In order to do so, measurements of player skill and player type can be recorded and interpolated to get an estimated tolerated frustration level. To determine the proper 8

interpolation, user testing will have to be done to determine interactions between player skill, player type, and frustration tolerance. User testing will involve having participants play a controlled game level. Participants may be tested with a prototype ADD system implementation. Surveys may be given to participants before being tested, immediately after they are tested, and a period of time after they are tested (for example, a week). Participants may also be asked to make verbal comments on their play experience while they are playing the game. In this way, the effect of difficulty and frustration can be tested over different lengths of time. Motivation levels may also be tested at this time to determine any interaction between motivation to continue playing the game, difficulty, and frustration. 3.1 Factors To Be Considered Miller (2004a) of 3D Realms Entertainment discussed the implementation of autodynamic difficulty in the game Max Payne. He described methods of judging the player s perceived difficulty by measuring average health, kills made per level, and number of times the player dies per level. He also claimed that counting the number of saved games was not reliable. Several key areas must be studied to provide optimal experience to the player. Some areas of interest are:

% % % % % % %

Player skill Player types and emotional motivators The effect of different rewards How different consequences of failure can impact motivation Player expectations Whether or not a task is perceived as optional or mandatory The player s overall tolerated frustration levels

Any interactions between these variables must also be determined. While player skill is very important when setting a dynamic difficulty level, it should not be the only measure to use when setting the difficulty. Areas that may have far more impact include the player s motivations and tolerance. 9

3.2

Measuring Skill Level The player s skill level could be measured by having the ADD system observe the

player s rate of success in a given task. If the success condition is that a player must jump from one location to another (for example, across a chasm, or up a steep cliff face), then the number of tries before the player succeeds could be measured. If the success condition is general success in combat with opponents, the measures taken can include:

% % % % %

Amount of time taken to defeat the opponent Amount of damage taken by opponents per confrontation Average player health level throughout a level The percentage of opponents successfully defeated per level The frequency with which more complex strategies are used

Measures of variables such as the number of saves and loads and the number of times the player dies per level would not be as successful in identifying player skill. Measuring the number of saves and loads by a player may be independent of the player s skill, as some players may save their progress frequently, whether or not they are successful in the game. Players who have succeeded in missions may even re-load the game to the point before the mission started in order to see if they can be more successful (for example, obtaining a higher score, defeating more opponents, improving their time taken to complete the mission, or even trying different methods of completing the mission). The frequency of saves and loads may also be affected by the user s personal life and choice of when they play the game: if the user plays the game in short spurts (for example, during short coffee breaks), their frequency of saving and loading will be quite high compared to someone who chooses only to play the game for longer periods of time. To consider loads and saves in measuring player skill may actually create a difficulty level that is inappropriate to the player s actual skill level. To measure the number of deaths per level may also pose a similar problem. The player may decide to quit the game upon death, either because of a high frustration level at having died, or because their immersion has been broken. If the player quits the game without saving, the information gained from the ADD system (i.e. the player has died x+1 times) will be lost, so that the next time they enter the game world the 10

difficulty level will be reset to the level it was at before the last save, and the player will therefore not benefit from the ADD system. 3.3 Measuring Player-Type Player type and emotional motivation may affect player experience if the player is forced to engage in activities that they are not interested in. An example is a situation in which a player who is interested in exploring may become frustrated if their explorations are constantly interrupted by opponents that they must fight in order to continue. 3.3.1 Arousal and emotion

Sykes and Brown (2003) studied how to measure arousal level through a gamepad. They found that the pressure of button presses was significantly correlated with difficulty levels in a game, with the buttons being pressed harder when a game s difficulty level was increased. They pointed out that future studies should look at determining the difference between positive arousal states (such as enjoyment) and negative arousal states (such as frustration). They also suggested that measuring galvanic skin response in studying fast-paced games could not be used to achieve accurate results, since galvanic skin response will be affected by muscle tension and perspiration. 3.3.2 Explorers versus achievers

The player type could be measured by having the ADD system observe the player s behaviour within the game. Achievers will tend to move through the game world in a linear fashion, simply working to complete the missions or levels in the game, without regard to exploring aspects of the world that do not concern missions. Explorers will typically manifest themselves in their moment-to-moment movement through the world, in movement displaying disinterest in completing missions or levels and in retracing or staying within previously explored areas for long periods of time. The movement of the Explorer through the game world may be quite slow compared to the Achiever s movement through the world (as the Explorers are taking their time to observe and experience the game world in all its glory). Explorers may also exhibit novel behaviour, such as trying actions that would not contribute to completing mission goals, like interacting with unusual objects such as turning virtual water faucets, chasing virtual chickens, and attempting to talk to inanimate objects. The degree to which these actions can be observed will depend on the game controls and in-game mechanics available to the user. User 11

testing can be done to determine what behaviour differentiates these two groups. 3.4 User Testing User testing will help calibrate the ADD system to find the ideal mapping between recorded user actions and estimated frustration tolerance. This will involve measuring the effects of different rewards and expectations, the consequences of failure, and self-reported frustration. Measuring the effect of different rewards will necessitate giving study participants a task in the game, presenting different rewards, and observing how the player responds to those rewards through survey questions and/or by video capture during or after play. This method can also be used to measure the effect of knowing what the reward will be before the task, and the effect of the task being optional or mandatory. Measuring how different consequences of failure can impact motivation will require presenting different consequences of failure, including minor consequence, moderate consequence, and severe consequence, and observing player reactions in response to each. Determining the interpolation of ADD measured statistics to tolerated frustration level will require identifying frustration levels in study participants. This can be done by telling participants that they can quit at any time and by measuring the time that it takes between when the participant first experiences failure to when they decide to quit, if they decide to quit. Asking the participant to make verbal comments during play on frustration levels can also be used. Once the participant quits the game or the task is complete, the participant can then be questioned to determine their self-assessed frustration level. Once the tolerated frustration levels have been determined in this fashion, an analysis of the interaction between this measure and the in-game measures (of player skill and player type) can be done in order to find out how to interpolate the in-game measures to achieve an estimation of any given player s tolerance for frustration. 3.4.1 Casual versus Core

Ip and Adams (2002) addressed ways of quantitatively measuring levels of core and casual in a given player. They proposed several criteria to differentiate the two groups and that could be investigated in a survey towards players. They also proposed a method to measure these areas quantitatively to get a score, which can then be used to categorize the player into one of five levels: ultra casual/non-gamers, casual gamers, transitional/moderate gamers, hardcore 12

gamers, and ultra hardcore/obsessive gamers. They suggested that the frequency distribution of players along this continuum would not follow a normal curve, but would rather reflect a logarithmic graph. While Ip, et al. (2002) did not implement their concept, these guidelines could be used to determine an optimal implementation of player-type identification in our ADD system, and to design a survey to measure these attributes for the testing of our system with users. 3.4.2 Potential issues in user testing

There may be several problems with asking participants for self-assessed data, including demand characteristics such as the good-subject tendency, and evaluation apprehension/social desirability. These phenomena may occur in studies when subjects change their normal behaviour in order to behave as they think the experimenter wishes them to act, or in order to appear to be as socially desirable or as normal as possible (McBurney, 2001, p. 177). Careful wording of the consent form and the questions will be required to minimize these issues. Deception may also be used in order to minimize demand characteristics (so that the subject does not know that they are in a study, or thinks that they are in a study about an unrelated topic). Careful consideration of these issues will be necessary so as to maintain ethical standards while minimizing demand characteristics. 3.5 Additional Ideas An initial conceptual model was designed in which player tolerance for frustration and measured player skill were mapped onto a two-dimensional graph where the former was mapped onto the y-axis and the later onto the x-axis. The initial concept was based on the idea that the player s frustration tolerance would be determined by the player s choice to a question posed at the beginning of the game. The question would simply ask the user what difficulty level they would like: Easy , Medium , or Difficult . The player s response would give an indication of their tolerance: if they chose Easy , they would be self-reporting a low tolerance for frustration, whereas if they chose Difficult , they would be self-reporting a high tolerance for frustration (or perhaps a high level of core ). Skill level would be determined by the measures previously discussed. The tolerance level would set the y-axis value in one of three positions (if there were three difficulty settings to chose from). This value would be used to determine the number of times a player would tolerate failure, and therefore act as a goal for the ADD system when 13

adjusting difficulty. Since there were fundamental problems involving the clarity of the model and conversion of the model to implementation strategies, and since current views of how to find player tolerance level have changed since the model s development, this model will not suffice. More thorough thought and investigation will be needed to develop a clearer and more representative conceptual model for thinking about auto-dynamic difficulty.

Difficulty Adjustment
Difficulty adjustment requires not only the consideration of what can be adjusted to make

a challenge more or less difficult, but also what motivators might make the user more or less tolerant of such difficulty levels. This may involve the types of rewards for success, consequences for failure, and player expectations. 4.1 Factors To Be Considered Rewards may affect motivation as players may become frustrated if they feel they are not being rewarded appropriately for their efforts. The player s motivations may also change if copious rewards have the effect of changing a player s locus of causality, or reason for the behaviour. Too many rewards, or rewards of the wrong type (for example, inappropriate rewards), may have the effect of changing whether the user is playing due to intrinsic motivation (motivation from internal factors, such as personal interest) or extrinsic motivation (motivation from external factors, such as getting rewards or avoiding punishment) (Woolfolk, 2003, p. 355), and may even decrease motivation (Kohn, as cited by Woolfolk, 2003, p. 224). The consequence of failure can also have a huge impact on motivation if it is unusually harsh or tedious. Many games force the player to replay large sections of the game upon death, while other games have a more lenient consequence to death. In the game Primal, the player controls two avatars, Jen (who is mortal, and therefore prone to dying) and Scree (who is immortal). If Jen dies in the game, the player must use Scree to find a portal, through which he can bring Jen back from death. This means that any progress that had been made to the point of Jen s death is not lost: opponents that have been killed stay dead, and items that have been collected stay in the player s possession. In essence, all that has been lost is the ground travelled in order to find one of the many portals dispersed throughout the game world. Contrast this with 14

a game such as Prince of Persia: The Sands of Time, which, upon death, forces the player to Retry the section between the last checkpoint and where the avatar died. All progress made from that checkpoint to where the avatar died has been lost. Therefore, previously killed opponents must be defeated again and previously solved puzzles must be solved again. This type of consequence to failure quickly becomes tedious. Player expectations may also affect motivation: if the player can see the reward that they will get by struggling through a difficult portion of the game, their motivation to complete that portion of the game may be different depending on the perceived value of the reward. Likewise, knowledge or lack of knowledge about the reward may affect motivation levels, both before the achievement of the reward, and afterwards (as the player may think all challenges afterward will have a similar reward level). Since motivation levels will ultimately have an impact on the player s tolerance of difficulty level, these factors will have to be considered. 4.2 Difficulty Adjustment Through Deception There are many ways of adjusting the difficulty of a level in response to user behaviour. The simplest methods involve deceiving the player, or having the game cheat by changing the game rules without the player s knowledge. Several of these methods will be considered. Note that for the purposes of clarity and conciseness, only cases involving adjusting difficulty downward (to an easier setting) will be discussed; however these changes can also be applied to adjust difficulty upward (to a more difficult setting). One way to modify the world or game rules without player knowledge is by changing external elements to the avatar, such as the game-world architecture/environments or opponents. In the case of environmental changes, simple structural changes can be made in the world to make navigation through the world easier. An example of this would be narrowing a chasm that the user must jump across. Changes made to opponents could involve decreasing enemy health levels, decreasing damage inflicted by enemies, and decreasing enemy hit accuracy. More complex methods may also include modifying enemy artificial intelligence. Another way to modify the game s difficulty is to change the internal elements to the avatar, such as increasing avatar health and increasing avatar weapon damage. Augmentation of 15

the player s hit accuracy can also be implemented, as was done in the game Max Payne (as cited by Miller, 2004a). Events that the player perceive as random can also be altered such that difficulty is modified. An example of this is increasing the frequency of in-game bonuses, such as health packs (Miller, 2004a). However, caution must be exercised so that the player does not notice a drastic change in this random event, or otherwise notice that it is not random. 4.3 Proactive Versus Retroactive Measurements and Corrections In adjusting difficulty levels, one must consider whether adjustment will take place after the player fails (retroactive adjustment), or if the measurements taken will be applied to difficulty settings later on in the game (proactive adjustment). Retroactive adjustment would take place when, after n attempts to complete a task have been failed, the task that was failed is made easier. Proactive adjustment would involve constantly measuring the player s performance on many types of tasks (i.e. jumping skill , aim accuracy , puzzle solving skill ), and then interpolating those measurements via some calculation to determine what the user s performance may be on future tasks based on past performance. Proactive adjustment would require much more in the way of user studies and research to determine what an appropriate interpolation calculation would be. While retroactive adjustment may seem easier to implement and more accurate (we know the player is currently having problems completing the failed task), it also has serious drawbacks. If we do not know what the current user s frustration tolerance is, we may be taking chances that they will lose patience with the game before the ADD system has a chance to become active. Another problem is that the user may attempt the task n-x times (where n is the point at which the ADD alters the difficulty level), and then exit the game without saving (as the user may not bother saving if they feel no progress has been made). When and if the user returns to try again, all previous information gained by the ADD system has been lost. In essence, while retroactive adjustment ADD may be present in the game, it may never have a chance to appropriately alter the difficulty level. Proactive adjustment can resolve the problems with retroactive adjustment in that it need not wait until a user has already experienced the difficulty before becoming active. If the ADD 16

can measure the user s performance on non-critical tasks, the user will never have to become frustrated because they cannot complete a task. Examples of the type of measurements that could be gathered from non-critical tasks is whether or not the player uses defensive tactics in battle, what the avatar s average health level is, or how many attempts does it take the player on average to master a new skill (for example, jumping up a steep cliff-face). The primary disadvantage in using proactive adjustment is that it may not accurately predict the user s future performance. This may be particularly problematic in cases where a user ceases playing a game for an extended period of time (for example, during the school year), causing a temporary decrease in performance when they begin playing again due to lack of practice. A combination of both retroactive and proactive adjustments may also prove useful in avoiding some of the disadvantages of each type of adjustment. 4.4 Difficulty Through Ranking And Scores For Core Players One way in which ADD can be augmented is by providing difficulty through the possibility of higher scores. As previously noted, there is a great difference in the type of gameplay experience sought by casual and core gamers. Core players will desire more challenge, and may become bored if they feel tasks could be achieved no matter what . In some way, however, this is the goal of ADD; that the player will find the difficulty somewhat but not overwhelmingly challenging. ADD allows that tasks will provide a challenge to the user, but that the task will be achievable for that user. Core players could be provided for by having a scoring system that can provide additional challenge, but that can be completely ignored by casual players. This technique has been used successfully in many different games, including Minesweeper (in which players who complete the levels with the fastest times can put their name on a Best Times scoreboard), Devil May Cry (in which the player is rewarded for using complex strategies with varying levels of verbal praise, like Dull or Cool! ), and Dance Dance Revolution Extreme (which provides letter grades for the user s accuracy and timing).

Implementation Of A Test Environment

Before implementing auto-dynamic difficulty, one must have an environment in which to

test it. For the purposes of this project, it was decided that the best way to do this was to use a 17

previously existing and openly available game engine to develop some test levels. ADD would later be added by altering available source code or scripts. 5.1 Initial Tools - The Quake II Engine Initially, efforts were put toward developing an environment for testing auto-dynamic difficulty using the Quake II engine using the source code made available by Vertigo Software in the form of the Quake II .NET build. The Quake II source code had been ported to native and managed C++ for use with Microsoft Visual Studio .NET 2003. A map editor was considered to facilitate modifying levels and to allow the customization and full control over some elements of gameplay to be tested, and also prevent user bias during testing in users who have had previous experience with the existing Quake II levels. Quake Army Knife (QuArK) was chosen to fulfill that role due to its apparent ease of use and provision of a tutorial. 5.2 The Issues With Quake II Quake II was released in 1997 and therefore suffers from the appearance of being a very old game in relation to current game technology. This is particularly apparent in the game graphics, as they are very coarse in comparison to newer games. It is believed that this would affect the study results, as it may result in different responses between participants depending on their degree of experience with video games. Participants who are unfamiliar with current video game technology may not notice that the game engine is older, and so there may be no adverse effects in their observed responses. However, participants who are more familiar with the state of current video game technology may react differently to the game experience, perhaps by exhibiting less enjoyment, or becoming bored or frustrated as a result of the poor graphics. This would result in a confounding of our results, as it would be difficult to determine whether effects were due to differences in participants tolerated frustration levels, dissatisfaction with the graphics, or some other participant expectancy effect. Another problem with Quake II is that it only permits a first-person view and therefore could present a problem in study participants referred to as simulator sickness . Simulator sickness occurs when the illusion of movement is presented to the eyes, yet the corresponding movement is not experienced by the somatosensory system. This can result in nausea, headaches, and dizziness (Owen, 1999; Wen, 2000), and may affect 20% to 40% of the 18

population (Owen, 1999). While simulator sickness first became evident during testing of virtual environments, it has since become apparent that the problem can be experienced while playing first-person 3-D games. It has been suggested that foreground occlusions (areas of the screen which stay stationary, or do not move with the background ) can reduce simulator sickness symptoms (Prothero, 1998). This suggests that games with a third-person perspective are therefore less likely to induce simulator sickness in users. Quake II however only provides a first-person perspective and therefore may cause simulator sickness in some participants. This would cause further confounding of the variables under study. Perhaps the most serious problem with using the Quake II engine however, is the limited form of gameplay possible. Quake II is a first-person shooter, and the game engine was not developed to support development of any other type of game. One of the main areas of particular interest to this project is the investigation of the frustration caused by different types of gameplay situations. One example is jumping puzzles , gameplay in which the user must navigate their avatar to jump from one location to another with some degree of accuracy (usually from one precariously high platform to another, both of which are surrounded by a chasm filled with molten hot lava). In first-person games, fine precision in this task is almost impossible due to the fact that the user usually cannot see their avatar s feet, the location on which they are standing, nor, once they have jumped and are in the air, the location on which they will land. For this reason, most games that place emphasis on jumping puzzles are implemented in a third-person view, in which the user has a view of the game world that includes their avatar (in most current third-person 3D games, this is usually from a few feet behind the avatar). Quake II does not lend itself well to this type of gameplay, and in fact allows little variance in gameplay outside of the realm of first-person shooters. This would greatly limit the type of prototypes that could be developed in this project, and could present validity problems when attempting to interpret and generalize participants responses to the ADD system. 5.3 The Solution: Unreal Tournament 2004 As a result of the problems discussed above, another game engine was considered. Unreal Tournament 2004 is a newer game with a more updated appearance and provides the option of a third-person perspective. The developers of Unreal Tournament 2004 also provide 19

UnrealEd, a level editor for the game engine. The fact that Unreal Tournament 2004 is a newer game would eliminate possible confounding that may occur due to Quake II s outdated graphics. The provision of a third-person perspective in Unreal Tournament 2004 may also reduce the chance of experiencing problems due to simulator sickness in study participants. While Unreal Tournament 2004 was designed primarily as a shooter game, UnrealEd does allow the development of a diverse selection of game types, including first- and third-person games, shooters, role-playing games, racing games, action games, and adventure games. This is probably the greatest advantage in using Unreal Tournament 2004 and UnrealEd over Quake II .NET and QuArK. In addition, Unreal Tournament 2004 provides the ability to dynamically change game levels in-game through the use of scripts. An additional benefit to using UnrealEd is that more support appears to be available, over Quake II .NET, including a comprehensive suite of tutorials on UnrealEd provided by 3D Buzz, Inc. Since the Unreal Tournament 2004 game engine appears to be closer to satisfying the needs of the project, Quake II .NET and QuArK have been abandoned in favour of Unreal Tournament 2004 and UnrealEd. 5.4 Learning How To Use UnrealEd Much work has been focussed on learning how to use the UnrealEd level editor to accomplish the goals of the project. Five different levels, each presenting a different challenge to the user, have been developed to implement and test ADD. UnrealEd allows the creation of levels using what are called builder brushes to create basic polygonal shapes and static meshes to create more complex, pre-defined polygonal shapes. Builder brushes can create polygons via subtraction (i.e. when making a room), and addition (i.e. when creating an object within a room), and other functionality is included such as polygon intersections and de-intersections. UnrealEd also necessitates texture mapping and lighting. Some usage of vertex editing of the polygonal shapes was also used to facilitate level building. Triggers are used to detect user actions (i.e. if a user tries to pick up something), which then emit Events. Other Triggers may listen for, and react to, these Events. Pairings of Triggers and Events can be used to enable interactions with the game world, such as opening doors, using 20

mechanical devices such as elevators, or creating particle effects. These can also be used to manipulate gameplay, such triggering an end game condition.

Figure 1 - A screenshot of UnrealEd with an early prototype level as the currently opened project. UnrealEd also provides special types of objects called Movers, which can move through the level upon detection of a particular Event. Movers are used in the creation mechanisms such as elevators. In that case, the elevator would be a Mover, and the button pushed to make the elevator go up would be a Trigger that emits an Event that would indicate to the Mover to begin a pre-specified movement. While UnrealEd allows building of simple levels, UnrealScript is a scripting language developed to allow more complex interactions and manipulations within the game. UnrealScript is a Java-based, object-oriented language, and has classes, objects, methods that can be called on the aforementioned, and some degree of polymorphism. It is thought that the best way to implement auto-dynamic difficulty in Unreal Tournament 2004 will be through the use of UnrealScript, since UnrealScript may facilitate keeping track of player statistics, such as counting 21

the number of attempts made to achieve a goal. 5.4.1 JumpLevel1 and JumpLevel2 A challenge has been developed in which the user must jump across a chasm to reach an artifact, which will end the level in a win condition. This challenge has been implemented in two maps, called JumpLevel1 and JumpLevel2. Triggers and Events have been used to end the game when the user tries to pick up a particular artifact placed in the level.

Figure 2 - An overhead view of JumpLevel1. The player starts the level on the left and moves toward the artifact on the right (the rightmost turquoise mesh). A deep depression (shown as a yellow rectangle) is present in the middle of the floor, and stairs down to this depression are shown in purple. Stairs are provided to so that the player will not become trapped within the depression if they fail the jump attempt. To make such a level, it was necessary to learn how to implement triggers and events in response to user actions, learn about basic architecture manipulation within the level, as well as manipulate the game type within the level editor, and how to manipulate variables through the UnrealScript scripting language. 22

If the user attempts to jump across the chasm in JumpLevel1 and fails, they will simply fall into a pit that they can climb out of via stairs to try again. In JumpLevel2, this pit is filled with lava, and will therefore cause avatar death. The implementation of both fatal and non-fatal forms of this challenge was done in order to observe how user variables are influenced by this difference. The difficulty level could be altered by the modification or movement of a broken bridge spanning the chasm. The bridge could be implemented as a Mover platform that could move (decreasing the width of the chasm to be crossed) according to the number of attempts made to cross it.

Figure 3 - Screenshot of UnrealEd with TimedLevel as the currently opened project. 5.4.2 TimedLevel A second challenge has been presented in which the user must run through a twisted corridor within a specified amount of time and trigger an object (similar to the artifact mentioned in the previously discussed levels) at the end of it. If the time limit is reached without the user having triggered the artifact, the player will lose. The difficulty level could be adjusted by altering the amount of time given to complete the task. 23

Figure 4 - Screenshot of UnrealEd with TurretLevel as the currently opened project. Note the turrets, which appear in beige in the Dynamic Light (lower left) view, and in red in the other views. 5.4.3 TurretLevel

A level has been developed that consists of a long hallway, with the aforementioned artifact this time placed at the end of the hallway. Along the sides of the hallway are placed turret guns that will automatically fire at the avatar. The challenge is to get to the end of the hallway and trigger the artifact before being fatally shot by the turrets. The difficulty level can be adjusted by altering the amount of damage dealt by the turret bullets, the aiming accuracy of the turrets, or possibly the number of offending turrets. 5.4.4 BotLevel1

A level has been implemented in which there are many bots, or opponents, to defeat before being able to trigger the win condition of the game. This level is currently implemented as just having many hundreds of bots statically placed in the level, but future work would look into the way the bots could be spawned dynamically within the level. Difficulty could be adjusted 24

by altering the damage dealt by the bots, the hit accuracy of the bots, and the number of bots dynamically spawned at once.

Figure 5 - Screenshot of UnrealEd with BotLevel1 as the currently opened project. Note that the bots in the Dynamic Light view appear as yellow wireframes in the other views.

Project Evaluation And Conclusion

This project has undertaken the initial steps necessary to begin a serious attempt at

implementing auto-dynamic difficulty. While more background literature would always be beneficial, the current level of background material is sufficient to provide a structure for the needs of the project. Unfortunately, many informal (and even formal) attempts at auto-dynamic that may have taken place have occurred within companies that do not want their competitors to know about their development processes. Miller (2004b) suggested that game companies might not even want their players to know about the very fact that auto-dynamic difficulty is present in their games. As a result, some of these previous efforts are not publicly available for review. Unfortunately, development of a test environment did not proceed as far as expected.

25

This was partly due to a change in tools halfway through the project, but also due to other time constraints and the fact that UnrealEd and UnrealScript were more complicated than initial impressions may have indicated. Overall, this project provided opportunity for substantial progress in the direction of implementation of auto-dynamic difficulty in video games. 6.1 Future Work There are a few more outstanding issues involved in the test environment, namely determining how to effectively alter the game rules to allow flow from one test level to the next, printing appropriate messages/instructions, and determining how certain currently statically implemented features may be dynamically spawned in-game (for example, bots or architectural features such as bridges or flooring). Immediate future work would focus on the effective usage of UnrealEd and UnrealScript. Once a full test environment is in place, work can proceed to determine how to implement an auto-dynamic difficulty system. Following initial pilot studies to determine any problems with the implementation, user group research can proceed to determine how to calibrate the ADD. This would involve gaining self-reported data from users to identify player type (perhaps taking some guidance from Ip and Adams (2002) categories), frustration tolerance assessment, and user testing of measured skill levels in different tasks and in-game behaviour (as measured by the in-game ADD system), and then determining how these variables interact. Specifically, it would be desirable to find ways through which player attributes can be predicted from the ADD measurements. The results of this can be used to determine what difficulty setting would be suitable to have as an initial setting and what adjustments would be appropriate for different measured skill levels and observed in-game behaviours. Further research could then be done to determine how adjustments to the difficulty could be made without player awareness.

26

Glossary
Achievement of a winning-condition requires overcoming both types of challenge. Some examples of action-adventure games include The Legend Of Zelda, Prince of Persia: The Sands of Time, and Beyond Good and Evil.

Action-adventure games: Games that are a combination of action games and adventure games.

Action games: Games for which the achievement of a winning-condition requires fast responses to action within the game. Typically does not require higher-order reasoning or puzzle solving. Some examples of action games include Pacman, Super Mario Brothers, and Devil May Cry. See also Action-adventure games. ADD: Auto-dynamic difficulty, or the ability for a video game to adapt its set difficulty level to match the player s skills and tolerance level. Adventure games: Games for which the achievement of a winning-condition requires a high degree of logic, reasoning, and problem solving in the context of exploration and narrative. Some examples of adventure games include King s Quest, Myst, and The Longest Journey. Avatar: The in-game persona in a video game that the user controls. Checkpoint: A point at which the game progress has been saved. See also Save. Double-jump: An ability provided in some games allowing the user to jump extraordinarily far or high by tapping the jump button twice in quick succession. First-person perspective: See First-person view. First-person view: A view of a game world from a position such that it appears one is viewing the world from the avatar s eyes. Normally does not show any view of the avatar. Galvanic skin response: A measure of the electrical conductivity of skin. Gamepad: A device specialized to take input for the use of playing video games; often uses buttons and analog controllers (joysticks). Many gamepads also detect duration and amount of pressure of button presses. Gameplay: For the purposes of this paper, gameplay refers to the mechanics through which a game is played or controlled; a set of constraints which guide how the user interacts with the game world, including rules, goal achievement conditions, and the actions the 27

interface allows the user to attempt. Load: A game load is the process of restoring previous saved game data to continue playing the game from that previous level of progress. See also Save. Mission: A single goal statement presented in the game; may involve a series of sub-goals/tasks. Racing games: Games in which gameplay consists of driving a vehicle. Achievement of a winning-condition typically requires getting to a specified location before opponents (computer agents or other players) do. Other tasks such as avoiding obstacles may also be necessary. Role-playing games: Games that require micro-management of avatar statistics, including avatar traits (i.e. strength, intelligence, dexterity), skills (i.e. spell casting, weapon use, blacksmithy) and health maintenance. Typically requires management of a team of avatars. Some examples of role-playing games include Diablo II, EverQuest, and Final Fantasy X-2. Save: A game save is the process of storing the game data (including progress made to that point) for use and continuation at a later time. See also Load. Shooter games: Games in which achievement of a winning-condition depends on shooting enemies with a ranged weapon. Some examples of shooter games include Return to Castle Wolfenstein, Doom, and Max Payne. Somatosensory system: the sensory system responsible for detection of the cutaneous senses, proprioception (the position of one s limbs) and kinesthesis (the movement of one s body and limbs). Third-person perspective: See Third-person view. Third-person view: A view of the game world from a position external to the avatar, such that the avatar is visible.

28

References
http://www.designersnotebook.com/Columns/032_Casual_versus_Core/032_casual_vers us_core.HTM

Adams, E. (2000). Casual versus core. Retrieved October 28, 2004, from

Adams, E. (2004). Postmodernism and the three types of immersion. Retrived July 14, 2004, from http://www.gamasutra.com/features/20040611/adams_01.shtml Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs [Electronic version]. Journal of MUD Research, 1(1). Dance Dance Revolution Extreme. (2004). Konami Digital Entertainment. Redwood City, CA: Konami. Devil May Cry. (2002). Capcom Entertainment. Sunnyvale, CA: Capcom. Dobnik, Verena. (2004). MSNBC - Surgeons may err less by playing video games. Retrieved on October 10, 2004, from http://msnbc.msn.com/id/4685909/ Grand Theft Auto III. (2002). Rockstar Games. New York, NY: Take-Two Interactive Software, Inc. Ip, B., & Adams, E. (2002). From casual to core: A statistical mechanism for studying gamer dedication. Retrieved on October 29, 2004, from http://www.gamasutra.com/features/20020605/ip_pfv.htm Lazzaro, N. (2004). Why we play games: Four keys to more emotion without story. Paper presented at the Game Developers Conference 2004. Abstract retrieved October 20, 2004, from http://www.xeodesign.com/whyweplaygames/xeodesign_whyweplaygames.pdf Legacy of Kain: Soul Reaver. (1999). Crystal Dynamics. San Francisco, CA: Eidos Interactive. McBurney, D. H. (2001). Research Methods. Belmont, CA: Wadsworth/Thomson Learning. Miller, Scott. (2004a, January 19). Auto-dynamic difficulty [Msg 1]. Message posted to http://dukenukem.typepad.com/game_matters/2004/01/autoadjusting_g.html Miller, Scott. (2004b, January 20). Auto-dynamic difficulty [Msg 30]. Message posted to http://dukenukem.typepad.com/game_matters/2004/01/autoadjusting_g.html Minesweeper. (1981-2001). Microsoft Corporation. Seattle, WA: Microsoft Corporation. 29

Owen, G. S. (1999). Simulator Sickness. Retrieved on January 22, 2005, from http://www.siggraph.org/education/materials/HyperVis/virtual.env/percept.iss/simulate.ht m Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction Design. New York, NY: John Wiley & Sons, Inc. Primal. (2003). Sony Computer Entertainment Europe. Foster City, CA: Sony Computer Entertainment America. Prince of Persia: Sands of Time. (2003). Ubisoft Entertainment. San Francisco, CA: Ubisoft, Inc. Prothero, J. D. (1998). The role of rest frames in vection, presence, and motion sickness. Retrieved on January 23, 2005, from http://www.hitl.washington.edu/publications/r-98-11 Quake II. (1997). id Software. Santa Monica: CA: Activision. Quake II .NET. (2003). Vertigo Software. Point Richmond, CA: Vertigo Software Inc. QuArK: Quake Army Knife. (2003). Armin Rigo. Retrieved on November 14, 2004, from http://dynamic.gamespy.com/~quark/download.php3 Rose, Kevin. (2003). Dark tip: build an Xshok controller. Retrieved on March 20, 2004, from http://www.g4tv.com/screensavers/features/44543/Dark_Tip_Build_an_Xshok_Controlle r.html Sykes, J., & Brown, S. (2003). Affective gaming: Measuring emotion through the gamepad. Conference on human factors in computing systems (pp. 732-733). New York, NY: ACM Press. UnrealEd. (2004). Epic Games. New York, NY: Atari Inc. Unreal Tournament 2004. (2004). Epic Games. New York, NY: Atari Inc. Wen, H. (2000). Sim Dizzy. Retrieved on January 20, 2005, from http://archive.salon.com/tech/feature/2000/08/11/sim_sickness/print.html Woolfolk, A., Winne, P. H., & Perry, N. E. (2003). Educational Psychology. Toronto, Ontario: Pearson Education Canada Inc.

30