ROBOCODE: USING GAMES TO TEACH ARTIFICIAL INTELLIGENCE

Ken Hartness
Sam Houston State University

ABSTRACT
This paper describes a game called Robocode that allows the development of virtual
robots using the Java language. By incorporating an enjoyable game into an artificial
intelligence class and providing students with tools for developing practical versions of the
algorithms discussed in class, the students should be able to better appreciate the theory and
develop greater confidence in their understanding of it.

KEEPING CLASSES INTERESTING

Small academic programs are often concerned with issues of student retention. Many
such programs would like to encourage students enrolled in a freshman class to consider
extending that class into a major. At the other end of the major, professors would like to
encourage students to enroll in courses that are interesting to teach. Professors teaching senior-
level courses, especially those not explicitly required by students' degree plans, may have to wait
until the last minute to discover whether the course will have enough students. Many modern
computer science students consider whether a course will have practical benefit for them and
often do not appreciate the benefits of a theoretical framework on which to build greater
knowledge. Of course, the notion of whether they will enjoy a class may influence their decision
to take it, as well.
In one case, a professor named William Clark converted an engineering course based on
lecture and homework so that it utilized a collection of CD's that included multimedia
presentations and a virtual laboratory. Students expressed greater satisfaction with the course
and more tended to view themselves as good students who understood the material; test scores,
however, only increased slightly [2]. If nontraditional teaching methods do not hurt learning and
leaves the students feeling more positive about what they learned, then these methods should
probably be incorporated into the course. The only difficulty is the overhead of creating this
nonstandard learning environment.
Fortunately, several computer games exist in source code form or with some form of
interface that supports extensions. Many students enjoy playing games and can appreciate, to a
point, the inner workings of such a program. Using such games as a starting point, students can
solve interesting problems and actually see the quality of their solutions unfolding on the
display. By providing them with enough code that they can construct interesting solutions in a
reasonable period of time, students learn important lessons about modularity while crafting
software that will keep them interested in the course and encourage other students to try it. Also,
games are useful from a teaching standpoint because they utilize several areas of computer
science.
Artificial intelligence certainly can benefit from a game engine that can be easily used or
modified. Clarke and Noriega [3] are working on a war strategy game with hooks for the
addition of A.I. modules. They are convinced that students will find A.I. much more interesting
and accessible with examples and projects based on this game. Cook and Holder [4] use a
simpler game to teach students about the need for internal representations of the world, natural
language processing, look-ahead search, plan generation, and machine learning. They have
students incrementally improve or modify the game to handle different problems. For example,
one of the modifications involves a
software seer that describes
situations on the game board in
English. The last version can
simply ignore these
pronouncements, or the student can
add support for natural language
processing and benefit from them.
Both of these examples
required the creation of a program
with all the complexity that a
computer game entails. Another
application, Robocode [7], has
already been written. Written in
the Java language, the game
supports the creation of new robots
Figure 1 - Robocode Battle Screen
that are free to use all of the power
of the Java language. Students can create autonomous agents using a variety of A.I. techniques,
then observe how their agent performs relative to others by watching the agents shoot and ram
each other.

ROBOCODE
Robocode [7] is a Java-based virtual robot game that is intended to teach some Java
programming techniques, like the usefulness of inheritance and object-oriented programming, in
general. It also provides an introduction to event-driven programming.
The Robocode game consists of a robot-development tool and a virtual arena (see fig. 1)
in which the robots can battle for supremacy. Creating a new robot results in a new Java class
derived from the Robot class. Robot objects have a run method with an endless loop that
describes the default behavior for the robot. In addition, other methods like onHitWall and
onScanRobot are invoked in response to events that are recognized by the game's simulation
engine. These event-driven methods provide the robot with input to which it can respond,
immediately, or record for later processing. Responses include commands like ahead(distance),
back(distance), turnLeft(degrees), turnRight(degrees), fire(strength), turnGunLeft(degrees), etc.
Each robot class can contain data, so it is possible to create a memory of past events and
a model of the world.
Proponents of an object-first approach to the CS1 course could use Robocode to illustrate
elements of object-oriented programming without introducing state information or control
structures, although this does limit the quality of the resulting robot [1]. Students are able to see
their objects represented as visual objects that perform their activities, or at least the
consequences of their calculations, on the computer screen. As new elements of the language are
learned, they can be applied to producing more sophisticated robot behavior.
Users of Robocode gradually learn strategies (hide, chase, chaotic, sit still and shoot
everything, etc.). Different strategies are tried or combined in different ways with other
strategies, then the modified robot is put in the game. If the robot fails to perform satisfactorily,
modifications can be quickly made and the robot played, again.
PRACTICAL APPLICATIONS OF A.I. TO ROBOCODE
In order to make Robocode convenient to use for testing A.I. algorithms, limited support
for different algorithms are provided to the students. These include a generic minimax game tree
search algorithm with an interface for students to develop their own state objects and evaluation
functions, a generic genetic algorithm and an interface for students to use in developing their
own genome representations and evaluation functions, and the skeleton for a reinforcement
learning algorithm called Q learning.
A library of strategies and supporting functions are provided to students so they can
easily create robots that try different strategies under different conditions and maintain a model
of the world around them.
The game tree search involves maintaining a model of the surrounding environment and
incrementally modifying it to reflect different behaviors on the part of a robot, then trying
different behaviors on the part of another robot, and so on. Minimax game tree search is
intended to support turn-taking games, but Robocode enforces a kind of lock-step processing.
Even though it appears to be a continuous game, it is reasonable to view the behaviors of the
robots in terms of discrete states with a finite number of successor states. In short, the game can
be viewed at some level as a turn-taking game.
Genetic algorithms can be explored by encoding the presence of different strategies and
responses. Create a GARobot that loads and interprets one of these encodings, then play a game
with a population of GARobots, each with a different encoding. When a robot is killed, its
performance is evaluated. When only one robot is left, the last GARobot performs genetic
selection, crossover, and mutation, selecting mostly the best performing robots.
Reinforcement learning, specifically Q learning, encodes the various situations in which
a robot may find itself. For each of these situations, a set of possible responses is encoded.
Weights, or Q values, are associated with each response. The algorithm will ideally learn to
associate the largest weight with the response or strategy that is most beneficial in a given
situation.
A state interface and response class provides the framework for the Q learning algorithm.
The students are guided in the creation of an implementation of state. Event handlers of the
Robot class allow the student to track elements of state like proximity of other robots and attack
by other robots. In addition, event handlers are executed that can notify the robot of a winning
or losing situation as well as individual successful attacks, allowing students to plan for
reinforcement of beneficial or detrimental behavior.
A supporting library and included examples of its use will simplify complex chores like
tracking other robots, predicting their location, and translating between relative and absolute
coordinates. The evolving software support system may be downloaded from
http://www.shsu.edu/~csc_kth/cs582.

CONCLUSION
Students will be encouraged to master difficult concepts if they can apply the knowledge
to interesting or fun situations. Games, like Robocode, can be used to provide an existing
framework so that students can produce significant behaviors without necessarily mastering all
of the details of creating a game.
REFERENCES
[1] Borge, R.E., Kaasbøll, J., What is "OO first"?, Seventh Workshop on Pedagogies and Tools
for Learning Object-Oriented Concepts (ECOOP 2003), 2003.

[2] Clark, W.M., Using multimedia and cooperative learning in and out of class, Proceedings of
the 1997 Frontiers in Education Conference, 1997.

[3] Clarke, D., Noriega, L., Games design for the teaching of artificial intelligence, Interactive
Convergence: Research in Multimedia, Aug. 7-9, 2003, Prague, Czech Republic, 2003.

[4] Cook, D.J., Holder, L.B., A client-server interactive tool for integrated artificial intelligence
curriculum, Proceedings of the FLAIRS Special Track on AI Education, pp. 206-210, 2001.

[5] Laird, J.E., Using a computer game to develop advanced AI, Computer 34 #7, July 2001, pp.
70-75, 2001.

[6] Loadholtes, N., IBM's Robocode: A Platform for Learning AI,

http://ai-depot.com/Essay/Robocode.html, copyrighted 2001-2003 (viewed 10/16/2003).

[7] Nelson, M., Robocode, IBM, http://robocode.alphaworks.ibm.com/home/home.html, 2001.