CSC514

INTRODUCTION TO
ARTIFICIAL INTELLIGENCE

Lecture 4

Dr. Babatunde Sawyerr

Department of Computer Sciences

AI AND GAMES

AI has a long history in the genre of games.

From the first intelligent Checkers player, to the team AI
developed for first-person-shooters, AI is at the core.
This lecture will cover aspects of game AI from:

traditional game playing of Checkers,

Chess, Othello, and
Go to more recent video games, and others.

The minimax algorithm and alpha-beta pruning, which are

central to traditional two-player games will be introduced.
Other algorithms used for AI will be discussed.
2

TWO-PLAYER GAMES

Two-player games are games in which two players

compete against each other. These are also known as
zero-sum games.
The goal then in playing a two-player game is choosing a
move that maximizes the score of the player and/or
minimizes the score of the competing player.
A zero-sum game is one in which the gain of one player is
balanced exactly by the loss of the other player.
Non-zero-sum games are those in which two players
attempt to get rewards from a banker by cooperating or
betraying the other player.
3

Tic-Tac-Toe

Tic-Tac-Toe

Tic-Tac-Toe is also open to numerous optimizations.

Consider, for example, the first X move in Figure 4.1. If
the board is rotated, only three unique moves are
actually possible.
Without optimization, there exist (9!) 362,880 nodes
within the complete game tree.
A static evaluation function (that measure the goodness
of a move) is used to determine the value of a given
move from a given game state. The evaluation function
identifies the relative value of a successor move from the
list of possible moves as a measure of the move quality
toward winning the game.
5

Tic-Tac-Toe

f (n) = win_positions-lose_positions
6

THE MINIMAX ALGORITHM

In simple games, algorithms exist that can search the game

trees to determine the best move to make from the
current state.
The most well known is called the Minimax algorithm.
The minimax algorithm is a useful method for simple twoplayer games. It is a method for selecting the best move
given an alternating game where each player opposes the
other working toward a mutually exclusive goal.
Each player knows the moves that are possible given a
current game state, so for each move, all subsequent
moves can be discovered.
7

THE MINIMAX ALGORITHM

Minimax can use one of two basic strategies:

1. The entire game tree is searched to the leaf nodes (endgames), and
2. The tree is searched only to a predefined depth and then
evaluated.
When we employ a strategy to restrict the search depth to a
maximum number of nodes (do not search beyond N levels of
the tree), the look ahead is restricted and we suffer from what
is called the horizon effect. When we cant see beyond the
horizon, it becomes easier to make a move that looks good
now, but leads to problems later as we move further into this
subtree.
8

THE MINIMAX ALGORITHM

THE MINIMAX ALGORITHM

10

THE MINIMAX ALGORITHM

Note that in a game where perfect information is available to

each player, and no mistakes are made, the end-result will
always be a draw.
An alternative to building the entire search tree is to reduce the
depth of the search, which implies that we may not encounter
leaf nodes.
This is also known as an imperfect information game and can
result in suboptimal strategies of play.
The advantage of reducing the search tree is that game play can
occur much more quickly and minimax can be used for games
of higher complexity (such as Chess or Checkers).
11

MINIMAX AND TIC-TAC-TOE

12

THE MINIMAX ALGORITHM

13

MINIMAX WITH ALPHA-BETA PRUNING

Alpha-beta pruning is a simple algorithm that minimizes

the game-tree search for moves that are obviously bad.
The basic idea of alpha-beta pruning is to identify moves
that are not beneficial, and remove them from the game
tree.
The higher in the game tree that branches are pruned the
greater effect in minimizing the search space of the tree.
In the algorithm behind alpha-beta pruning, two variables
called alpha and beta are calculated and maintained
during the depth-first search of the game tree.
14

MINIMAX WITH ALPHA-BETA PRUNING

The alpha variable defines the best move that can be

made to maximize (our best move) and the beta
variable defines the best move that can be made to
minimize (the opposing best move).
While we traverse the game tree, if alpha is ever greater
than or equal to beta, then the opponents move forces
us into a worse position (than our current best move).
In this case, we avoid evaluating this branch any further.

15

Minimax with Alpha-Beta Pruning

16

MINIMAX WITH ALPHA-BETA PRUNING

17

CLASSICAL GAME AI

The application of AI to games is one of search, knowledge,

and heuristics.
Understanding AI and games is important because it provides
a sandbox to test the efficacy of search algorithms.
Its also a means to understand the complexity of games.
Building a worthy AI algorithm for the game of Go is elusive.
Bridge-playing AI algorithms regularly win at the highest level
of Bridge championships.
Minimax with alpha-beta pruning is a popular technique
among intelligent game-playing programs, but heuristics also
play a big part in building faster and more efficient players.

AIs application in Checkers

Arthur Samuel, an early pioneer in AI and machine learning,

did some of the earliest work in giving computers the ability
to learn from experience.
In addition to programming a computer to play Checkers on
the IBM 701 computer, he pioneered the idea of letting the
program learn by competing against itself.
The resulting Checkers program competed and defeated the
fourth ranked player in USA.
The Checkers program built at the University of Alberta,
Canada, is the first program to win vs a human in the
machine world championship competition.

Checker-Board Representation

The data structure representation of a Checkers board is important

because it is a strong determiner in the efficiency of the search
and evaluation aspects of the program (as well as the amount of
overall memory used by the search game tree, opening book, and
endgame database).
A common representation is a simple 8 by 8 matrix that contains
one of six values (empty, red, black, red-king, and black-king).
An optimization of the 8 by 8 is the 10 by 10 model, which includes
a border of one cell around the entire board (with these cells
containing the static value offboard). This simplifies the move
generator in identifying illegal moves.

Techniques Employed by Checkers Programs

Checkers programs have some similarities with other types of AI

players such as Chess in that they have unique opening and ending
game phases. For the opening books:

Checkers has been widely studied from a theoretical perspective and

there is a large amount of work in investigating opening moves that can
lead to beneficial board configurations.
Commonly, a database of opening moves for a given strategy is
interrogated to select the first set of moves before the evaluation and
search functions are used to drive piece movement.

Static Evaluation Function:

The board evaluator is commonly a weighted numeric feature vector.

Particular features include the number of red pieces, number of black
pieces, disparity of pieces (number of red pieces minus the number of
black pieces), the number of red kings, etc.

Techniques Employed by Checkers Programs

Search Algorithm:

As with most classical games, minimax with alpha-beta pruning is used as

the means to search the game tree.
Checkers has an average branching factor of 10, which is less than Chess,
but large enough to make searching the entire tree infeasible.
While alpha-beta pruning does a great job of minimizing the search tree,
there are other techniques that can be applied heuristically to further
reduce the game trees search space.
A number of search enhancements exist such as windowing, where the
alpha and beta bounds are a window of the previously computed values
and can result in faster searching of the game tree.
Other modifications include Principal Variation Search (PVS), which
applies windowing to each node in the game tree.

Techniques Employed by Checkers Programs

Move History:

To speed up the search, a hash-table is constructed during play that

maintains board configurations and their characteristics. Since particular
boards can show up frequently during a game, they can be stored (with
their associated alpha and beta parameters from the minimax tree) to
minimize searching the particular subtree.
The hash table permits a fast lookup of the board configuration to see if
it has been seen before. If so, its alpha and beta parameters are
returned, which are then used by the alpha-beta search.
A common hash function used in Checkers hash-tables is called Zobrist
hashing. This hash function creates an XOR of the checker board, which
results in uniquely different hash results for different board
configurations (which is necessary for fast hash store and lookup to
ensure a hit).

Techniques Employed by Checkers Programs

End-game Database:

An end-game database contains a relation between board configurations

where a few pieces remain, and the strategy that will lead to a win.
These (typically compressed) databases encode the board in a compact
way and then use an index function to quickly identify the strategy to be
used.

Chess

Chess is an interesting test-bed for intelligent applications because

the game is rich and complex with a massive search space. For this
reason, traditional search algorithms are woefully inadequate to
play a reasonably intelligent game.
Chess is a game of perfect information, unlike games such as
Poker, where not all of the information is known to each player.
Both players see the same Chess board and know all moves that
are possible for both players.
Early Chess computers operated in a brute-force way, as the speed
of the computer was viewed as its greatest asset. Understanding
the complexity of the game of Chess, its now known that more is
required (but computational speed doesnt hurt).

Chess

Chess programs are commonly made up of three modules. The

first is a move generator which analyzes the current board and
identifies the legal moves that can be made.
The second module is the evaluation function, which computes a
relative utility for a given board configuration (how good a given
board is compared to others for a given board configuration).
The final module is the search algorithm which must efficiently
iterate through the available board configurations, given a move,
and decide which path to take through the tree to select the next
move to make.

Chess-Board Representation

A simple representation for a Chess board is an 8 by 8 two

Dimensional array with each cell containing an identifier
representing the state of the cell. For example, 0 would represent
an empty cell, 1 for a white pawn, -1 for a black pawn, etc.
Today, a common representation for Chess programs is the
bitboard.
Using a 64-bit word (available on many computers), each cell on
the Chess board can be represented by a bit in the 64-bit word.
Each bit simply determines if a piece is in the cell (1) or if the cell is
empty (0). But instead of having a single bitboard for the entire
Chess board, theres a bitboard for every type of piece on the
Chess board (one each for the white and black piece types).

Techniques Used in Chess programs

Opening Book Database:

Minimax Search with Alpha-Beta Pruning:

The first few moves in a Chess game are important to help establish good
board position. For this reason, many Chess systems employ a database
of opening moves for a given strategy that can be linearly searched.
Chess systems typically use a modified version of game-tree search by
performing only a shallow search of the game tree using minimax with
alpha-beta pruning.

Static Board Evaluation

Unless the game is near the end-game, the search of the game tree will
not encounter any leaf nodes. A good utility function that helps decide
which move to make given a nearby horizon is used in chess. The utility
function for Chess defines whether a given board configuration is good
or bad for the player.

Other Games

Other games include:

Othello
Go
Backgammon
Poker
Scrabble
Video games.