Volume 21, Number 3-4, June 2004

Michael G. Madden, Tom Howley:

Transfer of Experience Between Reinforcement Learning Environments with
Progressive Difficulty. 375-398
Electronic Edition (link) BibTeX

Experiments with Reinforcement Learning in Environments with Progressive Difficulty",

Michael G. Madden & T. Howley. Proceedings of 14th Irish Conference on Artificial
Intelligence & Cognitive Science, September 2003.
[Madden and Howley, 2004] M. G. Madden and T. Howley.
Transfer of experience between reinforcement learning environments
with progressive difficulty. Artificial Intelligence
Review, 21(34):375398, 2004.
This paper describes an extension to reinforcement learning (RL), in which a
standard RL algorithm is augmented with a mechanism for transferring experience gained
in one problem to new but related problems. In this approach, named Progressive RL, an
agent acquires experience of operating in a simple environment through experimentation,
and then engages in a period of introspection, during which it rationalises the experience
gained and formulates symbolic knowledge describing how to behave in that simple
environment. When subsequently experimenting in a more complex but related
environment, it is guided by this knowledge until it gains direct experience. A test domain
with 15 maze environments, arranged in order of difculty, is described. A range of
experiments in this domain are presented, that demonstrate the bene?t of Progressive
RL relative to a basic RL approach in which each puzzle is solved from scratch. The
experiments also analyse the knowledge formed during introspection, illustrate how
domain knowledge may be incorporated, and show that Progressive Reinforcement
Learning may be used to solve complex puzzles more quickly.

Cet article dcrit une prolongation au renfort apprenant (RL), dans lequel un algorithme
standard de RL est augment avec un mcanisme pour une exprience de transfert acquise dans
un problme de nouveaux mais relatifs problmes. Dans cette approche, appele Progressive
RL, un agent acquiert l'exprience du fonctionnement dans un environnement simple par
l'exprimentation, et puis s'engage dans une priode d'introspection, l'o elle rationalise
l'exprience acquise et formule la connaissance symbolique dcrivant comment se comporter
dans cet environnement simple. En exprimentant plus tard dans un environnement plus
complexe mais plus relatif, il est guid par cette connaissance jusqu' ce qu'il acquire une
exprience directe. Un domaine d'essai avec 15 environnements de labyrinthe, disposs par ordre
de difficult, est dcrit. Une gamme des expriences dans ce domaine sont prsentes, cela
dmontrent l'avantage de RL progressif relativement une approche de base de RL dans laquelle
chaque puzzle est rsolu partir de zro. Les expriences analysent galement la connaissance
forme pendant l'introspection, illustrent comment la connaissance de domaine peut tre

incorpore, et prouvent que l'tude progressive de renfort peut tre employe pour rsoudre des
puzzles de complexe plus rapidement.