Playing to train your video game avatar
PhD thesis of Ronan Le Hy (2007)
The goal of this work is to demonstrate how, by using the Bayesian inverse programmingtechnique, a player of a video game can teach an avatar how to play.
However, we first show how inverse programming is also very useful in simplifying the programming burden of a video game development team.
Bayesian inverse programming consists of expressing independently the conditional probabilities of the conditions, knowing the action. Although atypical, this modelling method appears to be convenient and generic and to lead to very simple learning schemes.
Today’s video games feature synthetic characters involved in complex interactions with human players. A synthetic character may have one of many different roles: a tactical enemy, a partner for the human, a strategic opponent, a simple unit among many, or a substitute for the player when he or she is unavailable.
In all of these cases, the game developer’s ultimate objective is for the synthetic character to act as if it were controlled by a human player. This implies the illusion of spatial reasoning, memory, common-sense reasoning, using goals, tactics, planning, communication and coordination, adaptation, unpredictability, and so on. In current commercial games, basic gesture and motion behaviours are generally satisfactory. More complex behaviours usually look much less lifelike. Sequencing elementary behaviours is an especially difficult problem, as compromises must be made between too-systematic ordering that always appears as an automaton and too-hectic behaviour that looks ridiculous.
We address this problem of real-time reactive selection of elementary behaviours for an agent playing a first-person shooter (FPS) game called Unreal Tournament. In this kind of game, a group of people play together via the Internet. Each of them can control a virtual avatar. This avatar may act and navigate in a virtual 3D environment. It may also interact with the avatars of other players or with autonomous characters called bots controlled by a program. In FPS games, the main interaction with the other players and bots consists of trying to slaughter them while surviving as long as possible. Programming bots is an important, difficult and time-consuming task for the game development team. Simplifying this task is our first objective in this work.
If an avatar could learn while being operated by the player, it could gain its autonomy and become a bot. Many players would love to have autonomous bots playing for them and like them when they are away from the keyboard. A first step in this direction is our second objective.