Video game, 2019
Personal Project, Individual work
Reinforcement Learning
Unity, Tensorflow,
OpenAI gym
Introduction
Machine learning perhaps has a lot of neat illustrative examples. Not sure why I found the necessity to
replicate one but I’m sure that it has its reasons and worked as an introduction into embedded agent simulations
in video games
for me and to explain machine learning to my peers.
The premise here is simple: A prototypical project
to experiment with reinforcement learning agents in unity. The training of the agents is handled locally, the
game has embedded
versions of the agents at different stages of training to illustrates their learning.
The project is
based on Adam Kelly’s project along similar lines with a number of modifications in the training parameters. The
two levels
test of the agent’s speed and ability for precision flight. The snow level specifically challenges the agent’s
depth perception using an invisible particle system to simulate fog.
Training an agent brain
What is the input vector?
Initially I wanted to use a camera and then base the training off the image feed. But upon further digging into
Adam’s project did not go down that path because:
- The ML agent (brain) does not make predictions all the time, only does so in intervals. This is because the predictions on TensorFlow (if on GPU) take up compute time and affect the graphics of the game. If on CPU, you would want to limit the graph compute time anyway.
- It would require two cameras for stereoscopic depth perception
- Its easier to use these preceptor probes as they are apathetic to the texture color etc (which is essentially noise in a Camera based system)
Training environment
The training system. Here apart from the outline on the training mechanism of boundary conditions, a curriculum for learning was used. This is because its easier to set loose goals for the agents and then when they learn that, to tighten up the goals and have them adjust to that.
Training progress
No training
An hour of training
Four hours of training
Gameplay and inferences
The agent acing the desert level
The agent getting confused in the snow level
Note the airplane flying in the snow scape and how it gets confused with the particles, hence flying in a uncertain up and down motion