Video game, 2019

Personal Project, Individual work

Reinforcement Learning
Unity, Tensorflow, OpenAI gym


Machine learning perhaps has a lot of neat illustrative examples. Not sure why I found the necessity  to replicate one but I’m sure that it has its reasons and worked as an introduction into embedded agent simulations in video games for me and to explain machine learning to my peers.

The premise here is simple: A prototypical project to experiment with reinforcement learning agents in unity. The training of the agents is handled locally, the game has embedded versions of the agents at different stages of training to illustrates their learning.

The project is based on Adam Kelly’s project along similar lines with a number of modifications in the training parameters. The two levels test of the agent’s speed and ability for precision flight. The snow level specifically challenges the agent’s depth perception using an invisible particle system to simulate fog.

Training an agent brain

What is the input vector?

Initially I wanted to use a camera and then base the training off the image feed. But upon further digging into Adam’s project did not go down that path because:

  1. The ML agent (brain) does not make predictions all the time, only does so in intervals. This is because the predictions on TensorFlow (if on GPU) take up compute time and affect the graphics of the game. If on CPU, you would want to limit the graph compute time anyway.
  2. It would require two cameras for stereoscopic depth perception
  3. Its easier to use these preceptor probes as they are apathetic to the texture color etc (which is essentially noise in a Camera based system)

Autopilot Perceptor Image

Training environment

The training system. Here apart from the outline on the training mechanism of boundary conditions, a curriculum for learning was used. This is because its easier to set loose goals for the agents and then when they learn that, to tighten up the goals and have them adjust to that.

Training progress

No training

An hour of training

Four hours of training

Gameplay and inferences

The agent acing the desert level

The agent getting confused in the snow level

Note the airplane flying in the snow scape and how it gets confused with the particles, hence flying in a uncertain up and down motion

Play the game:

In case there are errors try out the direct link here :