I’m Ernesto Felan and this is my engineer takeover post for Hacware Inc. I have been working at Hacware for about a year learning a lot about Big Data and Machine Learning. Lately, I've been working on a product that utilizes machine learning at work and I wanted to see if I could use some Machine Learning to play a game better than I could.
I’ve always wanted to see if I could utilize a Deep Neural Network to play games, after being inspired by some other projects such as training a Recurrent Neural Network (RNN) to play Mario Kart 64 and using Deep Q-Learning to teach an AI to learn to drive (very simply). I wanted to see what I could do with these ideas.
I knew I wanted to start off small, with a simple game that wouldn’t take too much time to create and that I think an A.I. could handle without too much extra work. After some discussions with my peers, I decided on Asteroids. Asteroids is a relatively simple game where the objective is to fly your spaceship around and blow up asteroids. The more asteroids you blow up the higher your score is.So I began coding the game. I knew I wanted to use Python to create the game even though Python isn’t the greatest for making games. I needed Python so I could utilize Tensorflow, the goto library for Deep Learning.
I decided on using the library Pyglet to create the game because I have seen projects before that utilized it well with Machine Learning. It also helped that the documentation for Pyglet provides a tutorial for creating Asteroids so I followed that as far as I could. I don’t necessarily agree with a lot of the ways the tutorial went about creating this game but I wanted to finish the game as fast as possible. You can check out the tutorial at the link below:
So after I followed the tutorial I had most of what I needed. All that was left was to add more parameters for the Neural Network to look at. So I began making lines that shoot out from the spaceship so our Neural Network would be able to see. I needed to make a new module for these sightlines:
LINE_LENGTH = 200
class SightLine():
"""Sight Line from the player"""
def init(self, rotation, player_ship):
angle_radians = -math.radians(rotation)
self.x = player_ship.x + math.cos(angle_radians)
self.y = player_ship.y + math.sin(angle_radians)
self.rotation = rotation
self.rotation_offset = rotation
self.x2 = self.x + math.cos(angle_radians) * LINE_LENGTH
self.y2 = self.y + math.sin(angle_radians) * LINE_LENGTH
self.intersect = (0, 0)This class simply needs the rotation we want the line to point and the position of the ship. Then I apply some code from the fire() method from the player class to determine the end point of the SightLine. Now we need an update function so that our SightLine stays with the spaceship:
def update(self, ship_x, ship_y, ship_rotation):
self.rotation = ship_rotation + self.rotation_offset
angle_radians = -math.radians(self.rotation)
self.x = ship_x
self.y = ship_y
self.x2 = self.x + math.cos(angle_radians) * LINE_LENGTH
self.y2 = self.y + math.sin(angle_radians) * LINE_LENGTHSo this function simply takes the ships x, y and makes that our x, y and performs the same math operations to get where our x2 and y2 should be. Now in order to see these SightLine objects, we need to instantiate them in our main game file, asteroid.py and draw them:
sight_objects =
[sightline.SightLine(0,player_ship),
sightline.SightLine(90, player_ship),
sightline.SightLine(45, player_ship),
sightline.SightLine(135, player_ship),
sightline.SightLine(180, player_ship),
sightline.SightLine(225, player_ship),
sightline.SightLine(270, player_ship),
sightline.SightLine(315, player_ship)]This is not the best way to do this but like I said before I am trying to finish this as fast as possible. So this code creates eight sight lines at every 45-degree interval from the ship. Now we need to call the update method for these objects. So in the asteroid.py update(dt) method add:
def update(dt):
...
for sight_object in sight_objects:
sight_object.update(player_ship.x,player_ship.y,
player_ship.rotation)
...
So now we have invisible lines following our spaceship however we still can’t see them. To fix that we go to the on_draw() method and add:
from pyglet.gl import *
...
@game_window.event
def on_draw():
...
glBegin(GL_LINES)
for sight_object in sight_objects:
glVertex2i(int(sight_object.x),int(sight_object.y))
glVertex2i(int(sight_object.x2), int(sight_object.y2))
glEnd()
...Now we should see something like this:
Now we have lines but they don’t do anything so to fix this there is a little bit more we need to do. First, we need to add a proper hitbox to the asteroids, the tutorial we followed kind of cheated when it came to collision detection and while that is fine for the ship and bullets the SightLine objects need to know exactly where an asteroid is intersecting it. To do this we add a field to the asteroid class called box:
class Asteroid(physicalobject.PhysicalObject):
def init(self, *args, **kwargs):
...
temp_x = self.image.width/2
temp_y = self.image.height/2
self.box = [((self.x-temp_x, self.y+temp_y),
(self.x+temp_x, self.y+temp_y)),
((self.x+temp_x, self.y+temp_y),
(self.x+temp_x, self.y-temp_y)),
((self.x+temp_x, self.y-temp_y),
(self.x-temp_x, self.y-temp_y)),
((self.x-temp_x, self.y-temp_y),
(self.x-temp_x, self.y+temp_y))]So this may look like a lot but all its doing is storing half the height and width of the sprite used for the asteroid and adding it to its x and y to make 8 points that make up a square around the asteroid. We also need to modify the update function for the asteroid to maintain this box:
def update(self, dt):
temp_x = self.image.width/2
temp_y = self.image.height/2
super(Asteroid, self).update(dt)
self.box = [((self.x-temp_x, self.y+temp_y),
(self.x+temp_x, self.y+temp_y)),
((self.x+temp_x, self.y+temp_y),
(self.x+temp_x, self.y-temp_y)),
((self.x+temp_x, self.y-temp_y),
(self.x-temp_x, self.y-temp_y)),
((self.x-temp_x, self.y-temp_y),
(self.x-temp_x, self.y+temp_y))]This change simply moves the box based on the asteroids new position.Now we need to add a function to our util.py module that will determine the intersection point for two lines for this I will be using a library called shapely.
It handles the line intersections for us:
from shapely.geometry import LineString
...
def intersect(line1, line2):
a = LineString(line1)
b = LineString(line2)
if a.intersects(b):
x = a.intersection(b)
return xSo this function creates LineString Objects from the lines that we pass it and first checks to see if they intersect and if they do get the intersection point and return it. Now we can utilize this function to see if our sight lines are intersecting an asteroid, so in the sightline.py add the method:
def collides_with(self, other_object):
if other_object.is_bullet:
return False
if other_object.is_ship:
return False
if other_object.is_sight_line:
return False
sight_line = ((self.x, self.y), (self.x2, self.y2))
intersects = []
for line in other_object.box:
intersect = game.util.intersect(sight_line,line)
if intersect is None:
continue
intersects.append(intersect)
break
if intersects == []:
self.intersect = (0,0)
return False...This method makes sure we are only checking for asteroids then gets the intersections on our sight line for each line of the asteroids box. And if there are no intersections set our self.intersect field to (0,0). Now we need to only return the closest intersection to the ship to we add a little bit more to this method:
...
closest = 10000
nearest_intersect = None
for intersect in intersects:
d = game.util.distance((self.x, self.y), (intersect.x,intersect.y))
if d < closest:
closest = d
nearest_intersect = intersect
if nearest_intersect is None:
return False
self.intersect = (nearest_intersect.x, nearest_intersect.y)
return TrueThis snippet checks each intersects distance from our ship and sets the sight line objects self.intersection field to the closest intersection.Now we need to make each sightline check if it's colliding with any asteroids for this we need to add onto our update function in the main game module:
def update(dt):
...
for sight_object in sight_objects:
for obj in game_objects:
sight_object.collides_with(obj)
if sight_object.intersect != (0,0):
print(sight_object.intersect)And that's it now each SightLine will check if it's colliding with each asteroid and print the coordinates of the intersection to the console. The console should look something like this when the lines go over an asteroid:
In the next part, I am going to take all the information from these sight lines and the ship itself to try and train a Neural Network to play Asteroids. The completed code for this project can be found on my personal Github.