Kyle W. Banks

Accessing the Goodreads API with Go

Sun, 27 Jan 2019 00:00:00 +0000

Goodreads is one of my favorite services, as I love reading (and spend quite a bit of time doing it, to be honest). My goal for 2019 is to read 75 books, and an additional 3 books in German as I’ve been learning the language over the last couple of years. Here’s where I’m at so far, and we’re only in January:

[4 stars] The Man in the High Castle
[4 stars] The Start-Up J Curve: The Six Steps to Entrepreneurial Success
[1 stars] The General Theory of Employment, Interest, and Money
[4 stars] Red Notice: A True Story of High Finance, Murder, and One Man’s Fight for Justice
[3 stars] Natural Language Processing in Action
[4 stars] An Astronaut’s Guide to Life on Earth
[4 stars] Excelsior!: The Amazing Life of Stan Lee
[4 stars] The End of the Fucking World
[4 stars] 30-Second Economics
[3 stars] The Grand Design
[5 stars] Travelers in the Third Reich: The Rise of Fascism: 1919–1945

I tend to read a lot of books on finance, physics, history and software development, with some fiction and comics mixed in from time to time. Of the books above, I’d most recommend Travelers in the Third Reich: The Rise of Fascism: 1919–1945 and An Astronaut’s Guide to Life on Earth. The former is an eye-opening look at what it was like to live and travel in Germany between the two world wars, and really gives you perspective on how the second world war came to be. The latter is an inspiring autobiography of Chris Hadfield, a Canadian astronaut who dreamed of becoming an astronaut long before it was even possible for Canadians.

Anyways, I thought it would be cool to add a sort-of “bookshelf” to my website, and noticed that there weren’t any solid clients for the Goodreads API written in Go, so I decided to put one together at github.com/KyleBanks/goodreads. It’s pretty straightforward, but first you’ll need to sign up for a developer key at goodreads.com/api/keys. Once you’ve got that, you’re ready to go.

First, you’ll want to initialize a goodreads.Client using your developer key. Here’s how you might do that, using an API_KEY environment variable:

package main

import (
    "os"

    "github.com/KyleBanks/goodreads"
)

func main() {
    key := os.GetEnv("API_KEY")	
    c := goodreads.NewClient(key)
}

After that, you can make calls to the API using the corresponding function. For instance, to call the user.show API endpoint, you’d use the UserShow function:

u, err := c.UserShow("user-id")

To quickly put together the markdown for the list of books above, here’s what I did:

reviews, _ := c.ReviewList("38763538", "read", "date_read", "", "d", 1, 11)
for i, rev := range reviews {
	fmt.Printf(" %d. [%d stars] [%s](%s)\n", i+1, rev.Rating, rev.Book.Title, rev.Book.Link)
}

This loads the read bookshelf for my Goodreads user ID, sorted by date_read in descending order, limiting to the 11 books I’ve read this year. It’s pretty straightforward, but there’s a lot of work to do in order to fill in all the API endpoints that Goodreads provides. If you’re interested, I’d be very grateful to receive pull requests at github.com/KyleBanks/goodreads and to hear what you’re using the client for!

Loading Unlabeled Images with ImageDataGenerator flow_from_directory in Keras

Fri, 25 Jan 2019 00:00:00 +0000

The ImageDataGenerator class in Keras is a really valuable tool. I’ve recently written about using it for training/validation splitting of images, and it’s also helpful for data augmentation by applying random permutations to your image dataset in an effort to reduce overfitting and improve the generalized performance of your models.

The flow_from_directory function is particularly helpful as it allows you to load batches of images from a labeled directory structure, where the name of each subdirectory is the class name for a classification task. For instance, you might use a directory structure like so for the MNIST dataset:

train/
	0/
	1/
	2/
	3/
	4/
	5/
	6/
	7/
	8/
	9/

In this case, each subdirectory of the train directory contains images corresponding to that particular class (ie. classes 0 through 9). Loading this dataset with ImageDataGenerator could be accomplished like so:

datagen = ImageDataGenerator()
train_data = datagen.flow_from_directory('./train')

You’ll then see output like so, indicating the number of images and classes discovered:

Found 1000 images belonging to 10 classes.

Frequently however, you’ll also have a test directory which doesn’t have subdirectories as you don’t know the classes of those images. Inside of test is simply a variety of images of unknown class, and you can’t use the flow_from_directory function like we did above as you’ll end up with the following issue:

datagen = ImageDataGenerator()
train_data = datagen.flow_from_directory('./test')

Found 0 images belonging to 0 classes.

It can’t find any classes because test has no subdirectories. Without classes it can’t load your images, as you see in the log output above. There is a workaround to this however, as you can specify the parent directory of the test directory and specify that you only want to load the test “class”:

datagen = ImageDataGenerator()
test_data = datagen.flow_from_directory('.', classes=['test'])

Found 1000 images belonging to 1 class.

The classes argument tells flow_from_directory that you only want to load images of the specified class(es), in this case the test “class”. Now test_data will contain only the one test “class”, allowing you to work with your test dataset just as you would the labeled training data from above.

In fact, you can also use this trick to exclude subdirectories of your dataset for whatever reason. Going back to the MNIST problem above, if you only cared about recognizing digits 0 through 3, you could do something like this:

datagen = ImageDataGenerator()
train_data = datagen.flow_from_directory('./train', classes=['0', '1', '2', '3'])

Found 400 images belonging to 4 classes.

Training/Validation Split with ImageDataGenerator in Keras

Tue, 22 Jan 2019 00:00:00 +0000

Keras comes bundled with many helpful utility functions and classes to accomplish all kinds of common tasks in your machine learning pipelines. One commonly used class is the ImageDataGenerator. As the documentation explains:

Generate batches of tensor image data with real-time data augmentation. The data will be looped over (in batches).

Until recently though, you were on your own to put together your training and validation datasets, for instance by creating two separate folder structures for your images to be used in conjunction with the flow_from_directory function.

For example, the old way would be to do something like so:

TRAIN_DIR = './datasets/training'
VALIDATION_DIR = './datasets/validation'

datagen = ImageDataGenerator(rescale=1./255)

train_generator = datagen.flow_from_directory(TRAIN_DIR)
val_generator = datagen.flow_from_directory(VALIDATION_DIR)

Recently however (here’s the pull request, if you’re curious), a new validation_split parameter was added to the ImageDataGenerator that allows you to randomly split a subset of your training data into a validation set, by specifying the percentage you want to allocate to the validation set:

datagen = ImageDataGenerator(validation_split=0.2, rescale=1./255)

Then when you invoke flow_from_directory, you pass the subset parameter specifying which set you want:

train_generator = datagen.flow_from_directory(
    TRAIN_DIR, 
    subset='training'
)

val_generator = datagen.flow_from_directory(
    TRAIN_DIR,
    subset='validation'
)

You’ll note that both generators are being loaded from the TRAIN_DIR, the only difference is one uses the training subset and the other uses the validation subset.

And that’s all, it’s as easy as specifying the two parameters as needed. Now that you have your training and validation sets, did you know you can also use the ImageDataGenerator to load unlabeled images, such as an unlabeled test dataset?

Running Multiple make Targets Concurrently

Sat, 19 Jan 2019 00:00:00 +0000

make is a widely used and powerful took for coordinating common tasks within a project, and you’ll often file a Makefile sitting in the root of open source repositories, allowing you to quickly see how to perform common tasks such as running tests, compiling and running, etc. There is also a wealth of tooling around make, one of my favorites being Modern Make, or mmake by TJ Holowaychuk which adds some niceties around make without changing anything about the original command.

If you’re unfamiliar with make, or it’s just been a while, here’s a quick refresher. Create a Makefile containing task definitions for your project. Here’s a relevant section of the Makefile for the website https://whatsthecodeforthat.com that we’ll be working with later on:

local-server:
	hugo server -D
.PHONY: local-server

local-browser:
	open http://localhost:1313
.PHONY: local-browser

This Makefile contains two targets, local-server and local-browser. The first, local-server compiles the hugo project and runs a local server to host the compiled website. The second, local-browser, simply opens the default browser to the hugo development server address on localhost.

Executing either of these targets is then simply:

$ make local-server
$ make local-browser

But herein lays the problem and the purpose of this post, as local-server is a blocking command which means I’m unable to run local-browser right afterwards. To fix this, I could open a second terminal tab and use one for the server and one for opening the browser, but that’s pretty clunky and I generally have enough tabs as it is.

`make` Concurrently

The solution is to run the two commands concurrently, using the -j argument of make:

$ make -help
Usage: make [options] [target] ...
Options:
  ...
  -j [N], --jobs[=N]          Allow N jobs at once; infinite jobs with no arg.

As the -help description explains, -j allows you to run a variable number of jobs (or targets) at once.

This is how it looks in practice:

$ make -j 2 local-server local-browser

This command will run the two commands simultaneously, in this case starting the local server while concurrently opening the browser. Pretty handy, but we can even do one better by creating a new make target that handles this. Back in the Makefile:

local:
	make -j 2 local-server local-browser
.PHONY: local

local-server:
	hugo server -D
.PHONY: local-server

local-browser:
	open http://localhost:1313
.PHONY: local-browser

Nothing changed with local-server or local-browser, the only difference is a new local target that recursively calls make with the -j 2 argument and the two other targets. Now running the two commands concurrently is as simple as:

$ make local

There are a number of use cases for this kind of parallel functionality, but one of my favorites is to parallelize slow build processes (especially in legacy projects) to reduce wait time during clean, compilation, startup, etc.

Using a Convolutional Neural Network to Play Conway's Game of Life with Keras

Wed, 02 Jan 2019 00:00:00 +0000

The goal of this post is to train a neural network to play Conway’s Game of Life without explicitly teaching it the rules of the game.

Speaking of which, if you’re not familiar with Conway’s Game of Life, the rules are as follows:

The universe of the Game of Life is an infinite, two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, alive or dead, (or populated and unpopulated, respectively). Every cell interacts with its eight neighbours, which are the cells that are horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

Any live cell with fewer than two live neighbors dies, as if by underpopulation.

Any live cell with two or three live neighbors lives on to the next generation.

Any live cell with more than three live neighbors dies, as if by overpopulation.

Any dead cell with exactly three live neighbors becomes a live cell, as if by reproduction.

The initial pattern constitutes the seed of the system. The first generation is created by applying the above rules simultaneously to every cell in the seed; births and deaths occur simultaneously, and the discrete moment at which this happens is sometimes called a tick. Each generation is a pure function of the preceding one. The rules continue to be applied repeatedly to create further generations.

Wikipedia, Conway’s Game of Life

Why do this? Mostly for fun, and to learn a little bit about convolutional neural networks. It’s probably overkill for Conway’s Game, but it’s still pretty interesting to see how it performs.

Without further ado, let’s jump into it.

Game Logic

The first thing to do is define a function that takes a game board as input and returns the next state. Luckily there are plenty of implementations available online, such as the following from https://jakevdp.github.io/blog/2013/08/07/conways-game-of-life/. Basically, it takes a game board matrix as input where 0 represents a dead cell and 1 represents a living cell, and returns a matrix of the same size but containing the state of each cell on the next iteration of the game.

import numpy as np

def life_step(X):
    live_neighbors = sum(np.roll(np.roll(X, i, 0), j, 1)
                     for i in (-1, 0, 1) for j in (-1, 0, 1)
                     if (i != 0 or j != 0))
    return (live_neighbors == 3) | (X & (live_neighbors == 2)).astype(int)

Generate Game Boards

With the game logic in place, the next thing we’ll want is some way to randomly generate game boards (frames) and a way to visualize them. generate_frames creates num_frames random game boards with a particular shape and a predefined probability of each cell being ‘alive’, and render_frames draws image representations of two game boards side-by-side for comparison, where living cells are white and dead cells are black:

import matplotlib.pyplot as plt

def generate_frames(num_frames, board_shape=(100,100), prob_alive=0.15):
    return np.array([
        np.random.choice([False, True], size=board_shape, p=[1-prob_alive, prob_alive])
        for _ in range(num_frames)
    ]).astype(int)

def render_frames(frame1, frame2):
    plt.subplot(1, 2, 1)
    plt.imshow(frame1.flatten().reshape(board_shape), cmap='gray')

    plt.subplot(1, 2, 2)
    plt.imshow(frame2.flatten().reshape(board_shape), cmap='gray')

Let’s see what these frames look like:

board_shape = (20, 20)
board_size = board_shape[0] * board_shape[1]
probability_alive = 0.15

frames = generate_frames(10, board_shape=board_shape, prob_alive=probability_alive)
print(frames.shape) # (num_frames, board_w, board_h)

(10, 20, 20)

print(frames[0])

[[0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0],
 [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0],
 [0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1],
 [1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0],
 [1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0],
 [0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0],
 [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0],
 [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0]])

The following takes the integer representation of a game board from above, and renders it as an image. To the right it also renders the next state of the game board using the life_step function:

render_frames(frames[1], life_step(frames[1]))

Construct a Training and Validation Set

Now that we can generate data, we’ll generate training, validation and test sets.

Each element in the y_train/y_val/y_test arrays will represent the next Game of Life board for each board frame in X_train/X_val/X_test.

def reshape_input(X):
    return X.reshape(X.shape[0], X.shape[1], X.shape[2], 1)

def generate_dataset(num_frames, board_shape, prob_alive):
    X = generate_frames(num_frames, board_shape=board_shape, prob_alive=prob_alive)
    X = reshape_input(X)
    y = np.array([
        life_step(frame) 
        for frame in X
    ])
    return X, y

train_size = 70000
val_size   = 10000
test_size  = 20000

print("Training Set:")
X_train, y_train = generate_dataset(train_size, board_shape, probability_alive)
print(X_train.shape)
print(y_train.shape)

Training Set:
(70000, 20, 20, 1)
(70000, 20, 20, 1)

print("Validation Set:")
X_val, y_val = generate_dataset(val_size, board_shape, probability_alive)
print(X_val.shape)
print(y_val.shape)

Validation Set:
(10000, 20, 20, 1)
(10000, 20, 20, 1)

print("Test Set:")
X_test, y_test = generate_dataset(test_size, board_shape, probability_alive)
print(X_test.shape)
print(y_test.shape)

Test Set:
(20000, 20, 20, 1)
(20000, 20, 20, 1)

Build a Convolutional Neural Network

Now we can take a first crack at building a Convolutional Neural Network using Keras. The key here is the kernel_size of (3, 3) and strides of 1. This instructs the CNN to look at a 3x3 matrix of surrounding cells for each 1 cell of the board it looks at, including the current cell itself.

For example, if the following were a game board and we were at the middle cell x, it would look at all the cells marked with an exclamation mark ! and the x cell. It would then move along 1 cell to the right, and do the same, repeating over and over until it’s looked at every cell, and its neighbors, of the entire board.

The rest of the network is pretty basic, so I won’t go into much detail about it. If you’re curious about anything, I’d recommend checking out the documentation for the class in question.

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Conv2D, MaxPool2D

# CNN Properties
filters = 50
kernel_size = (3, 3) # look at all 8 neighboring cells, plus itself
strides = 1
hidden_dims = 100

model = Sequential()
model.add(Conv2D(
    filters, 
    kernel_size,
    padding='same',
    activation='relu',
    strides=strides,
    input_shape=(board_shape[0], board_shape[1], 1)
))
model.add(Dense(hidden_dims))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Let’s look at the model using the summary function:

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_9 (Conv2D)            (None, 20, 20, 50)        500       
_________________________________________________________________
dense_17 (Dense)             (None, 20, 20, 100)       5100      
_________________________________________________________________
dense_18 (Dense)             (None, 20, 20, 1)         101       
_________________________________________________________________
activation_9 (Activation)    (None, 20, 20, 1)         0         
=================================================================
Total params: 5,701
Trainable params: 5,701
Non-trainable params: 0
_________________________________________________________________

Train and Save the Model

With the CNN defined, let’s train the model and save it to disk:

def train(model, X_train, y_train, X_val, y_val, batch_size=50, epochs=2, filename_suffix=''):
    model.fit(
        X_train, y_train, 
        batch_size=batch_size, 
        epochs=epochs,
        validation_data=(X_val, y_val)
    )
    
    with open('cgol_cnn{}.json'.format(filename_suffix), 'w') as file:
        file.write(model.to_json())
    model.save_weights('cgol_cnn{}.h5'.format(filename_suffix))


train(model, X_train, y_train, X_val, y_val, filename_suffix='_basic')

Train on 70000 samples, validate on 10000 samples
Epoch 1/2
70000/70000 [==============================] - 27s 388us/step 
    - loss: 0.1324 - acc: 0.9651 - val_loss: 0.0833 - val_acc: 0.9815
Epoch 2/2
70000/70000 [==============================] - 27s 383us/step 
    - loss: 0.0819 - acc: 0.9817 - val_loss: 0.0823 - val_acc: 0.9816

This model achieves a little over 98% accuracy on both the training and validation sets, which is pretty good for a first pass. Let’s try to figure out where we’re making mistakes.

Try it Out

Let’s take a look at a prediction for a random game board and see how it does. First, generate a single game board and take a look at the correct subsequent frame:

X, y = generate_dataset(1, board_shape=board_shape, prob_alive=probability_alive)

render_frames(X[0].flatten().reshape(board_shape), y)

Next, let’s perform the prediction and see how many were cells were incorrectly predicted:

pred = model.predict_classes(X)
print(np.count_nonzero(pred.flatten() - y.flatten()), "incorrect cells.")

4 incorrect cells.

Next, let’s compare the correct next frame versus the predicted frame:

render_frames(y, pred.flatten().reshape(board_shape))

It’s not terrible, but can you see where it failed? It appears to be failing to predict the cells around the edges of the game board. Take a look at the following where non-zero values indicate incorrect predictions:

print(pred.flatten().reshape(board_shape) - y.flatten().reshape(board_shape))

[[ 0  0  0  0  0  0  0 -1  0  0  0  0  0  0  0  0  0 -1 -1  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  0  0 -1  0  0  0  0  0  0  0  0  0  0  0  0  0]]

As you can see, all of the non-zero values are located on the edges of the game board. Let’s take a look at the full testing set and see if that observation holds true.

View Frequent Errors using Test Set

We’ll write a function that renders a heat map depicting where the model is making errors, and invoke it using the entire testing set:

def view_prediction_errors(model, X, y):
    y_pred = model.predict_classes(X)
    sum_y_pred = np.sum(y_pred, axis=0).flatten().reshape(board_shape)
    sum_y = np.sum(y, axis=0).flatten().reshape(board_shape)

    plt.imshow(sum_y_pred - sum_y, cmap='hot', interpolation='nearest')
    plt.show()

view_prediction_errors(model, X_test, y_test)

All of the errors are around the edges, and the model performs the worst in the corners. This makes sense since the CNN cannot look around the edges, but the Game of Life logic in life_step does. For example, consider the following. Currently, when looking at the edge cell x below, the CNN only sees the x and the ! cells:

But what we really want, and what life_step is doing, is to look at the cells on the opposite side as well:

The same logic applies to corner cells, like so:

To fix this, the padding property of the Conv2D needs to somehow look at the opposite side of the game board. Alternatively, each input board could be pre-processed to pad the edges with the opposite side, and then Conv2D can simply drop the first/last column and row. Since we’re at the mercy of Keras and the padding functionality it provides, which don’t support what we’re looking for, we’ll have to resort to adding our own padding.

Fixing the Edge Issue with Custom Padding

We need to pad each game board with the opposite value to simulate how life_step looks across the game board for edge values. We can use np.pad with mode=’wrap’ to accomplish this. For instance, consider the following array and the padded output below:

x = np.array([
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
])

print(np.pad(x, (1, 1), mode='wrap'))

[[9, 7, 8, 9, 7],
 [3, 1, 2, 3, 1],
 [6, 4, 5, 6, 4],
 [9, 7, 8, 9, 7],
 [3, 1, 2, 3, 1]]

Notice now how the first column/row and the last column/row are mirrors of the opposite side of the original matrix, and the middle 3x3 matrix is the original value x. For example, cell [1][1] has been replicated on the far side in cell [4][1], and likewise [0][1] contains [3][1]. In every direction, and even in the corners, the array has been wrapped to contain its opposite side, just how we want. This will allow the CNN to consider the wrapped game board and properly handle the edge cases (pun intended).

Now we can write a function to pad all our input matrices:

def pad_input(X):
    return reshape_input(np.array([
        np.pad(x.reshape(board_shape), (1,1), mode='wrap')
        for x in X
    ]))

X_train_padded = pad_input(X_train)
X_val_padded = pad_input(X_val)
X_test_padded = pad_input(X_test)

print(X_train_padded.shape)
print(X_val_padded.shape)
print(X_test_padded.shape)

(70000, 22, 22, 1)
(10000, 22, 22, 1)
(20000, 22, 22, 1)

All the datasets are now padded with their wrapped columns/rows, allowing the CNN to look around the opposite side of the game board just like life_step does. Because of this, each game board is now 22x22 instead of the original 20x20.

Next, the CNN must be reconstructed to drop the padding values using padding=’valid’ (which tells Conv2D to drop the edges, although thats not immediately obvious), and updated to handle the new input_shape. This way when we pass in the padded 22x22 game boards we still get 20x20 game boards as output, since we’ll drop the first and last columns/rows. The rest remains identical:

model_padded = Sequential()
model_padded.add(Conv2D(
    filters, 
    kernel_size,
    padding='valid',
    activation='relu',
    strides=strides,
    input_shape=(board_shape[0] + 2, board_shape[1] + 2, 1)
))
model_padded.add(Dense(hidden_dims))
model_padded.add(Dense(1))
model_padded.add(Activation('sigmoid'))

model_padded.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model_padded.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_10 (Conv2D)           (None, 20, 20, 50)        500       
_________________________________________________________________
dense_19 (Dense)             (None, 20, 20, 100)       5100      
_________________________________________________________________
dense_20 (Dense)             (None, 20, 20, 1)         101       
_________________________________________________________________
activation_10 (Activation)   (None, 20, 20, 1)         0         
=================================================================
Total params: 5,701
Trainable params: 5,701
Non-trainable params: 0
_________________________________________________________________

Now we can train using the padded inputs:

train(
    model_padded, 
    X_train_padded, y_train, X_val_padded, y_val, 
    filename_suffix='_padded'
)

Train on 70000 samples, validate on 10000 samples
Epoch 1/2
70000/70000 [==============================] - 27s 389us/step - loss: 0.0604 - acc: 0.9807 - val_loss: 4.5475e-04 - val_acc: 1.0000
Epoch 2/2
70000/70000 [==============================] - 27s 382us/step - loss: 1.7058e-04 - acc: 1.0000 - val_loss: 5.9932e-05 - val_acc: 1.0000

Validation and training accuracy are up to 100% from the roughly 98% we got before adding the padding. Let’s see the test error:

view_prediction_errors(model_padded, X_test_padded, y_test)

Perfect! The black heat map indicates that there is no variance in the values, meaning we successfully predicted every cell, for every game.

And that’s all there is to it. This was a fun little exercise to play with convolutional neural networks, without requiring a big data set. Feel free to check it out at github.com/KyleBanks/conways-gol-cnn.

Introducing modoc, a Lightweight Framework for Large Markdown Documents

Sat, 26 May 2018 00:00:00 +0100

I thoroughly enjoy writing and generally when I write, I do so using Markdown for formatting. Markdown is great because it can be easily exported to HTML, a PDF, Word Documents, etc. There’s great tooling around Markdown, abd it’s a well supported markup language.

My only problem is when I’m writing larger documents, such as if you were to write book, you quickly end up with a massive, unwieldly document. A few problems that come to mind:

If you want to go back and reorganize sections or chapters of your document it’s usually pretty difficult as you have to recreate your headings as the heading levels won’t match up.
How do you go about managing a table of contents? You could manually manage it, but updating the links and text, and maintaining order and depth of each heading is a pain.

What I wanted was a small, lightweight framework that can be used to organize a larger document into smaller, more manageable chunks, but still output to a single Markdown file complete with table of contents. That’s where modoc comes in.

With modoc you simply create a directory structure where each directory has a TITLE and README.md file, and modoc does all the rest. Here’s what the directory structure for the README document of the opensource modoc repository looks like:

$ tree examples/readme/
examples/readme/
├── 1-features
│   ├── README.md
│   └── TITLE
├── 2-usage
│   ├── 1-install
│   │   ├── README.md
│   │   └── TITLE
│   ├── 2-command-line
│   │   ├── 1-init
│   │   │   ├── README.md
│   │   │   └── TITLE
│   │   ├── 2-compile
│   │   │   ├── README.md
│   │   │   └── TITLE
│   │   ├── README.md
│   │   └── TITLE
│   ├── 3-project-structure
│   │   ├── README.md
│   │   └── TITLE
│   ├── 4-examples
│   │   ├── README.md
│   │   └── TITLE
│   └── TITLE
├── 3-authors
│   ├── 1-contributing
│   │   ├── README.md
│   │   └── TITLE
│   ├── README.md
│   └── TITLE
├── 4-license
│   ├── README.md
│   └── TITLE
├── README.md
└── TITLE

11 directories, 23 files

Directories are loaded into modoc recursively and in alphabetical order, and the titles, sections, and table of contents are created from there. In the example above, you end up with a document that looks like so:

# Features
...
# Usage
...
## Install
...
## Command-Line
...
### init
...
### compile
...

(Take a look at the full document at github.com/KyleBanks/modoc.)

modoc also generates the table of contents at the top of the document, complete with links to each section, so you don’t have to worry about it:

## Table of Contents

- [Features](#features)
- [Usage](#usage)
   - [Install](#install)
   - [Command Line](#command-line)
      - [init](#init)
      - [compile](#compile)
   - [Project Structure](#project-structure)
   - [Examples](#examples)
- [Authors](#authors)
   - [Contributing](#contributing)
- [License](#license)

This makes it really easy to structure and organize larger writing projects without having a full-blown framework to get in your way. Check it out on GitHub at github.com/KyleBanks/modoc and try it for yourself. If you have any ideas or run into trouble, don’t hesitate to create an issue!

Transfer Learning and Retraining Inception/MobileNet with TensorFlow and Docker

Sat, 09 Dec 2017 00:00:00 +0000

TensorFlow is a modern machine learning framework that provides tremendous power and opportunity to developers and data scientists. One of those opportunities is to use the concept of Transfer Learning to reduce training time and complexity by repurposing a pre-trained model.

From the official docs:

Modern object recognition models have millions of parameters and can take weeks to fully train. Transfer learning is a technique that shortcuts a lot of this work by taking a fully-trained model for a set of categories like ImageNet, and retrains from the existing weights for new classes. In this example we’ll be retraining the final layer from scratch, while leaving all the others untouched. For more information on the approach you can see this paper on Decaf.

Though it’s not as good as a full training run, this is surprisingly effective for many applications, and can be run in as little as thirty minutes on a laptop, without requiring a GPU.

In this tutorial we’ll learn how to utilize Transfer Learning to repurpose a pre-trained Inception or MobileNet model provided by TensorFlow to serve a new purpose.

What’s unique about this tutorial however, is that we’ll do it all without installing TensorFlow, instead performing training and predictions entirely through Docker. Why? Because it simplifies the process of getting this working on multiple machines, whether that is multiple developers and data scientists, your continuous integration process, or your production pipelines. Performing training in Docker ensures that no matter what machine is used to train your models, it will work without additional setup.

If you want to skip ahead and just get to work with the pre-built Docker images, head over to KyleBanks/tensorflow-docker-retrain on GitHub and follow the quick instructions there.

Gathering Images

The first (and honestly, most challenging) step of any image classification problem is to gather the image dataset. For this tutorial we’ll reuse the MNIST database of handwritten digits (0-9), which contains 60,000 training examples and 10,000 testing examples. This saves considerable time in gathering and preparing a custom dataset, and lets us get right into the real work of training and predicting.

In order to further reduce the amount of work we need to do to get started, I’ve converted the dataset into JPG format, which you can download and unzip from here: mnist-jpegs.zip. After downloading and unzipping you should have a directory structure like so:

test-images/
    0/
    1/
    2/
    3/
    4/
    5/
    6/
    7/
    8/  
    9/ 
train-images/
    0/
    1/
    2/
    3/
    4/
    5/
    6/
    7/
    8/  
    9/ 

Within these directories you’ll find thousands of 28x28 images that look like so:

The way TensorFlow expects images to be laid out is exactly as seen in the directory structure above: one folder per label, each containing the appropriate images. In this case our labels are the digits 0 through 9, so we have a folder for each. The test-images and train-images folders are going to be used by us to separate our training images from the images we’ll test on, and are not important to TensorFlow necessarily.

If you have your own image dataset that you want to work with, please feel free to do so! You’ll simply need to structure it like you see above, one folder per label where the folder name is the label name. For instance, if we were classifying images of comic book characters we might have a directory structure of:

iron-man/
captain-america/
green-lantern/
batman/

Training

Now that we have our dataset, it’s time to get to work. We’ll start by creating a Docker image to handle training: train-classifier/Dockerfile. It’s important to put this in the train-classifier directory as we’ll be creating multiple Docker images for different purposes by the end of the tutorial.

FROM gcr.io/tensorflow/tensorflow:1.4.0

ENV IMAGE_SIZE 128
ENV OUTPUT_GRAPH tf_files/retrained_graph.pb
ENV OUTPUT_LABELS tf_files/retrained_labels.txt
ENV ARCHITECTURE mobilenet_0.50_${IMAGE_SIZE}
ENV TRAINING_STEPS 1000

VOLUME /output
VOLUME /input

RUN curl -O https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/image_retraining/retrain.py

ENTRYPOINT python -m retrain \
  --how_many_training_steps="${TRAINING_STEPS}" \
  --model_dir=/output/tf_files/models/ \
  --output_graph=/output/"${OUTPUT_GRAPH}" \
  --output_labels=/output/"${OUTPUT_LABELS}" \
  --architecture="${ARCHITECTURE}" \
  --image_dir=/input

Let’s walk through this from top-to-bottom.

First, we use the official TensorFlow v1.4.0 image (the latest at time of writing) as our base, which comes with TensorFlow preinstalled and configured for us, one of the many benefits of Docker.

Next we define the default training parameters as environment variables which will allow us to eventually override them at runtime as necessary. We set our image size to 128 pixels even though our images are 28x28, because that’s the minimum size allowed, meaning our images will be upscaled, which is fine because they are all being upscaled the same way. Next we define the output location of the trained graph and the labels text file that will be generated by the training script. For the ARCHITECTURE you can see we’re using MobileNet with a size of 0.50 and the image size as the suffix. I’m using MobileNet here in order to reduce training time and size of the trained model, but it does sacrifice some performance. If you wish to use Inception you can set the value of ARCHITECTURE to inception_v3. For more on the architecture and what these values all mean, I recommend you read Other Model Architectures from the official documentation. Finally we indicate 1000 training steps, which allows us to train quickly without doing a huge training cycle (the default is 4000), at the potential cost of performance. Generally once you have a good idea that your model will perform well you’ll want to play with increasing the training steps and training for longer to see if the improved performance is worth the increased training time.

Next we mount two volumes, /output and /input. Volumes allow us to share data between the host machine and the Docker container. /output will be used to retrieve the trained model and labels from the Docker container once training is complete, and /input will be used to provide the Docker container with our training images. We’ll come back to these shortly.

Now it’s time to retrieve the retraining script, which we do by downloading it from the official TensorFlow repository on GitHub. We could write our own, but it’s definitely easier to start out with the existing training script until you need to do something custom.

Finally, we define the entrypoint of the Docker container to be the retraining script invoked with our parameters. As you can see we are instructing the retraining script on how many training steps to perform, the output locations, the architecture to retrain, and the location of our training image set.

With the Dockerfile created, we can go ahead and build the image:

$ docker build -t train-classifier train-classifier/

With the image built, it’s time to invoke it. Since the environment variables are all setup to match what we intend, all that we need to do is specify the location of the two volumes: /output and /input:

$ docker run -it \  
    -v $(pwd)/output:/output \
    -v $(pwd)/train-images:/input
    train-classifier

Here we’ve set /output equal to a new output directory in the present working directory, and /input as the path to the train-images/ directory containing the MNIST training image dataset.

If all goes well, you should eventually see output similar to:

...
INFO:tensorflow: Step 980: Train accuracy = 96.0%
INFO:tensorflow: Step 980: Cross entropy = 0.122227
INFO:tensorflow: Step 980: Validation accuracy = 92.0% (N=100)
INFO:tensorflow: Step 990: Train accuracy = 97.0%
INFO:tensorflow: Step 990: Cross entropy = 0.164350
INFO:tensorflow: Validation accuracy = 93.0% (N=100)
INFO:tensorflow: Step 999: Train accuracy = 95.0%
INFO:tensorflow: Step 999: Cross entropy = 0.159146
INFO:tensorflow: Step 999: Validation accuracy = 95.0% (N=100)
INFO:tensorflow:Final test accuracy = 94.1% (N=6060)

Note: It may take 10-20 minutes to perform training, hence why we chose to use time-friendly variables in the Dockerfile environment configuration above.

Using the Retrained Model for Predictions

Now that the model is trained it’s ready to perform some predictions! Again we’ll create a Docker image to bundle the prediction logic which will be invoked against a folder containing images to predict - our test-images/ folder from the MNIST dataset in this case. Since these images were not used for training they will serve as a valid image set to test with.

To start with let’s create a predictor/ folder beside the train-classifier/ folder, as these will be our two Docker images. Inside predictor/ let’s add a new Dockerfile:

FROM gcr.io/tensorflow/tensorflow:1.4.0

ENV IMAGE_SIZE 128
ENV IMAGE_MEAN 0
ENV IMAGE_STD 255
ENV GRAPH_FILE retrained_graph.pb
ENV LABELS_FILE retrained_labels.txt
ENV INPUT_LAYER input
ENV OUTPUT_LAYER final_result

VOLUME /model
VOLUME /input

RUN curl -O https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/label_image/label_image.py
ADD predict.sh .

ENTRYPOINT ["/bin/bash", "predict.sh"]

Again, let’s walk through this line by line. We start off the same as before by using the standard TensorFlow Docker image as our base, specifying version 1.4.0, the latest at the time of writing.

Next up we’re going to define the customizable runtime properties as environment variables, using defaults appropriate for this task. You’ll see we use the same image size as in training, and the same graph and label file names that our train-classifier container created for us. What’s changed though is we now have a few new variables, namely IMAGE_MEAN, IMAGE_STD, INPUT_LAYER, and OUTPUT_LAYER. The image mean and std values specified simply match the defaults in the TensorFlow prediction script but allow you to tune them as needed, and the input and output layer names match the layer names in our trained model.

Next we’ll mount two volumes just like we did in training, but this time we have /model which maps to the folder containing the trained model and label files, and /input which maps to the folder containing images to predict.

Next up we download the label_image.py script from the official TensorFlow repository and add our own predict.sh script that we’ll see in a moment.

Finally we define the entrypoint of the container to be the predict.sh script below:

#!/bin/bash
echo "-------------------------------------------------"
echo -e "\n\n"
env
echo -e "\n\n"
echo "-------------------------------------------------"
echo -e "\n\nPredicting $@\n\n"
echo "-------------------------------------------------"

python -m label_image \
    --input_width="$IMAGE_SIZE" \
    --input_height="$IMAGE_SIZE" \
    --input_mean="$IMAGE_MEAN" \
    --input_std="$IMAGE_STD" \
    --input_layer="$INPUT_LAYER" \
    --output_layer="$OUTPUT_LAYER" \
    --graph=/model/"$GRAPH_FILE" \
    --labels=/model/"$LABELS_FILE" \
    --image=/input/$@

This bash script simply prints the environment variables and the name of the image we’re predicting, and then calls the label_image script with the set configurations. This script is mostly just helpful to see what parameters you’re using and the image being predicted, and you can modify it as you see fit. Alternatively you could place the python script execution in the Dockerfile just like we did in the train-classifier version - it’s all up to you.

Alright, let’s build the predictor image:

$ docker build -t predictor predictor/

In a moment you should have a built predictor image that we can use to perform predictions. In order to predict, we’ll invoke the predictor by mounting the two required volumes, where /model is the path to our output/tf_files directory and /input is the path to the test images folder containing the image to predict, and finally the name of the image to predict.

For example, let’s predict test-images/5/1376.jpg:

$ docker run -it \
    -v $(pwd)/output/tf_files:/model \
    -v $(pwd)/test-images/5:/input \
    predictor 1376.jpg

If all goes well you should get a response like so:

-------------------------------------------------


Predicting 1376.jpg


-------------------------------------------------
5 0.953844
3 0.0283552
8 0.0168694
0 0.000464269
6 0.000233758

What this tells us is that the label 5 was predicted with 95% confidence, and in fact the predictor was correct. Go ahead and try it out on several images to see the results!

Conclusion

Transfer Learning in Docker turns out to be rather straightforward (assuming you have some Docker experience), and greatly simplifies the setup process. Installing TensorFlow can be a painful process at times, so having to do so on you and your teams’ machines, as well as your production and staging environments is less than ideal. Docker simplifies the process and allows you to reuse the same Docker image on as many machines as you like, with no additional setup.

Check out the project on GitHub at KyleBanks/tensorflow-docker-retrain and let me know what you think!

OpenGL & Go Tutorial Part 3: Implementing the Game

Sun, 12 Mar 2017 00:00:00 +0000

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Welcome back to the OpenGL & Go Tutorial! If you haven’t gone through Part 1 and Part 2 you’ll definitely want to take a step back and check them out.

At this point you should have a grid system created and a matrix of cells to represent each unit of the grid. Now it’s time to implement Conway’s Game of Life using the grid as the game board.

Let’s get started!

Implement Conway’s Game

One of the keys to Conway’s game is that each cell must determine its next state based on the current state of the board, at the same time. This means that if Cell (X=3, Y=4) changes state during its calculation, its neighbor at (X=4, Y=4) must determine its own state based on what (X=3, Y=4) was, not what is has become. Basically, this means we must loop through the cells and determine their next state without modifying their current state before we draw, and then on the next loop of the game we apply the new state and repeat.

In order to accomplish this, we’ll add two booleans to the cell struct:

type cell struct {
    drawable uint32

    alive     bool
    aliveNext bool

    x int
    y int
}

Here we’ve added alive and aliveNext, the former being the cell’s current state and the later being the state it has calculated for the next turn.

Now let’s add two functions that we’ll use to determine the cell’s state:

// checkState determines the state of the cell for the next tick of the game.
func (c *cell) checkState(cells [][]*cell) {
    c.alive = c.aliveNext
    c.aliveNext = c.alive
    
    liveCount := c.liveNeighbors(cells)
    if c.alive {
        // 1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
        if liveCount < 2 {
            c.aliveNext = false
        }
        
        // 2. Any live cell with two or three live neighbours lives on to the next generation.
        if liveCount == 2 || liveCount == 3 {
            c.aliveNext = true
        }
        
        // 3. Any live cell with more than three live neighbours dies, as if by overpopulation.
        if liveCount > 3 {
            c.aliveNext = false
        }
    } else {
        // 4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
        if liveCount == 3 {
            c.aliveNext = true
        }
    }
}

// liveNeighbors returns the number of live neighbors for a cell.
func (c *cell) liveNeighbors(cells [][]*cell) int {
    var liveCount int
    add := func(x, y int) {
        // If we're at an edge, check the other side of the board.
        if x == len(cells) {
            x = 0
        } else if x == -1 {
            x = len(cells) - 1
        }
        if y == len(cells[x]) {
            y = 0
        } else if y == -1 {
            y = len(cells[x]) - 1
        }
        
        if cells[x][y].alive {
            liveCount++
        }
    }
    
    add(c.x-1, c.y)   // To the left
    add(c.x+1, c.y)   // To the right
    add(c.x, c.y+1)   // up
    add(c.x, c.y-1)   // down
    add(c.x-1, c.y+1) // top-left
    add(c.x+1, c.y+1) // top-right
    add(c.x-1, c.y-1) // bottom-left
    add(c.x+1, c.y-1) // bottom-right
    
    return liveCount
}

In checkState we set the current state (alive) equal to what we calculated on the last iteration (aliveNext). Next we count the number of neighbors, and determine the aliveNext state according to the rules of the game. The rules should be relatively clear and they’re documented in the code above, so I won’t go over them again here.

What’s more interesting is the liveNeighbors function where we return the number of neighbors to the current cell that are in an alive state. We define an inner function called add that will do some repetitive validation on X and Y coordinates. What it does is check if we’ve passed a number that exceeds the bounds of the board - for example, if cell (X=0, Y=5) wants to check on its neighbor to the left, it has to wrap around to the other side of the board to cell (X=9, Y=5), and likewise for the Y-axis.

Below the inner add function we call add with each of the cell’s eight neighbors, depicted below:

[
    [-, -, -],
    [N, N, N],
    [N, C, N],
    [N, N, N],
    [-, -, -]
]

In this depiction, each cell labeled N is a neighbor to C.

Now in our main function, where we have our core game loop, let’s call checkState on each cell prior to drawing:

func main() {
    ...
    
    for !window.ShouldClose() {
        for x := range cells {
            for _, c := range cells[x] {
                c.checkState(cells)
            }
        }

        draw(cells, window, program)
    }
}

Now our core game logic is all set, we just need to modify the cell draw function to skip drawing if the cell isn’t alive:

func (c *cell) draw() {
    if !c.alive {
            return
    }

    gl.BindVertexArray(c.drawable)
    gl.DrawArrays(gl.TRIANGLES, 0, int32(len(square)/3))
}

If you run the game now you’ll see a plain black screen, not the glorious simulation we’ve been working so hard to achieve. So why’s that? Well its because the simulation is working! None of our cells are alive, so none of them are drawing themselves.

Let’s fix that. Back in makeCells we’ll use a random number between 0.0 and 1.0 to set the initial state of the game. We’ll define a constant threshold of 0.15 meaning that each cell has a 15% chance of starting in an alive state:

import (
    "math/rand"
    "time"
    ...
)

const (
    ...
    
    threshold = 0.15
)


func makeCells() [][]*cell {
    rand.Seed(time.Now().UnixNano())

    cells := make([][]*cell, rows, rows)
    for x := 0; x < rows; x++ {
        for y := 0; y < columns; y++ {
            c := newCell(x, y)
            
            c.alive = rand.Float64() < threshold
            c.aliveNext = c.alive
            
            cells[x] = append(cells[x], c)
        }
    }

	return cells
}

We start by adding two imports for the rand (random) and time packages, and defining our threshold constant. Then in makeCells we use the current time as the randomization seed, giving each game a unique starting state. You can also specify a particular seed value to always get an identical game, useful if you want to replay interesting simulations.

Next in the loop, after creating a cell with the newCell function we set its alive state equal to the result of a random float, between 0.0 and 1.0, being less than threshold (0.15). Again, this means each cell has a 15% chance of starting out alive. You can play with this number to increase or decrease the number of living cells at the outset of the game. We also set aliveNext equal to alive, otherwise we’ll get a massive die-off on the first iteration because aliveNext will always be false!

Now go ahead and give it a run, and you’ll likely see a quick flash of cells that you can’t make heads or tails of. The reason is that your computer is probably way too fast and is running through (or even finishing) the simulation before you have a chance to really see it.

Let’s reduce the game speed by introducing a frames-per-second limitation in the main loop:

const (
    ...
    
    fps = 2
)

func main() {
    ...
    
    for !window.ShouldClose() {
        t := time.Now()

        for x := range cells {
            for _, c := range cells[x] {
                c.checkState(cells)
            }
        }
        
        if err := draw(prog, window, cells); err != nil {
            panic(err)
        }
        
        time.Sleep(time.Second/time.Duration(fps) - time.Since(t))
    }
}

At the top we define our fps equal to 2, meaning we will perform 2 game iterations per second. We get a reference to the current time at the beginning of the loop, and at the end we sleep for one second divided by our FPS, minus the time that elapsed during the frame. For instance, at an FPS of 2 this means we want each loop to take 500 milliseconds. So, we divide one second by our FPS 2, giving us 500 milliseconds, but we also subtract the time that was spent processing the frame using time.Since(t) where t is the time we recorded at the beginning.

Now you should be able to see some patterns, albeit very slowly. Increase the FPS to 10 and the size of the grid to 100x100 and you should see some really cool simulations:

const (
    ...
    
    rows = 100
    columns = 100
    
    fps = 10
    
    ...
)

Running now you should be able to see something along the lines of this:

Try playing with the constants to see how they impact the simulation - cool right? Your very first OpenGL application with Go!

What’s Next?

This concludes the OpenGL with Go Tutorial, but that doesn’t mean you should stop now. Here’s a few challenges to further improve your OpenGL (and Go) knowledge:

Give each cell a unique color.
Allow the user to specify, via command-line arguments, the grid size, frame rate, seed and threshold. You can see this one implemented on GitHub at github.com/KyleBanks/conways-gol.
Change the shape of the cells into something more interesting, like a hexagon.
Use color to indicate the cell’s state - for example, make cells green on the first frame that they’re alive, and make them yellow if they’ve been alive more than three frames.
Automatically close the window if the simulation completes, meaning all cells are dead or no cells have changed state in the last two frames.
Move the shader source code out into their own files, rather than having them as string constants in the Go source code.

Summary

Hopefully this tutorial has been helpful in gaining a foundation on OpenGL (and maybe even Go)! It was a lot of fun to make so I can only hope it was fun to go through and learn.

As I’ve mentioned, OpenGL can be very intimidating, but it’s really not so bad once you get started. You just want to break down your goals into small, achievable steps, and enjoy each victory because while OpenGL isn’t always as tough as it looks, it can certainly be very unforgiving. One thing that I have found helpful when stuck on OpenGL issues was to understand that the way go-gl is generated means you can always use C code as a reference, which is much more popular in tutorials around the internet. The only difference usually between the C and Go code is that functions in Go are prefixed with gl. instead of gl, and constants are prefixed with gl instead of GL_. This vastly increases the pool of knowledge you have to draw from!

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Checkpoint

Here’s the final contents of main.go:

package main

import (
	"fmt"
	"log"
	"math/rand"
	"runtime"
	"strings"
	"time"

	"github.com/go-gl/gl/v4.1-core/gl" // OR: github.com/go-gl/gl/v2.1/gl
	"github.com/go-gl/glfw/v3.2/glfw"
)

const (
	width  = 500
	height = 500

	vertexShaderSource = `
		#version 410
		in vec3 vp;
		void main() {
			gl_Position = vec4(vp, 1.0);
		}
	` + "\x00"

	fragmentShaderSource = `
		#version 410
		out vec4 frag_colour;
		void main() {
			frag_colour = vec4(1, 1, 1, 1.0);
		}
	` + "\x00"

	rows    = 100
	columns = 100

	threshold = 0.15
	fps       = 10
)

var (
	square = []float32{
		-0.5, 0.5, 0,
		-0.5, -0.5, 0,
		0.5, -0.5, 0,

		-0.5, 0.5, 0,
		0.5, 0.5, 0,
		0.5, -0.5, 0,
	}
)

type cell struct {
	drawable uint32

	alive     bool
	aliveNext bool

	x int
	y int
}

func main() {
	runtime.LockOSThread()

	window := initGlfw()
	defer glfw.Terminate()
	program := initOpenGL()

	cells := makeCells()
	for !window.ShouldClose() {
		t := time.Now()

		for x := range cells {
			for _, c := range cells[x] {
				c.checkState(cells)
			}
		}

		draw(cells, window, program)

		time.Sleep(time.Second/time.Duration(fps) - time.Since(t))
	}
}

func draw(cells [][]*cell, window *glfw.Window, program uint32) {
	gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
	gl.UseProgram(program)

	for x := range cells {
		for _, c := range cells[x] {
			c.draw()
		}
	}

	glfw.PollEvents()
	window.SwapBuffers()
}

func makeCells() [][]*cell {
	rand.Seed(time.Now().UnixNano())

	cells := make([][]*cell, rows, rows)
	for x := 0; x < rows; x++ {
		for y := 0; y < columns; y++ {
			c := newCell(x, y)

			c.alive = rand.Float64() < threshold
			c.aliveNext = c.alive

			cells[x] = append(cells[x], c)
		}
	}

	return cells
}
func newCell(x, y int) *cell {
	points := make([]float32, len(square), len(square))
	copy(points, square)

	for i := 0; i < len(points); i++ {
		var position float32
		var size float32
		switch i % 3 {
		case 0:
			size = 1.0 / float32(columns)
			position = float32(x) * size
		case 1:
			size = 1.0 / float32(rows)
			position = float32(y) * size
		default:
			continue
		}

		if points[i] < 0 {
			points[i] = (position * 2) - 1
		} else {
			points[i] = ((position + size) * 2) - 1
		}
	}

	return &cell{
		drawable: makeVao(points),

		x: x,
		y: y,
	}
}

func (c *cell) draw() {
	if !c.alive {
		return
	}

	gl.BindVertexArray(c.drawable)
	gl.DrawArrays(gl.TRIANGLES, 0, int32(len(square)/3))
}

// checkState determines the state of the cell for the next tick of the game.
func (c *cell) checkState(cells [][]*cell) {
	c.alive = c.aliveNext
	c.aliveNext = c.alive

	liveCount := c.liveNeighbors(cells)
	if c.alive {
		// 1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
		if liveCount < 2 {
			c.aliveNext = false
		}

		// 2. Any live cell with two or three live neighbours lives on to the next generation.
		if liveCount == 2 || liveCount == 3 {
			c.aliveNext = true
		}

		// 3. Any live cell with more than three live neighbours dies, as if by overpopulation.
		if liveCount > 3 {
			c.aliveNext = false
		}
	} else {
		// 4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
		if liveCount == 3 {
			c.aliveNext = true
		}
	}
}

// liveNeighbors returns the number of live neighbors for a cell.
func (c *cell) liveNeighbors(cells [][]*cell) int {
	var liveCount int
	add := func(x, y int) {
		// If we're at an edge, check the other side of the board.
		if x == len(cells) {
			x = 0
		} else if x == -1 {
			x = len(cells) - 1
		}
		if y == len(cells[x]) {
			y = 0
		} else if y == -1 {
			y = len(cells[x]) - 1
		}

		if cells[x][y].alive {
			liveCount++
		}
	}

	add(c.x-1, c.y)   // To the left
	add(c.x+1, c.y)   // To the right
	add(c.x, c.y+1)   // up
	add(c.x, c.y-1)   // down
	add(c.x-1, c.y+1) // top-left
	add(c.x+1, c.y+1) // top-right
	add(c.x-1, c.y-1) // bottom-left
	add(c.x+1, c.y-1) // bottom-right

	return liveCount
}

// initGlfw initializes glfw and returns a Window to use.
func initGlfw() *glfw.Window {
	if err := glfw.Init(); err != nil {
		panic(err)
	}
	glfw.WindowHint(glfw.Resizable, glfw.False)
	glfw.WindowHint(glfw.ContextVersionMajor, 4)
	glfw.WindowHint(glfw.ContextVersionMinor, 1)
	glfw.WindowHint(glfw.OpenGLProfile, glfw.OpenGLCoreProfile)
	glfw.WindowHint(glfw.OpenGLForwardCompatible, glfw.True)

	window, err := glfw.CreateWindow(width, height, "Conway's Game of Life", nil, nil)
	if err != nil {
		panic(err)
	}
	window.MakeContextCurrent()

	return window
}

// initOpenGL initializes OpenGL and returns an intiialized program.
func initOpenGL() uint32 {
	if err := gl.Init(); err != nil {
		panic(err)
	}
	version := gl.GoStr(gl.GetString(gl.VERSION))
	log.Println("OpenGL version", version)

	vertexShader, err := compileShader(vertexShaderSource, gl.VERTEX_SHADER)
	if err != nil {
		panic(err)
	}

	fragmentShader, err := compileShader(fragmentShaderSource, gl.FRAGMENT_SHADER)
	if err != nil {
		panic(err)
	}

	prog := gl.CreateProgram()
	gl.AttachShader(prog, vertexShader)
	gl.AttachShader(prog, fragmentShader)
	gl.LinkProgram(prog)
	return prog
}

// makeVao initializes and returns a vertex array from the points provided.
func makeVao(points []float32) uint32 {
	var vbo uint32
	gl.GenBuffers(1, &vbo)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.BufferData(gl.ARRAY_BUFFER, 4*len(points), gl.Ptr(points), gl.STATIC_DRAW)

	var vao uint32
	gl.GenVertexArrays(1, &vao)
	gl.BindVertexArray(vao)
	gl.EnableVertexAttribArray(0)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.VertexAttribPointer(0, 3, gl.FLOAT, false, 0, nil)

	return vao
}

func compileShader(source string, shaderType uint32) (uint32, error) {
	shader := gl.CreateShader(shaderType)

	csources, free := gl.Strs(source)
	gl.ShaderSource(shader, 1, csources, nil)
	free()
	gl.CompileShader(shader)

	var status int32
	gl.GetShaderiv(shader, gl.COMPILE_STATUS, &status)
	if status == gl.FALSE {
		var logLength int32
		gl.GetShaderiv(shader, gl.INFO_LOG_LENGTH, &logLength)

		log := strings.Repeat("\x00", int(logLength+1))
		gl.GetShaderInfoLog(shader, logLength, nil, gl.Str(log))

		return 0, fmt.Errorf("failed to compile %v: %v", source, log)
	}

	return shader, nil
}

OpenGL & Go Tutorial Part 2: Drawing the Game Board

Sun, 12 Mar 2017 00:00:00 +0000

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Welcome back to the OpenGL & Go Tutorial! If you haven’t gone through Part 1 you’ll definitely want to take a step back and check it out.

At this point you should be the proud creator of a magnificent white triangle, but we’re not in the business of using triangles as our game unit so it’s time to turn the triangle into a square, and then we’ll make an entire grid of them.

Let’s get started!

Make a Square out of Triangles

Before we can make a square, let’s turn our triangle into a right-angle. Open up main.go and change the triangle definition to look like so:

triangle = []float32{
    -0.5, 0.5, 0,
    -0.5, -0.5, 0,
    0.5, -0.5, 0,
}

What we’ve done is move the X-coordinate of the top vertex to the left (-0.5), giving us a triangle like so:

Easy enough, right? Now let’s make a square out of two of these. Let’s rename triangle to square and add a second, inverted right-angle triangle to the slice:

square = []float32{
    -0.5, 0.5, 0,
    -0.5, -0.5, 0,
    0.5, -0.5, 0,

    -0.5, 0.5, 0,
    0.5, 0.5, 0,
    0.5, -0.5, 0,
}

Note: You’ll also need to rename the two references to triangle to be square, namely in main and draw.

Here we’ve doubled the number of points by adding a second set of three vertices to be our upper top-right triangle to complete the square. Run it for glory:

Great, now we have the ability to draw a square! OpenGL isn’t so tough after all, is it?

Draw a Grid of Squares covering the Window

Now that we can draw one square, how about 100 of them? Let’s create a cell struct to represent each unit of our grid so that we can be flexible in the number of squares we draw:

type cell struct {
    drawable uint32
    
    x int
    y int
}

The cell contains a drawable which is a square Vertex Array Object just like the one we created above, and an X and Y coordinate to dictate where on the grid this cell resides.

We’re also going to want two more constants that define the size and shape of our grid:

const (
    ...
    
    rows = 10
    columns = 10
)

Now let’s add a function to create the grid:

func makeCells() [][]*cell {
    cells := make([][]*cell, rows, rows)
    for x := 0; x < rows; x++ {
        for y := 0; y < columns; y++ {
            c := newCell(x, y)
            cells[x] = append(cells[x], c)
        }
    }
    
    return cells
}

Here we create a multi-dimensional slice to represent our game’s board, and populate each element of the matrix with a cell using a new function called newCell which we’ll write in just a moment.

Before moving on, let’s take a moment to visualize what makeCells is creating. We’re creating a slice that is equal in length to the number of rows on the grid, and each of these slices contains a slice of cells, equal in length to the number of columns. If we were to define rows and columns each equal to two, we’d create the following matrix:

[
    [cell, cell],
    [cell, cell]
]

We’re creating a much larger matrix that’s 10x10 cells:

[
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell],
    [cell, cell, cell, cell, cell, cell, cell, cell, cell, cell]
]

Now that the we understand the shape and representation of the matrix we’re creating, let’s have a look at newCell which we use to actually populate the matrix:

func newCell(x, y int) *cell {
    points := make([]float32, len(square), len(square))
    copy(points, square)

    for i := 0; i < len(points); i++ {
        var position float32
        var size float32
        switch i % 3 {
        case 0:
                size = 1.0 / float32(columns)
                position = float32(x) * size
        case 1:
                size = 1.0 / float32(rows)
                position = float32(y) * size
        default:
                continue
        }

        if points[i] < 0 {
                points[i] = (position * 2) - 1
        } else {
                points[i] = ((position + size) * 2) - 1
        }
    }

    return &cell{
        drawable: makeVao(points),

        x: x,
        y: y,
    }
}

There’s quite a lot going on in this function so let’s break it down. The first thing we do is create a copy of our square definition. This allows us to change its contents to customize the current cell’s position, without impacting any other cells that are also using the square slice. Next we iterate over the points copy and act based on the current index. We use a modulo operation to determine if we’re at an X (i % 3 == 0) or Y (i % 3 == 1) coordinate of the shape (skipping Z since we’re operating in two dimensions) and determine the size (as a percentage of the entire game board) of the cell accordingly, as well as it’s position based on the X and Y coordinate of the cell on the game board.

Next, we modify the points which currently contain a combination of 0.5, 0 and -0.5 as we defined them in the square slice. If the point is less than zero, we set it equal to the position times 2 (because OpenGL coordinates have a range of 2, between -1 and 1), minus 1 to normalize to OpenGL coordinates. If the position is greater than or equal to zero, we do the same thing but add the size we calculated.

The purpose of this is to set the scale of each cell so that it fills only its percentage of the game board. Since we have 10 rows and 10 columns, each cell will be given 10% of the width and 10% of the height of the game board.

Finally, after all the points have been scaled and positioned, we create a cell with the X and Y coordinate provided, and set the drawable field equal to a Vertex Array Object created from the points slice we just manipulated.

Alright, now in main we can remove our call to makeVao and replace it with a call to makeCells. We’ll also change draw to take the matrix of cells instead of a single vao:

func main() {
    ...
    
    // vao := makeVao(square)
    cells := makeCells()
    
    for !window.ShouldClose() {
        draw(cells, window, program)
    }
}

func draw(cells [][]*cell, window *glfw.Window, program uint32) {
    gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
    gl.UseProgram(program)

    // TODO

    glfw.PollEvents()
    window.SwapBuffers()
}

Now we’ll need each cell to know how to draw itself. Let’s add a draw function to the cell:

func (c *cell) draw() {
    gl.BindVertexArray(c.drawable)
    gl.DrawArrays(gl.TRIANGLES, 0, int32(len(square) / 3))
}

This should look familiar, its nearly identical to how we were drawing the square vao in draw previously, the only difference being we BindVertexArray using c.drawable, which is the cell’s vao we created in newCell.

Back in the main draw function, we can loop over each cell and have it draw itself:

func draw(cells [][]*cell, window *glfw.Window, program uint32) {
    gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
    gl.UseProgram(program)

    for x := range cells {
        for _, c := range cells[x] {
            c.draw()
        }
    }

    glfw.PollEvents()
    window.SwapBuffers()
}

As you can see we loop over each of the cells and call its draw function. If you run the application you should see the following:

Is this what you expected? What we’ve done is create a square for each row and column on the grid, and colored it in, effectively filling the entire game board!

We can see an visualize individual cells by commenting out the for-loop for a moment and doing the following:

// for x := range cells {
//     for _, c := range cells[x] {
//         c.draw()
//     }
// }

cells[2][3].draw()

This draws only the cell located at coordinate (X=2, Y=3). As you can see, each individual cell takes up a small portion of the game board, and is responsible for drawing its own space. We can also see that our game board has its origin, that is the (X=0, Y=0) coordinate, in the bottom-left corner of the window. This is simply a result of the way our newCell function calculates the position, and could be made to use the top-right, bottom-right, top-left, center, or any other position as its origin.

Let’s go ahead and remove the cells[2][3].draw() line and uncomment the for-loop, leaving us with the fully drawn grid we had above.

Summary

Alright - we can now use two triangles to draw a square, and we have ourselves a game board! We should be proud, we’ve covered a lot of ground up to this point and to be completely honest, the hardest part is behind us now!

Next up in Part 3 we’ll implement the core game logic and see some cool simulations!

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Checkpoint

Here’s the contents of main.go at this point of the tutorial:

package main

import (
	"fmt"
	"log"
	"runtime"
	"strings"

	"github.com/go-gl/gl/v4.1-core/gl" // OR: github.com/go-gl/gl/v2.1/gl
	"github.com/go-gl/glfw/v3.2/glfw"
)

const (
	width  = 500
	height = 500

	vertexShaderSource = `
		#version 410
		in vec3 vp;
		void main() {
			gl_Position = vec4(vp, 1.0);
		}
	` + "\x00"

	fragmentShaderSource = `
		#version 410
		out vec4 frag_colour;
		void main() {
			frag_colour = vec4(1, 1, 1, 1.0);
		}
	` + "\x00"

	rows    = 10
	columns = 10
)

var (
	square = []float32{
		-0.5, 0.5, 0,
		-0.5, -0.5, 0,
		0.5, -0.5, 0,

		-0.5, 0.5, 0,
		0.5, 0.5, 0,
		0.5, -0.5, 0,
	}
)

type cell struct {
	drawable uint32

	x int
	y int
}

func main() {
	runtime.LockOSThread()

	window := initGlfw()
	defer glfw.Terminate()
	program := initOpenGL()

	cells := makeCells()
	for !window.ShouldClose() {
		draw(cells, window, program)
	}
}

func draw(cells [][]*cell, window *glfw.Window, program uint32) {
	gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
	gl.UseProgram(program)

	for x := range cells {
		for _, c := range cells[x] {
			c.draw()
		}
	}

	glfw.PollEvents()
	window.SwapBuffers()
}

func makeCells() [][]*cell {
	cells := make([][]*cell, rows, rows)
	for x := 0; x < rows; x++ {
		for y := 0; y < columns; y++ {
			c := newCell(x, y)
			cells[x] = append(cells[x], c)
		}
	}

	return cells
}

func newCell(x, y int) *cell {
	points := make([]float32, len(square), len(square))
	copy(points, square)

	for i := 0; i < len(points); i++ {
		var position float32
		var size float32
		switch i % 3 {
		case 0:
			size = 1.0 / float32(columns)
			position = float32(x) * size
		case 1:
			size = 1.0 / float32(rows)
			position = float32(y) * size
		default:
			continue
		}

		if points[i] < 0 {
			points[i] = (position * 2) - 1
		} else {
			points[i] = ((position + size) * 2) - 1
		}
	}

	return &cell{
		drawable: makeVao(points),

		x: x,
		y: y,
	}
}

func (c *cell) draw() {
	gl.BindVertexArray(c.drawable)
	gl.DrawArrays(gl.TRIANGLES, 0, int32(len(square)/3))
}

// initGlfw initializes glfw and returns a Window to use.
func initGlfw() *glfw.Window {
	if err := glfw.Init(); err != nil {
		panic(err)
	}
	glfw.WindowHint(glfw.Resizable, glfw.False)
	glfw.WindowHint(glfw.ContextVersionMajor, 4)
	glfw.WindowHint(glfw.ContextVersionMinor, 1)
	glfw.WindowHint(glfw.OpenGLProfile, glfw.OpenGLCoreProfile)
	glfw.WindowHint(glfw.OpenGLForwardCompatible, glfw.True)

	window, err := glfw.CreateWindow(width, height, "Conway's Game of Life", nil, nil)
	if err != nil {
		panic(err)
	}
	window.MakeContextCurrent()

	return window
}

// initOpenGL initializes OpenGL and returns an intiialized program.
func initOpenGL() uint32 {
	if err := gl.Init(); err != nil {
		panic(err)
	}
	version := gl.GoStr(gl.GetString(gl.VERSION))
	log.Println("OpenGL version", version)

	vertexShader, err := compileShader(vertexShaderSource, gl.VERTEX_SHADER)
	if err != nil {
		panic(err)
	}

	fragmentShader, err := compileShader(fragmentShaderSource, gl.FRAGMENT_SHADER)
	if err != nil {
		panic(err)
	}

	prog := gl.CreateProgram()
	gl.AttachShader(prog, vertexShader)
	gl.AttachShader(prog, fragmentShader)
	gl.LinkProgram(prog)
	return prog
}

// makeVao initializes and returns a vertex array from the points provided.
func makeVao(points []float32) uint32 {
	var vbo uint32
	gl.GenBuffers(1, &vbo)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.BufferData(gl.ARRAY_BUFFER, 4*len(points), gl.Ptr(points), gl.STATIC_DRAW)

	var vao uint32
	gl.GenVertexArrays(1, &vao)
	gl.BindVertexArray(vao)
	gl.EnableVertexAttribArray(0)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.VertexAttribPointer(0, 3, gl.FLOAT, false, 0, nil)

	return vao
}

func compileShader(source string, shaderType uint32) (uint32, error) {
	shader := gl.CreateShader(shaderType)

	csources, free := gl.Strs(source)
	gl.ShaderSource(shader, 1, csources, nil)
	free()
	gl.CompileShader(shader)

	var status int32
	gl.GetShaderiv(shader, gl.COMPILE_STATUS, &status)
	if status == gl.FALSE {
		var logLength int32
		gl.GetShaderiv(shader, gl.INFO_LOG_LENGTH, &logLength)

		log := strings.Repeat("\x00", int(logLength+1))
		gl.GetShaderInfoLog(shader, logLength, nil, gl.Str(log))

		return 0, fmt.Errorf("failed to compile %v: %v", source, log)
	}

	return shader, nil
}

OpenGL & Go Tutorial Part 1: Hello, OpenGL

Sun, 12 Mar 2017 00:00:00 +0000

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Introduction

OpenGL is pretty much the gold standard for any kind of graphics work, from desktop GUIs to games to mobile applications and even the web, I can almost guarantee you’ve viewed something rendered by OpenGL today. However, regardless of how popular and useful OpenGL is, it can be quite intimidating to get started compared to more high-level graphics libraries.

The purpose of this tutorial is to give you a starting point and basic understanding of OpenGL, and how to utilize it with Go. There are bindings for OpenGL in just about every language and Go is no exception with the go-gl packages, a full suite of generated OpenGL bindings for various OpenGL versions.

The tutorial will walk through a few phases outlined below, with our end goal being to implement Conway’s Game of Life using OpenGL to draw the game board in a desktop window. The full source code is available on GitHub at github.com/KyleBanks/conways-gol so feel free to check it out if you get stuck or use it as a reference if you decide to go your own way.

Before we get started, we need to get an understanding of what Conway’s Game of Life actually is. Here’s the summary from Wikipedia:

The Game of Life, also known simply as Life, is a cellular automaton devised by the British mathematician John Horton Conway in 1970.

The “game” is a zero-player game, meaning that its evolution is determined by its initial state, requiring no further input. One interacts with the Game of Life by creating an initial configuration and observing how it evolves, or, for advanced “players”, by creating patterns with particular properties.

Rules

The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, alive or dead, or “populated” or “unpopulated” (the difference may seem minor, except when viewing it as an early model of human/urban behaviour simulation or how one views a blank space on a grid). Every cell interacts with its eight neighbours, which are the cells that are horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.

Any live cell with two or three live neighbours lives on to the next generation.

Any live cell with more than three live neighbours dies, as if by overpopulation.

Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

And without further ado, here’s a demo of what we’ll be building:

In our simulation, a white cell indicates that it is alive, and black cell indicates that it is not.

Outline

The tutorial is going to cover a lot of ground starting with the basics, however it will assume you have a minimal working knowledge of Go - at the very least you should know the basics of variables, slices, functions and structs, and have a working Go environment setup. I’ve developed the tutorial using Go version 1.8, but it should be compatible with previous versions as well. There is nothing particularly novel here in the Go implementation, so if you have experience in any similar programming language you should be just fine.

As for the tutorial, here’s what we’ll be covering:

Part 1: Hello, OpenGL: Install and Setup OpenGL and GLFW, Draw a Triangle to the Window
Part 2: Drawing the Game Board: Make a Square out of Triangles, Draw a Grid of Squares covering the Window
Part 3: Implementing the Game: Implement Conway’s Game

The final source code is available on GitHub but each Part includes a Checkpoint at the bottom containing the code up until that point. If anything is ever unclear or if you feel lost, check the bottom of the post for the full source!

Let’s get started!

Install and Setup OpenGL and GLFW

We’ve introduced OpenGL but in order to use it we’re going to need a window to draw on to. GLFW is a cross-platform API for OpenGL that allows us to create and reference a window, and is also provided by the go-gl suite.

The first thing we need to do is decide on an OpenGL version. For the purposes of this tutorial we’ll use OpenGL v4.1 but you can use v2.1 just fine if your system doesn’t have the latest OpenGL versions. In order to install OpenGL we’ll do the following:

# For OpenGL 4.1
$ go get github.com/go-gl/gl/v4.1-core/gl

# Or 2.1
$ go get github.com/go-gl/gl/v2.1/gl

Next up, let’s install GLFW:

$ go get github.com/go-gl/glfw/v3.2/glfw

With these two packages installed, we’re ready to get started! We’re going to start by creating main.go and importing the packages (and a couple others we’ll need in a moment).

package main

import (
    "log"
    "runtime"

    "github.com/go-gl/gl/v4.1-core/gl" // OR: github.com/go-gl/gl/v2.1/gl
    "github.com/go-gl/glfw/v3.2/glfw"
)

Next lets define the main function, the purpose of which is to initialize OpenGL and GLFW and display the window:

const (
    width  = 500
    height = 500
)

func main() {
    runtime.LockOSThread()

    window := initGlfw()
    defer glfw.Terminate()

    for !window.ShouldClose() {
        // TODO
    }
}

// initGlfw initializes glfw and returns a Window to use.
func initGlfw() *glfw.Window {
    if err := glfw.Init(); err != nil {
            panic(err)
    }
    
    glfw.WindowHint(glfw.Resizable, glfw.False)
    glfw.WindowHint(glfw.ContextVersionMajor, 4) // OR 2
    glfw.WindowHint(glfw.ContextVersionMinor, 1)
    glfw.WindowHint(glfw.OpenGLProfile, glfw.OpenGLCoreProfile)
    glfw.WindowHint(glfw.OpenGLForwardCompatible, glfw.True)

    window, err := glfw.CreateWindow(width, height, "Conway's Game of Life", nil, nil)
    if err != nil {
            panic(err)
    }
    window.MakeContextCurrent()

    return window
}

Alright let’s take a minute to walk through this and see what’s going on. First we define a couple constants, width and height - these will determine the size of the window, in pixels.

Next we have the main function. Here we instruct the runtime package to LockOSThread(), which ensures we will always execute in the same operating system thread, which is important for GLFW which must always be called from the same thread it was initialized on. Speaking of which, next we call initGlfw to get a window reference, and defer terminating. The window reference is then used in a for-loop where we say as long as the window should remain open, do something. We’ll come back to this in a bit.

initGlfw is our next function, wherein we call glfw.Init() to initialize the GLFW package. After that, we define some global GLFW properties, including disabling window resizing and the properties of our OpenGL version. Next it’s time to create a glfw.Window which is where we’re going to do our future drawing. We simply tell it the width and height we want, as well as a title, and then call window.MakeContextCurrent, binding the window to our current thread. Finally, we return the window.

If you build and run the program now, you should see… nothing. This makes sense, because we’re not actually doing anything with the window yet.

Let’s fix that by defining a new function that initializes OpenGL:

// initOpenGL initializes OpenGL and returns an intiialized program.
func initOpenGL() uint32 {
    if err := gl.Init(); err != nil {
            panic(err)
    }
    version := gl.GoStr(gl.GetString(gl.VERSION))
    log.Println("OpenGL version", version)

    prog := gl.CreateProgram()
    gl.LinkProgram(prog)
    return prog
}

initOpenGL, like our initGlfw function above, initializes the OpenGL library and creates a program. A program gives us a reference to store shaders, which can then be used for drawing. We’ll come back to this in a bit, but for now just know that OpenGL is initialized and we have a program reference. We also print out the OpenGL version which can be helpful for debugging.

Back in main, let’s call this new function:

func main() {
    runtime.LockOSThread()

    window := initGlfw()
    defer glfw.Terminate()
    
    program := initOpenGL()

    for !window.ShouldClose() {
        draw(window, program)
    }
}

You’ll notice now that we have our program reference, we’re calling a new draw function within our core window loop. Eventually this function will draw all of our cells to visualize the game state, but for now its just going to clear the window so we get a black screen:

func draw(window *glfw.Window, program uint32) {
    gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
    gl.UseProgram(prog)
    
    glfw.PollEvents()
    window.SwapBuffers()
}

The first thing we do is call gl.Clear to remove anything from the window that was drawn last frame, giving us a clean slate. Next we tell OpenGL to use our program reference, which currently does nothing. Finally, we tell GLFW to check if there were any mouse or keyboard events (which we won’t be handling in this tutorial) with the PollEvents function, and tell the window to SwapBuffers. Buffer swapping is important because GLFW (like many graphics libraries) uses double buffering, meaning everything you draw is actually drawn to an invisible canvas, and only put onto the visible canvas when you’re ready - which in this case, is indicated by calling SwapBuffers.

Alright, we’ve covered a lot here, so let’s take a moment to see the fruits of our labors. Go ahead and run the program now, and you should get to see your first visual:

Beautiful.

Draw a Triangle to the Window

We’ve made some serious progress, even if it doesn’t look like much, but we still need to actually draw something. We’ll start by drawing a triangle, which may at first seem like it would be more difficult to draw than the squares we’re eventually going to, but you’d be mistaken for thinking so. What you may not know is that triangles are probably the easiest shapes to draw, and in fact we’ll eventually be making our squares out of triangles anyways.

Alright so we want to draw a triangle, but how? Well, we draw shapes by defining the vertices of the shapes and providing them to OpenGL to be drawn. Let’s first define our triangle at the top of main.go:

var (
    triangle = []float32{
        0, 0.5, 0, // top
        -0.5, -0.5, 0, // left
        0.5, -0.5, 0, // right
    }
)

This looks weird, but let’s break it down. First we have a slice of float32, which is the datatype we always use when providing vertices to OpenGL. The slice contains 9 values, three for each vertex of a triangle. The top line, 0, 0.5, 0, is the top vertex represented as X, Y, and Z coordinates, the second line is the left vertex, and the third line is the right vertex. Each of these pairs of three represents the X, Y, and Z coordinates of the vertex relative to the center of the window, between -1 and 1. So the top point has an X of zero because its X is in the center of the window, a Y of 0.5 meaning it will be up one quarter (because the range is -1 to 1) of the window relative to the center of the window, and a Z of zero. For our purposes because we are drawing only in two dimensions, our Z values will always be zero. Now have a look at the left and right vertices and see if you can understand why they are defined as they are - it’s okay if it isn’t immediately clear, we’re going to see it on the screen soon enough so we’ll have a perfect visualization to play with.

Okay, we have a triangle defined, but now we need to draw it. In order to draw it, we need what’s called a Vertex Array Object or vao which is created from a set of points (what we defined as our triangle), and can be provided to OpenGL to draw. Let’s create a function called makeVao that we can provide with a slice of points and have it return a pointer to an OpenGL vertex array object:

// makeVao initializes and returns a vertex array from the points provided.
func makeVao(points []float32) uint32 {
    var vbo uint32
    gl.GenBuffers(1, &vbo)
    gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
    gl.BufferData(gl.ARRAY_BUFFER, 4*len(points), gl.Ptr(points), gl.STATIC_DRAW)
    
    var vao uint32
    gl.GenVertexArrays(1, &vao)
    gl.BindVertexArray(vao)
    gl.EnableVertexAttribArray(0)
    gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
    gl.VertexAttribPointer(0, 3, gl.FLOAT, false, 0, nil)
    
    return vao
}

First we create a Vertex Buffer Object or vbo to bind our vao to, which is created by providing the size (4 x len(points)) and a pointer to the points (gl.Ptr(points)). You may be wondering why it’s 4 x len(points) - why not 6 or 3 or 1078? The reason is we are using float32 slices, and a 32-bit float has 4 bytes, so we are saying the size of the buffer, in bytes, is 4 times the number of points.

Now that we have a buffer, we can create the vao and bind it to the buffer with gl.BindBuffer, and finally return the vao. This vao will then be used to draw the triangle!

Back in main:

func main() {
    ...

    vao := makeVao(triangle)
    for !window.ShouldClose() {
        draw(vao, window, program)
    }
}

Here we call **makeVao** to get our **vao** reference from the **triangle** points we defined before, and pass it as a new argument to the **draw** function:

func draw(vao uint32, window *glfw.Window, program uint32) {
    gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
    gl.UseProgram(program)
    
    gl.BindVertexArray(vao)
    gl.DrawArrays(gl.TRIANGLES, 0, int32(len(triangle) / 3))
    
    glfw.PollEvents()
    window.SwapBuffers()
}

Then we bind OpenGL to our vao so it knows what we’re talking above when we tell it to DrawArrays, and tell it the length of the triangle (divided by three, one for each X, Y, Z coordinate) slice so it knows how many vertices to draw.

If you run the application at this point you might be expecting to see a beautiful triangle in the center of the window, but unfortunately you would be mistaken. There’s still one thing left to do, you see we’ve told OpenGL that we want to draw a triangle, but we need to tell it how to draw the triangle.

In order to do that, we need what are called fragment and vertex shaders, which are beyond the scope of this tutorial. All you really need to understand for this application is that shaders are their own mini-programs (written in OpenGL Shader Language or GLSL) that typically run on the GPU. They’re really interesting and force you to think in a different way to typically programming tasks, but again they’re out of scope for this tutorial. For our purposes, just understand that a vertex shader manipulates the vertices to be drawn by OpenGL and generates the data passed to the fragment shader, which then determines the color of each fragment (you can just consider a fragment to be a pixel) to be drawn to the screen.

We start by adding two more imports and a function called compileShader:

import (
    "strings"
    "fmt"
)

func compileShader(source string, shaderType uint32) (uint32, error) {
    shader := gl.CreateShader(shaderType)
    
    csources, free := gl.Strs(source)
    gl.ShaderSource(shader, 1, csources, nil)
    free()
    gl.CompileShader(shader)
    
    var status int32
    gl.GetShaderiv(shader, gl.COMPILE_STATUS, &status)
    if status == gl.FALSE {
        var logLength int32
        gl.GetShaderiv(shader, gl.INFO_LOG_LENGTH, &logLength)
        
        log := strings.Repeat("\x00", int(logLength+1))
        gl.GetShaderInfoLog(shader, logLength, nil, gl.Str(log))
        
        return 0, fmt.Errorf("failed to compile %v: %v", source, log)
    }
    
    return shader, nil
}

The purpose of this function is to receive the shader source code as a string as well as its type, and return a pointer to the resulting compiled shader. If it fails to compile we’ll get an error returned containing the details.

Now let’s define the shaders and compile them from makeProgram. Back up in our const block where we define width and height:

vertexShaderSource = `
    #version 410
    in vec3 vp;
    void main() {
        gl_Position = vec4(vp, 1.0);
    }
` + "\x00"

fragmentShaderSource = `
    #version 410
    out vec4 frag_colour;
    void main() {
        frag_colour = vec4(1, 1, 1, 1);
    }
` + "\x00"

As you can see these are strings containing GLSL source code for two shaders, one for a vertex shader and another for a fragment shader. The only thing special about these strings is that they both end in a null-termination character, \x00 - a requirement for OpenGL to be able to compile them. Make note of the fragmentShaderSource, this is where we define the color of our shape in RGBA format using a vec4. You can change the value here, which is currently RGBA(1, 1, 1, 1) or white, to change the color of the triangle.

Also of note is that both programs start with #version 410, which you should change to #version 120 if using OpenGL 2.1. 120 is not a typo - use 120 not 210 if you’re on OpenGL 2.1!

Next in initOpenGL we’ll compile the shaders and attach them to our program:

func initOpenGL() uint32 {
	if err := gl.Init(); err != nil {
		panic(err)
	}
	version := gl.GoStr(gl.GetString(gl.VERSION))
	log.Println("OpenGL version", version)
    
    vertexShader, err := compileShader(vertexShaderSource, gl.VERTEX_SHADER)
    if err != nil {
        panic(err)
    }
    fragmentShader, err := compileShader(fragmentShaderSource, gl.FRAGMENT_SHADER)
    if err != nil {
        panic(err)
    }
    
    prog := gl.CreateProgram()
    gl.AttachShader(prog, vertexShader)
    gl.AttachShader(prog, fragmentShader)    
    gl.LinkProgram(prog)
    return prog
}

Here we call our compileShader function with the vertex shader, specifying its type as a gl.VERTEX_SHADER, and do the same with the fragment shader but specifying its type as a gl.FRAGMENT_SHADER. After compiling them, we attach them to our program by calling gl.AttachShader with our program and each compiled shader.

And now we’re finally ready to see our glorious triangle! Go ahead and run, and if all is well you’ll see:

Summary

Amazing, right! All that for a single triangle, but I promise you we’ve setup the majority of the OpenGL code that will serve us for the rest of the tutorial. I highly encourage you to take a few minutes to play with the code and see if you can move, resize, and change the color of the triangle. OpenGL can be very intimidating, and it can feel at times like its difficult to understand what’s going on, but understand that this isn’t magic - it only looks like it is.

In the next part of the tutorial we’ll make a square out of two right-angled triangles - see if you can try and figure this part out before moving on. If not, don’t worry because we’ll be walking through the code in Part 2, followed by creating a full grid of squares that will act as our game board.

Finally, in Part 3 we continue by using the grid to implement Conway’s Game of Life!

Part 1: Hello, OpenGL | Part 2: Drawing the Game Board | Part 3: Implementing the Game

The full source code of the tutorial is available on GitHub.

Checkpoint

Here’s the contents of main.go at this point of the tutorial:

package main

import (
	"fmt"
	"log"
	"runtime"
	"strings"

	"github.com/go-gl/gl/v4.1-core/gl" // OR: github.com/go-gl/gl/v2.1/gl
	"github.com/go-gl/glfw/v3.2/glfw"
)

const (
	width  = 500
	height = 500

	vertexShaderSource = `
		#version 410
		in vec3 vp;
		void main() {
			gl_Position = vec4(vp, 1.0);
		}
	` + "\x00"

	fragmentShaderSource = `
		#version 410
		out vec4 frag_colour;
		void main() {
			frag_colour = vec4(1, 1, 1, 1.0);
		}
	` + "\x00"
)

var (
	triangle = []float32{
		0, 0.5, 0,
		-0.5, -0.5, 0,
		0.5, -0.5, 0,
	}
)

func main() {
	runtime.LockOSThread()

	window := initGlfw()
	defer glfw.Terminate()
	program := initOpenGL()

	vao := makeVao(triangle)
	for !window.ShouldClose() {
		draw(vao, window, program)
	}
}

func draw(vao uint32, window *glfw.Window, program uint32) {
	gl.Clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
	gl.UseProgram(program)

	gl.BindVertexArray(vao)
	gl.DrawArrays(gl.TRIANGLES, 0, int32(len(triangle)/3))

	glfw.PollEvents()
	window.SwapBuffers()
}

// initGlfw initializes glfw and returns a Window to use.
func initGlfw() *glfw.Window {
	if err := glfw.Init(); err != nil {
		panic(err)
	}
	glfw.WindowHint(glfw.Resizable, glfw.False)
	glfw.WindowHint(glfw.ContextVersionMajor, 4)
	glfw.WindowHint(glfw.ContextVersionMinor, 1)
	glfw.WindowHint(glfw.OpenGLProfile, glfw.OpenGLCoreProfile)
	glfw.WindowHint(glfw.OpenGLForwardCompatible, glfw.True)

	window, err := glfw.CreateWindow(width, height, "Conway's Game of Life", nil, nil)
	if err != nil {
		panic(err)
	}
	window.MakeContextCurrent()

	return window
}

// initOpenGL initializes OpenGL and returns an intiialized program.
func initOpenGL() uint32 {
	if err := gl.Init(); err != nil {
		panic(err)
	}
	version := gl.GoStr(gl.GetString(gl.VERSION))
	log.Println("OpenGL version", version)

	vertexShader, err := compileShader(vertexShaderSource, gl.VERTEX_SHADER)
	if err != nil {
		panic(err)
	}

	fragmentShader, err := compileShader(fragmentShaderSource, gl.FRAGMENT_SHADER)
	if err != nil {
		panic(err)
	}

	prog := gl.CreateProgram()
	gl.AttachShader(prog, vertexShader)
	gl.AttachShader(prog, fragmentShader)
	gl.LinkProgram(prog)
	return prog
}

// makeVao initializes and returns a vertex array from the points provided.
func makeVao(points []float32) uint32 {
	var vbo uint32
	gl.GenBuffers(1, &vbo)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.BufferData(gl.ARRAY_BUFFER, 4*len(points), gl.Ptr(points), gl.STATIC_DRAW)

	var vao uint32
	gl.GenVertexArrays(1, &vao)
	gl.BindVertexArray(vao)
	gl.EnableVertexAttribArray(0)
	gl.BindBuffer(gl.ARRAY_BUFFER, vbo)
	gl.VertexAttribPointer(0, 3, gl.FLOAT, false, 0, nil)

	return vao
}

func compileShader(source string, shaderType uint32) (uint32, error) {
	shader := gl.CreateShader(shaderType)

	csources, free := gl.Strs(source)
	gl.ShaderSource(shader, 1, csources, nil)
	free()
	gl.CompileShader(shader)

	var status int32
	gl.GetShaderiv(shader, gl.COMPILE_STATUS, &status)
	if status == gl.FALSE {
		var logLength int32
		gl.GetShaderiv(shader, gl.INFO_LOG_LENGTH, &logLength)

		log := strings.Repeat("\x00", int(logLength+1))
		gl.GetShaderInfoLog(shader, logLength, nil, gl.Str(log))

		return 0, fmt.Errorf("failed to compile %v: %v", source, log)
	}

	return shader, nil
}

Kyle W. Banks

Accessing the Goodreads API with Go

Loading Unlabeled Images with ImageDataGenerator flow_from_directory in Keras

Training/Validation Split with ImageDataGenerator in Keras

Running Multiple make Targets Concurrently

make Concurrently

Using a Convolutional Neural Network to Play Conway's Game of Life with Keras

Game Logic

Generate Game Boards

Construct a Training and Validation Set

Build a Convolutional Neural Network

Train and Save the Model

Try it Out

View Frequent Errors using Test Set

Fixing the Edge Issue with Custom Padding

Introducing modoc, a Lightweight Framework for Large Markdown Documents

Transfer Learning and Retraining Inception/MobileNet with TensorFlow and Docker

Gathering Images

Training

Using the Retrained Model for Predictions

Conclusion

OpenGL & Go Tutorial Part 3: Implementing the Game

Implement Conway’s Game

What’s Next?

Summary

Checkpoint

OpenGL & Go Tutorial Part 2: Drawing the Game Board

Make a Square out of Triangles

Draw a Grid of Squares covering the Window

Summary

Checkpoint

OpenGL & Go Tutorial Part 1: Hello, OpenGL

Introduction

Outline

Install and Setup OpenGL and GLFW

Draw a Triangle to the Window

Summary

Checkpoint

`make` Concurrently