Kyle Banks

Running Multiple make Targets Concurrently

Written by @kylewbanks on Jan 19, 2019.

make is a widely used and powerful took for coordinating common tasks within a project, and you’ll often file a Makefile sitting in the root of open source repositories, allowing you to quickly see how to perform common tasks such as running tests, compiling and running, etc. There is also a wealth of tooling around make, one of my favorites being Modern Make, or mmake by TJ Holowaychuk which adds some niceties around make without changing anything about the original command.

If you’re unfamiliar with make, or it’s just been a while, here’s a quick refresher. Create a Makefile containing task definitions for your project. Here’s a relevant section of the Makefile for the website https://whatsthecodeforthat.com that we’ll be working with later on:

local-server:
	hugo server -D
.PHONY: local-server

local-browser:
	open http://localhost:1313
.PHONY: local-browser

This Makefile contains two targets, local-server and local-browser. The first, local-server compiles the hugo project and runs a local server to host the compiled website. The second, local-browser, simply opens the default browser to the hugo development server address on localhost.

Executing either of these targets is then simply:

$ make local-server
$ make local-browser

But herein lays the problem and the purpose of this post, as local-server is a blocking command which means I’m unable to run local-browser right afterwards. To fix this, I could open a second terminal tab and use one for the server and one for opening the browser, but that’s pretty clunky and I generally have enough tabs as it is.

make Concurrently

The solution is to run the two commands concurrently, using the -j argument of make:

$ make -help
Usage: make [options] [target] ...
Options:
  ...
  -j [N], --jobs[=N]          Allow N jobs at once; infinite jobs with no arg.

As the -help description explains, -j allows you to run a variable number of jobs (or targets) at once.

This is how it looks in practice:

$ make -j 2 local-server local-browser

This command will run the two commands simultaneously, in this case starting the local server while concurrently opening the browser. Pretty handy, but we can even do one better by creating a new make target that handles this. Back in the Makefile:

local:
	make -j 2 local-server local-browser
.PHONY: local

local-server:
	hugo server -D
.PHONY: local-server

local-browser:
	open http://localhost:1313
.PHONY: local-browser

Nothing changed with local-server or local-browser, the only difference is a new local target that recursively calls make with the -j 2 argument and the two other targets. Now running the two commands concurrently is as simple as:

$ make local

There are a number of use cases for this kind of parallel functionality, but one of my favorites is to parallelize slow build processes (especially in legacy projects) to reduce wait time during clean, compilation, startup, etc.

Let me know if this post was helpful on Twitter @kylewbanks or down below!