Introduction to make

Notes published the
8 - 9 minutes to read, 1879 words
Categories: build system, scripting
Keywords: build system, make, scripting

While I use makefiles daily, I do not write them that often.

They are either written by someone else, for example when I use a third-party library, or by a tool, for example when using CMake as build system 🗄️.

Make is ubiquitous, it is available in all POSIX systems, and there are releases for Windows too.

Lately, I have begun writing more makefiles because I needed a tool for managing tasks, executing arbitrary commands, and handling dependencies. Note that for specific jobs, like compiling and linking programs, existing higher-level and less generic build systems are probably more appropriate.

Using make has some drawbacks:

  • very terse syntax, and not always expressive

  • has edge cases and quirks (for example cannot handle files and paths with spaces), that can be fixed, but will not because of backward compatibility

  • AFAIK it is not simple to lint, debug, or test

  • no automatic parallelism (like, for example, ninja)

    • needs to specify manually (with -j, notice that it can be "automated" with the environment variable MAKEFLAGS)

    • output of commands might get garbled

  • no colorized output for errors

  • issues with paths with special characters, most can be overcome, but not all

  • uses file timestamps for tracking changes, and cannot be changed

  • it uses extensively the underlying shell

  • the entry barrier for complex tasks is higher, just like for bash or similar shells

On the plus side:

  • available on every system (even Windows through WSL, Cygwin, or MSYS, and Android with Termux)

  • used in many situations (mainly software development), so getting proficient with it can help in other tasks

  • the entry barrier for simple and medium-sized tasks is low, just like bash or compatible shells

  • there exists many resources (albeit most are too simplistic)

  • it uses the underlying shell, which lowers the entry barrier and makes simple tasks simple to understand

  • integration with other tools (syntax highlighting, autocompletion, …​)

What stroke me is that as of today there are no real make alternatives or some tool that behaves similarly without being too opinionated. While most alternative build systems are surely able to handle all tasks I had in mind, as they can execute other binaries, they focus on specific tasks, like compiling C++ or java source files. Thus for my needs, I would leave most provided features on the table.

  • CMake relies on other tools, it generates the makefile. Thus it has, the same limitation of make, even if it is easier to maintain.

  • Maven and Ant are XML-based and require the JVM to be installed

  • Gradle, requires the JVM

  • Scons

  • waf, as it is not packaged for Debian, and has major incompatibilities between releases

Thus I decided to stick with make for my current use cases.

Hello World

main.txt:
	printf 'Hello World\n' > main.txt

This makefile defines a single target, which creates the file main.txt.

Normally a target consists of an output and a list of input files. In this example, there are no input files.

Even for such a simple makefile, there is one advantage over a shell script: autocompletion.

In zsh (bash can be configured with bash-completion too), if I type make ma followed by Tab ↹, it gets completed to make main.txt.

It is possible to create autocompletion files for custom shell scripts, but with a makefile, it is not necessary.

Hello World with input file

Let’s create an input file with "Hello World\n", and update the makefile accordingly:

# otherwise echo
main.txt: source.txt
	cat source.txt > main.txt

Compared to a shell script we have the following features

  • autocompletion

  • the target main.txt is executed only if source.txt is changed (based on the timestamp)

Make variables

The previous makefile has some unnecessary repetitions: the output file and the input file(s).

Make provides two automatic variables 🗄️ for accessing those; the output file name is saved as $@, and the input file names, space-separated, as $^.

main.txt: source.txt
	cat $^ > $@

In case main.txt depends on multiple files, note that make also defines $< as the first input file name.

In this example, using $< or $^ is equivalent.

If you need to access the n-th parameter, you can use following pattern $(word n,$^); for example, for accessing the second parameter: $(word 2,$^)

Error handling, avoid corrupted files

There are still a couple of issues.

If the user presses Ctrl+C before cat is executed, the file main.txt might have been already created by the shell.

This means that a subsequent call to make will not execute the target again, as the timestamp of main.txt is newer than source.txt. In this case, using cp instead of cat might fix this particular issue

main.txt: source.txt
	cp $^ $@

If the operation fails or is aborted while the copy is in progress, the output file might not have been created yet.

There exists a better and more robust fix, that covers this and other use cases.

Adding an empty target .DELETE_ON_ERROR ensures that on error, the output file is deleted (as long as make itself does not crash).

.DELETE_ON_ERROR:

main.txt: source.txt
	cp $^ $@

In case of an error (this means cp ends with an exit code different than 0), main.txt if present, is deleted by make.

Unfortunately, it does not work well when piping, for example, if command fails, but pipe-commands succeeds, then main.txt is not deleted.

.DELETE_ON_ERROR:

main.txt: source.txt
	command $^ | pipe-command > $@

This problem can be circumvented by creating temporary files

.DELETE_ON_ERROR:

main.txt: source.txt
	command $^ > $@.tmp
	cat $@.tmp | pipe-command > $@

Compared to a shell script, this makefile has the following advantages

  • autocompletion

  • main.txt is recreated only if source.txt is changed

  • on error, files are deleted/no files are created (unless piping is involved)

  • input-output files are clearly marked

PHONY targets (GNU make specific, should be harmless elsewhere)

Sometimes it is useful to create targets that do not create any files. For example, in the case of multiple targets, it is standard practice to have a target named all, to execute all other targets, but that does not create a file named "all". Another example would be a clean target, that removes the files created by the other targets, and does not create a file named "clean".

.DELETE_ON_ERROR:
.PHONY: all clean

all: main.txt

main.txt: source.txt
	cp $^ $@

clean:
	rm main.txt

rebuild if makefile changes

Some handwritten makefiles I’ve seen, do not acknowledge it, but the makefile itself is generally a dependency too!

For example, when changing the makefile from

main.txt: source.txt
	cat $^ > $@

to

main.txt: source.txt source2.txt
	cat $^ > $@

we would like, if we execute make main.txt, the target to be executed, even if source.txt and source2.txt have not been changed. Unfortunately make will not update main.txt, if main.txt is newer than source2.txt.

Thus adding makefile as a dependency ensures that if the makefile is changed, then main.txt is recreated too. But as makefile does generally not influence directly the output of a command, most of the time it can be filtered out from $^. This can be accomplished with filter-out:

.DELETE_ON_ERROR:
.PHONY: all clean

all: main.txt

main.txt: source.txt source2.txt makefile
	cat $(filter-out makefile,$^) > $@

clean:
	rm main.txt

Oder-only prerequisites

A dependency added with | is documented as order-only prerequisite 🗄️. In particular, it does not cause a rebuild if such dependency is newer than the target.

A perfect example of such dependency would be an HTML file and a CSS file. The HTML file depends on the CSS file, but there is no need to rebuild the HTML file if the CSS file changes, as those files are "combined" together at runtime, and not build time.

.DELETE_ON_ERROR:
.PHONY: all clean

index.html: source.html makefile | style.css
	cp $(filter-out makefile,$^) $@

style.css: source.css makefile
	cp $(filter-out makefile,$^) $@

clean:
	rm index.html style.css

out-of-source builds

The target clean is unfortunately error-prone too, even if errors are easy to correct. It is especially irritating that we need to enlist every file we have created. Generally, it cannot work correctly if we change the makefile, as it cannot track targets that we have removed from the makefile.

A simpler solution is creating an output directory and saving there all files. In this case, clean consists of removing only this one directory, which makes it future-proof, correct, and easy to maintain.

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR := $(CURDIR)/out

all: $(OUT_DIR)/main

$(OUT_DIR)/main.txt: source.txt source2.txt makefile | $(OUT_DIR)
	cat $(filter-out makefile,$^) > $@

clean:
	rm -rf -- $(OUT_DIR)

$(OUT_DIR):
	mkdir $@

The variable CURDIR is defined by make itself, and point to the directory where the makefile is located.

A possible alternative to | $(OUT_DIR) as dependency is to add $(OUT_DIR) (without |), and then use filter-out to remove $(OUT_DIR) and makefile. As $(OUT_DIR) is a directory, not a file with content that can change, tracking its timestamp (which seems to be problematic) is not necessary. We just want to ensure that the directory exists before creating our output file, otherwise, we will get an error.

The main disadvantage of an out-of-source build is that the target names are now all prefixed with the output directory. Instead of make main.txt we have to type make out/main.txt unless we create a PHONY main.txt target that depends on out/main.txt.

Colorized output

With many targets, it might be useful to colorize similar targets with the same color. In general, it might not work well when running make in parallel.

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR := $(CURDIR)/out

all: $(OUT_DIR)/main

PRINTBEG := @printf "\e[1;34m"
PRINTEND := @printf "\e[0m"

$(OUT_DIR)/main: source makefile
	$(PRINTBEG)
	cp $(filter-out makefile,$^) $@ # or use $< instead of $^
	$(PRINTEND)

clean:
	rm -r -f -- $(OUT_DIR)

$(OUT_DIR):
	mkdir -p $@

Getting fancy with patterns

While it is perfectly fine to enlist all files we want to create, it does not scale well. Often, for a given set of files, we want to do the same action. For example, for every .cpp file we want to create a corresponding .obj file, or for every .adoc or .md file we want to create a corresponding .html file.

Enlisting all files, and especially writing for every file the same set of instructions is a tedious and error-prone task.

In make it is possible to use wildcards 🗄️ and pattern rules 🗄️.

For managing my .adoc files, I wrote something similar to the following makefile

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR := $(CURDIR)/public
ASCIIDOC := asciidoctor

SOURCES := $(wildcard adoc/*/index.adoc)
HTMLSOURCES := $(patsubst adoc/%/index.adoc, $(OUT_DIR)/post/%/index.html, $(SOURCES))

all: $(HTMLSOURCES)

clean:
	rm -rf -- $(OUT_DIR)

$(OUT_DIR):
	mkdir $@

$(OUT_DIR)/post/%/index.html: adoc/%/index.adoc makefile | $(OUT_DIR)
	$(ASCIIDOC) $< --destination-dir $(dir $@) --out-file=$(notdir $@)

Note that dir and notdir are not command-line utilities, but make builtins 🗄️.

Do you want to share your opinion? Or is there an error, some parts that are not clear enough?

You can contact me anytime.