Introduction to make


6 - 8 minutes read, 1599 words
Categories: build system scripting
Keywords: build system make scripting

While I use makefiles daily, I do not write them that often.

They are either written by someone else, for example when I use a third-party library or by a tool, for example when using CMake as build system.

Make is ubiquitous, it is available in all POSIX systems, and there are releases for Windows too.

Lately, I have begun writing more makefiles because I needed a tool for managing tasks, execute arbitrary commands, and handle dependencies. What might be interesting is that I did not need to decide not to write makefiles for building programs, as I already use a CMake or other specialized build systems for that.

Using make has some drawbacks:

  • very terse syntax, and not always expressive

  • has edge cases and quirks (for example cannot handle files and paths with spaces), that can be fixed, but will not because of backward compatibility

  • AFAIK it is not simple to lint, debug or test

  • no automatic parallelism (like, for example, ninja)

    • need to specify manually (with -j, notice that it can be "automated" with the environment variable MAKEFLAGS)

    • output of commands might get garbled

  • no colorized output for errors

  • issues with paths with special characters, most can be overcome, but not all

  • uses file timestamps for tracking changes, and cannot be changed

  • it uses extensively the underlying shell

  • the entry barrier for complex tasks is higher, just like for bash or similar shells

On the plus side:

  • available on every system (even Windows through WSL, Cygwin, or MSYS, and android with Termux)

  • used in many situations (mainly software development), so getting proficient with it can help in other tasks

  • the entry barrier for simply and medium-sized tasks is low, just like bash or compatible shells

  • there exists many resources (albeit most are too simplistic)

  • it uses the underlying shell, which lowers the entry barrier and makes simple tasks simple to understand

  • integration with other tools (syntax highlighting, autocompletion, …​)

What stroke me is that as of today there are no real make alternatives or some tool that behaves similarly without being too opinionated. While most alternatives are surely able to handle all tasks I had in mind, as they can execute other binaries, they focus on specific tasks, like compiling C++ or java source files. Thus for my needs, I would leave most provided features on the table.

  • CMake relies on other tools, it generates the makefile. Thus it has, the same limitation of make, even if it is easier to maintain.

  • Maven and Ant are XML-based and require the JVM to be installed

  • Gradle, requires the JVM

  • Scons

  • waf, as it is not packaged for Debian, and has major incompatibilities between releases

Thus I decided to stick with make for my current use-cases.

Hello World

# otherwise echo
main.txt:
	printf 'Hello World\n' > main.txt

This makefile defines a single target, which creates the file main.txt.

Normally a target consists of an output and a list of input files. In this example, there are no input files.

Even for such a simple makefile, there is one advantage over a shell script: autocompletion.

In zsh (bash can be configured with bash-completion too), if I type make ma<tab>, it gets completed to make main.txt. It is possible to create autocompletion files for custom shell scripts, but with a makefile, it is not necessary.

Hello World with input file

Let’s create an input file with printf 'Hello World\n' > source.txt, and update the makefile accordingly:

# otherwise echo
main.txt: source.txt
	cat source.txt > main.txt

Compared to a shell script we have the following features

  • autocompletion

  • main.txt is recreated only if source.txt is changed

Make variables

The previous makefile has some unnecessary repetitions: the output file and the input file(s).

Make provides two automatic variables for accessing those; the output file is saved as $@, and the input files, space-separated, as $^.

main.txt: source.txt
	cat $^ > $@

Error handling, avoid corrupted files

There are still a couple of issues.

If cat source.txt fails for some reason (for example by the user pressing Ctrl+c), main.txt is still created, as it was created by the shell before cat got executed. This has the downside that a subsequent call to make will not execute the main.txt target again, as the timestamp of main.txt is newer than source.txt. In this case, using cp instead of cat fixes some errors

main.txt: source.txt
	cp $^ $@

If the operation fails or is aborted while the copy is in progress, the output file might not have been created yet. Thus even if source.txt does not change, then make will execute the target again.

But there is a better fix, that covers all use-cases.

Adding an empty target .DELETE_ON_ERROR ensures that on error, the output file is deleted.

.DELETE_ON_ERROR:

main.txt: source.txt
	cp $^ $@

In case of error (exit code different than 0), main.txt is deleted.

Unfortunately, it does not work well when piping, for example, if command fails, but pipe-commands succeeds, then main.txt is not deleted.

.DELETE_ON_ERROR:

main.txt: source.txt
	command $^ | pipe-command > $@

This problem can be circumvented by creating temporary files

.DELETE_ON_ERROR:

main.txt: source.txt
	command $^ > $@.tmp
	cat $@.tmp | pipe-command > $@

Compared to a shell script, this makefile has the following advantages

  • autocompletion

  • main.txt is recreated only if source.txt is changed

  • on error, files are deleted/no files are created (unless piping is involved)

  • input-output files are clearly marked

phony targets (GNU-ism, Mostly Harmless elsewhere).

Sometimes it is useful to create targets that do not create any files. For example, If we have multiple files, we would like to have one target, normally named all, to execute all our other targets (and not creating a file all). Another example would be a clean target, that removes every file created by the other targets.

.DELETE_ON_ERROR:
.PHONY: all clean

all: main.txt

main.txt: source.txt
	cp $^ $@

clean:
	rm main.txt

rebuild if makefile changes

Some handwritten makefiles I’ve seen, do not acknowledge it, but the makefile itself is generally a dependency too!

For example, changing

main.txt: source.txt
	cat $^ > $@

to

main.txt: source.txt source2.txt
	cat $^ > $@

Then make might does not create main.txt, if main.txt is never then source2.txt.

Thus adding makefile as dependency ensures that if the makefile is changed, then main.txt is recreated too. But as makefile does generally not influence directly the output of a command, most of the time it can be filtered out from $^. This can be accomplished with filter-out:

.DELETE_ON_ERROR:
.PHONY: all clean

all: main.txt

main.txt: source.txt source2.txt makefile
	cat $(filter-out makefile,$^) > $@

clean:
	rm main.txt

out-of-source builds

The target clean is unfortunately error-prone too, even if errors are easy to correct. It is especially not-nice that we need to enlist every file we have created. Also generally it cannot work correctly if we change the makefile, as it cannot track targets that we have just deleted from the makefile.

A simpler solution is creating an output directory, and save there also all temporary files. In this case, clean consists of removing only this one directory, which makes it future-proof, correct, and easy to maintain.

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR=out

all: $(OUT_DIR)/main

$(OUT_DIR)/main.txt: source.txt source2.txt makefile | $(OUT_DIR)
	cat $(filter-out makefile,$^) > $@

clean:
	rm -rf $(OUT_DIR)

$(OUT_DIR):
	mkdir $@

Alternative to ` | $(OUT_DIR)` as dependency is simply adding $(OUT_DIR), and then use filter-out to remove $(OUT_DIR) and makefile. As $(OUT_DIR)` is a directory, not a file with content that can change, tracking its timestamp (which seems to be problematic) is not unnecessary. We just want to ensure that the directory exists before creating our output file, otherwise, we would get an error.

The main disadvantage of an out-of-source build is that the target names are now all prefixed with the output directory. Instead of make main.txt we have to type make out/main.txt

colorized output

With many targets, it might be useful to colorize similar targets with the same color. In general, it might not work well when running make in parallel.

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR=out

all: $(OUT_DIR)/main

PRINTBEG := @printf "\e[1;34m"
PRINTEND := @printf "\e[0m"

$(OUT_DIR)/main: source makefile
	$(PRINTBEG)
	cp $(filter-out makefile,$^) $@ # or use $< instead of $^
	$(PRINTEND)

clean:
	rm -rf $(OUT_DIR)

$(OUT_DIR):
	mkdir $@

Getting fancy with patterns

While it is perfectly fine to enlist all files we want to create, it does not scale well. Often, for a given set of files, we want to do the same action. For example, for every .cpp file we want to create a corresponding .obj file, or for every .adoc file we want to create a corresponding .html file.

Enlisting all files, and especially writing for every file the same set of instructions is a tedious and error-prone task.

In make it is possible to use wildcards and pattern rules.

For managing my .adoc files, I wrote something similar to the following makefile

.DELETE_ON_ERROR:
.PHONY: all clean

OUT_DIR=out

all: $(HTMLARTICLES)

clean:
	rm -rf $(OUT_DIR)

$(OUT_DIR):
	mkdir $@

ARTICLES := $(wildcard post/*/index.adoc)
HTMLARTICLES := $(patsubst post/%/index.adoc, $(OUT_DIR)/post/%/index.html, $(ARTICLES))

$(OUT_DIR)/post/%/index.html: post/%/index.adoc makefile
	$(ASCIIDOC) $< --destination-dir $(OUT_DIR)/$(basename $(dir $<)) --out-file=index.html

Do you want to share your opinion? Or is there an error, same parts that are not clear enough?

You can contact me here.