Introduction to make
While I use makefiles daily, I do not write them that often.
They are either written by someone else, for example when I use a third-party library or by a tool, for example when using CMake as build system.
Make is ubiquitous, it is available in all POSIX systems, and there are releases for Windows too.
Lately, I have begun writing more
makefiles because I needed a tool for managing tasks, execute arbitrary commands, and handle dependencies. What might be interesting is that I did not need to decide not to write makefiles for building programs, as I already use a CMake or other specialized build systems for that.
Using make has some drawbacks:
very terse syntax, and not always expressive
has edge cases and quirks (for example cannot handle files and paths with spaces), that can be fixed, but will not because of backward compatibility
AFAIK it is not simple to lint, debug or test
no automatic parallelism (like, for example, ninja)
need to specify manually (with
-j, notice that it can be "automated" with the environment variable
output of commands might get garbled
no colorized output for errors
issues with paths with special characters, most can be overcome, but not all
uses file timestamps for tracking changes, and cannot be changed
it uses extensively the underlying shell
the entry barrier for complex tasks is higher, just like for bash or similar shells
On the plus side:
available on every system (even Windows through WSL, Cygwin, or MSYS, and android with Termux)
used in many situations (mainly software development), so getting proficient with it can help in other tasks
the entry barrier for simply and medium-sized tasks is low, just like bash or compatible shells
there exists many resources (albeit most are too simplistic)
it uses the underlying shell, which lowers the entry barrier and makes simple tasks simple to understand
integration with other tools (syntax highlighting, autocompletion, …)
What stroke me is that as of today there are no real make alternatives or some tool that behaves similarly without being too opinionated. While most alternatives are surely able to handle all tasks I had in mind, as they can execute other binaries, they focus on specific tasks, like compiling C++ or java source files. Thus for my needs, I would leave most provided features on the table.
CMake relies on other tools, it generates the
makefile. Thus it has, the same limitation of
make, even if it is easier to maintain.
Maven and Ant are XML-based and require the JVM to be installed
Gradle, requires the JVM
waf, as it is not packaged for Debian, and has major incompatibilities between releases
Thus I decided to stick with
make for my current use-cases.
# otherwise echo main.txt: printf 'Hello World\n' > main.txt
This makefile defines a single target, which creates the file
Normally a target consists of an output and a list of input files. In this example, there are no input files.
Even for such a simple makefile, there is one advantage over a shell script: autocompletion.
In zsh (bash can be configured with bash-completion too), if I type
make ma<tab>, it gets completed to
make main.txt. It is possible to create autocompletion files for custom shell scripts, but with a makefile, it is not necessary.
Hello World with input file
Let’s create an input file with
printf 'Hello World\n' > source.txt, and update the makefile accordingly:
# otherwise echo main.txt: source.txt cat source.txt > main.txt
Compared to a shell script we have the following features
main.txt is recreated only if source.txt is changed
The previous makefile has some unnecessary repetitions: the output file and the input file(s).
Make provides two automatic variables for accessing those; the output file is saved as
$@, and the input files, space-separated, as
main.txt: source.txt cat $^ > $@
Error handling, avoid corrupted files
There are still a couple of issues.
cat source.txt fails for some reason (for example by the user pressing Ctrl+c),
main.txt is still created, as it was created by the shell before
cat got executed. This has the downside that a subsequent call to make will not execute the
main.txt target again, as the timestamp of
main.txt is newer than
source.txt. In this case, using
cp instead of
cat fixes some errors
main.txt: source.txt cp $^ $@
If the operation fails or is aborted while the copy is in progress, the output file might not have been created yet. Thus even if source.txt does not change, then make will execute the target again.
But there is a better fix, that covers all use-cases.
Adding an empty target
.DELETE_ON_ERROR ensures that on error, the output file is deleted.
.DELETE_ON_ERROR: main.txt: source.txt cp $^ $@
In case of error (exit code different than 0),
main.txt is deleted.
Unfortunately, it does not work well when piping, for example, if
command fails, but
pipe-commands succeeds, then
main.txt is not deleted.
.DELETE_ON_ERROR: main.txt: source.txt command $^ | pipe-command > $@
This problem can be circumvented by creating temporary files
.DELETE_ON_ERROR: main.txt: source.txt command $^ > $@.tmp cat $@.tmp | pipe-command > $@
Compared to a shell script, this
makefile has the following advantages
main.txt is recreated only if source.txt is changed
on error, files are deleted/no files are created (unless piping is involved)
input-output files are clearly marked
phony targets (GNU-ism, Mostly Harmless elsewhere).
Sometimes it is useful to create targets that do not create any files. For example, If we have multiple files, we would like to have one target, normally named
all, to execute all our other targets (and not creating a file all). Another example would be a
clean target, that removes every file created by the other targets.
.DELETE_ON_ERROR: .PHONY: all clean all: main.txt main.txt: source.txt cp $^ $@ clean: rm main.txt
rebuild if makefile changes
Some handwritten makefiles I’ve seen, do not acknowledge it, but the
makefile itself is generally a dependency too!
For example, changing
main.txt: source.txt cat $^ > $@
main.txt: source.txt source2.txt cat $^ > $@
Then make might does not create
main.txt is never then
Thus adding makefile as dependency ensures that if the makefile is changed, then
main.txt is recreated too. But as
makefile does generally not influence directly the output of a command, most of the time it can be filtered out from
$^. This can be accomplished with
.DELETE_ON_ERROR: .PHONY: all clean all: main.txt main.txt: source.txt source2.txt makefile cat $(filter-out makefile,$^) > $@ clean: rm main.txt
clean is unfortunately error-prone too, even if errors are easy to correct. It is especially not-nice that we need to enlist every file we have created. Also generally it cannot work correctly if we change the makefile, as it cannot track targets that we have just deleted from the makefile.
A simpler solution is creating an output directory, and save there also all temporary files. In this case,
clean consists of removing only this one directory, which makes it future-proof, correct, and easy to maintain.
.DELETE_ON_ERROR: .PHONY: all clean OUT_DIR=out all: $(OUT_DIR)/main $(OUT_DIR)/main.txt: source.txt source2.txt makefile | $(OUT_DIR) cat $(filter-out makefile,$^) > $@ clean: rm -rf $(OUT_DIR) $(OUT_DIR): mkdir $@
Alternative to ` | $(OUT_DIR)` as dependency is simply adding
$(OUT_DIR), and then use
filter-out to remove
$(OUT_DIR)` is a directory, not a file with content that can change, tracking its timestamp (which seems to be problematic) is not unnecessary. We just want to ensure that the directory exists before creating our output file, otherwise, we would get an error.
The main disadvantage of an out-of-source build is that the target names are now all prefixed with the output directory. Instead of
make main.txt we have to type
With many targets, it might be useful to colorize similar targets with the same color. In general, it might not work well when running
make in parallel.
.DELETE_ON_ERROR: .PHONY: all clean OUT_DIR=out all: $(OUT_DIR)/main PRINTBEG := @printf "\e[1;34m" PRINTEND := @printf "\e[0m" $(OUT_DIR)/main: source makefile $(PRINTBEG) cp $(filter-out makefile,$^) $@ # or use $< instead of $^ $(PRINTEND) clean: rm -rf $(OUT_DIR) $(OUT_DIR): mkdir $@
Getting fancy with patterns
While it is perfectly fine to enlist all files we want to create, it does not scale well. Often, for a given set of files, we want to do the same action. For example, for every
.cpp file we want to create a corresponding
.obj file, or for every
.adoc file we want to create a corresponding
Enlisting all files, and especially writing for every file the same set of instructions is a tedious and error-prone task.
For managing my
.adoc files, I wrote something similar to the following makefile
.DELETE_ON_ERROR: .PHONY: all clean OUT_DIR=out all: $(HTMLARTICLES) clean: rm -rf $(OUT_DIR) $(OUT_DIR): mkdir $@ ARTICLES := $(wildcard post/*/index.adoc) HTMLARTICLES := $(patsubst post/%/index.adoc, $(OUT_DIR)/post/%/index.html, $(ARTICLES)) $(OUT_DIR)/post/%/index.html: post/%/index.adoc makefile $(ASCIIDOC) $< --destination-dir $(OUT_DIR)/$(basename $(dir $<)) --out-file=index.html