Automating the Build Process

Before You Start…

Get together with your partner and clone this GitHub repository.

The activity is split into multiple parts. After certain parts we will ask you to pause and reflect a bit on what you did in that part. Please do your best to NOT scroll to the next section until you've finished reflecting (because the next section may contain "spoilers" for the reflection)!

Review the C++ Code

The repository contains a simple C++ program that consists of three source files:

• cow.cpp and cow.hpp — code that defines a Cow class
• main.cpp — a small program that uses the Cow class

At this point you're pretty used to typing the commands to compile and link C++ programs. But it might feel tiresome to have to remember all the commands and type them in every time you want to build the program. Wouldn't it be nice if we could automate this process?

Review build-all.py

Build the code by running

python3 build-all.py

Confirm that the executable works by running ./main.

Next, try modifying one of the C++ source files (e.g., add a cout statement somewhere) and re-running python3 build-all.py. Then, reflect on the following question:

Once you're done reflecting, scroll down to proceed to the next part.

Review rebuild.py

We're going to throw away build-all.py and replace it with rebuild.py, which is a smarter build script that only rebuilds what is necessary (but can also build everything for the first time, too).

Look over the code for rebuild.py and run it by typing

python3 rebuild.py

Experiment with editing one of the C++ source files and confirm that it rebuilds exactly what is necessary. Try also just removing main (with rm main) and confirm it just does the linking step. Then, reflect on the following question:

When you're ready to move on, run

rm *.o main

Then scroll down to proceed to the next part.

Try running make.py

A problem with rebuild.py is that it's very much hard-coded to this particular program—what files depend on other files and what commands to run are baked into the code.

So make.py is a generalization of the build process, which turns the dependencies into data. Look at the code and find the definition of buildRules. It's a dictionary that specifies both what the prerequisite files are for a given file (i.e., what things it needs to compile) and the compilation command to run for that file.

The most important part of make.py is the definition of buildRules. Look it over and make sure you understand what it is saying. (There is a block comment above it that explains the format, but if you're unsure, ask!)

Next, read over the make function. It's a little long, but it should be fairly self explanatory.

Then run

python3 make.py

make.py is rather chatty about what it is doing and why. Read through the output that it produced explaining its actions. Then, as you did with rebuild.py, you may want to try modifying a C++ source file and confirming that make.py rebuilds exactly what needs to be rebuilt.

The specification in buildRules also includes a rule for how to build a file called clean—have a go at figuring out what will happen when you run it.

To check you answer, run:

python3 make.py clean

Is it good or bad that the file called clean is never actually created by this rule? (FWIW, this is known as a “phony” target; the make algorithm always thinks it needs to do what's necessary to build this target.)

Getting to Know Makefiles — example1.mak

In 1976, after commiserating with his colleagues about the complexity of building programs, Stuart Feldman at Bell Labs wrote the first version of a C program called make that does what our make.py script does, with a few enhancements. It reads a file (known as a “makefile”) to get the same data that was in buildRules. Check out example1.mak; it's basically the same data, just in a different (and easier to type!) format.

Notice that the format is

target : prerequisites
	commands

where the commands are indented by a single TAB character (not spaces!). This file format is super simple to make it easy for a program to read it. Before contining, briefly reflect on the following question:

You can see how a makefile works by running

cs70-make -f example1.mak

You can also just make a specific target; for example,

cs70-make -f example1.mak clean

runs the clean target (which deletes files that were created when make ran on some other target).

Feel free to experiment with changing files and making sure that cs70-make only rebuilds what is necessary.

More make features — example2.mak

Look over example2.mak and try it out. This makefile adds macros (a.k.a. variables) to avoid having to copy and paste the same text in multiple places.

In the makefile, we can define a macro like this:

CXXFLAGS = -g -Wall -std=c++17

The variable CXXFLAGS is now bound to the text -g -Wall -std=c++17. Then, elsewhere in the makefile (basically anywhere), we can use $(CXXFLAGS) to substitute that text. Now if we want to change the compilation flags, we only have to change them in one place.

We've also added another phony target called all. This commonly provided target is most useful when we have several executables (or other components, like documentation) that we want to build as we can list all our executables as prerequisites for all.

If you don't specify a target on the command line, make defaults to building the all target's dependencies.

Advanced make features — example3.mak

Look over example3.mak and try it out. It uses some even more advanced make features that allow us to type even less code.

Automatic Variables

When writing the build commands for a rule, we often want to refer to the target and prerequisites of that rule. make provides automatic variables for this purpose:

• $@ — the target of the rule (the @ sign looks a bit like a target)
• $< — the first prerequisite of the rule (the < points to the left towards the first prerequisite)
• $^ — all the prerequisites of the rule (the ^ points up to all the prerequisites listed above)

Thus, instead of writing

cow.o: cow.cpp cow.hpp
	$(CXX) $(CXXFLAGS) -c cow.cpp

we can write

cow.o: cow.cpp cow.hpp
	$(CXX) $(CXXFLAGS) -c $<

meaning “compile the first prerequisite to make the target”.

Similarly, for

main: main.o cow.o
	$(CXX) $(CXXFLAGS) -o main main.o cow.o

we can write

main: main.o cow.o
	$(CXX) $(CXXFLAGS) -o $@ $^

meaning “link all the prerequisites to make the target”.

Suffix Rules

In a suffix rule, we say “here's how to transform a .xxx file into a .yyy file”. For example, we can say “here's how to transform a .cpp file into a .o file”:

.cpp.o:
	$(CXX) $(CXXFLAGS) -c $<

This special kind of rule says “if no one gave you any build commands for building a particular .o file from a .cpp file, use these commands”.

So you can just define a single suffix rule for .cpp to .o and then you only need to write the dependencies for each .o file, as you can see in example3.mak.

clang++ -std=c++17 -MM *.cpp

Want More Information or Another Perspective?

The Working with Makefiles help page provides a summary of the key aspects of Makefiles, using a different program as the running example.

If you find other helpful resources, please share them with the class on Piazza!

(When logged in, completion status appears here.)