make is great if everything works—you type gmake install and lo! the right things get compiled and installed in the right places. Our goal is to make this happen often, but somehow it often doesn't; instead some weird error message eventually emerges from the bowels of a directory you didn't know existed.
The purpose of this section is to give you a road-map to help you figure out what is going right and what is going wrong.
Debugging Makefiles is something of a black art, but here's a couple of tricks that we find particularly useful. The following command allows you to see the contents of any make variable in the context of the current Makefile:
$ make show VALUE=HS_SRCS
where you can replace HS_SRCS with the name of any variable you wish to see the value of.
GNU make has a -d
option which generates
a dump of the decision procedure used to arrive at a conclusion
about which files should be recompiled. Sometimes useful for
tracking down problems with superfluous or missing
recompilations.
To get started, let us look at the
Makefile for an imaginary small
fptools project, small.
Each project in fptools has its own directory
in FPTOOLS_TOP
, so the
small project will have its own directory
FPOOLS_TOP/small/
. Inside the
small/ directory there will be a
Makefile, looking something like
this:
# Makefile for fptools project "small" TOP = .. include $(TOP)/mk/boilerplate.mk SRCS = $(wildcard *.lhs) $(wildcard *.c) HS_PROG = small include $(TOP)/target.mk
this Makefile has three sections:
The first section includes
[1]
a file of “boilerplate” code from the level
above (which in this case will be
FPTOOLS_TOP
/mk/boilerplate.mk).
As its name suggests, boilerplate.mk
consists of a large quantity of standard
Makefile code. We discuss this
boilerplate in more detail in Section 8.5.
Before the include statement, you
must define the make variable
TOP
to be the directory containing the mk
directory in which the boilerplate.mk
file is. It is not OK to simply say
include ../mk/boilerplate.mk # NO NO NO
Why? Because the boilerplate.mk
file needs to know where it is, so that it can, in turn,
include other files. (Unfortunately,
when an included file does an
include, the filename is treated relative
to the directory in which gmake is being
run, not the directory in which the
included sits.) In general,
every file foo.mk assumes
that
$(TOP)
/mk/foo.mk
refers to itself. It is up to the
Makefile doing the
include to ensure this is the case.
Files intended for inclusion in other
Makefiles are written to have the
following property: after
foo.mk is included,
it leaves TOP
containing the same value
as it had just before the include
statement. In our example, this invariant
guarantees that the include for
target.mk will look in the same
directory as that for boilerplate.mk.
The second section defines the following standard
make variables:
SRCS
(the source files from which is to be built), and
HS_PROG
(the executable binary to be built). We will discuss in
more detail what the “standard variables” are,
and how they affect what happens, in Section 8.7.
The definition for SRCS
uses the
useful GNU make construct
$(wildcard $pat$),
which expands to a list of all the files matching the
pattern pat in the current directory. In
this example, SRCS
is set to the list
of all the .lhs and
.c files in the directory. (Let's
suppose there is one of each, Foo.lhs
and Baz.c.)
The last section includes a second file of standard code, called target.mk. It contains the rules that tell gmake how to make the standard targets (Section 7.8). Why, you ask, can't this standard code be part of boilerplate.mk? Good question. We discuss the reason later, in Section 8.4.
You do not have to include the target.mk file. Instead, you can write rules of your own for all the standard targets. Usually, though, you will find quite a big payoff from using the canned rules in target.mk; the price tag is that you have to understand what canned rules get enabled, and what they do (Section 8.7).
In our example Makefile, most of the work is done by the two included files. When you say gmake all, the following things happen:
gmake figures out that the object files are Foo.o and Baz.o.
It uses a boilerplate pattern rule to compile Foo.lhs to Foo.o using a Haskell compiler. (Which one? That is set in the build configuration.)
It uses another standard pattern rule to compile Baz.c to Baz.o, using a C compiler. (Ditto.)
It links the resulting .o files
together to make small, using the Haskell
compiler to do the link step. (Why not use
ld? Because the Haskell compiler knows
what standard libraries to link in. How did
gmake know to use the Haskell compiler to
do the link, rather than the C compiler? Because we set the
variable HS_PROG
rather than
C_PROG
.)
All Makefiles should follow the above three-section format.
Larger projects are usually structured into a number of sub-directories, each of which has its own Makefile. (In very large projects, this sub-structure might be iterated recursively, though that is rare.) To give you the idea, here's part of the directory structure for the (rather large) GHC project:
$(FPTOOLS_TOP)/ghc/ Makefile mk/ boilerplate.mk rules.mk docs/ Makefile ...source files for documentation... driver/ Makefile ...source files for driver... compiler/ Makefile parser/...source files for parser... renamer/...source files for renamer... ...etc...
The sub-directories docs,
driver, compiler, and
so on, each contains a sub-component of GHC, and each has its
own Makefile. There must also be a
Makefile in
$(FPTOOLS_TOP)
/ghc.
It does most of its work by recursively invoking
gmake on the Makefiles
in the sub-directories. We say that
ghc/Makefile is a non-leaf
Makefile, because it does little
except organise its children, while the
Makefiles in the sub-directories are all
leaf Makefiles. (In
principle the sub-directories might themselves contain a
non-leaf Makefile and several
sub-sub-directories, but that does not happen in GHC.)
The Makefile in ghc/compiler is considered a leaf Makefile even though the ghc/compiler has sub-directories, because these sub-directories do not themselves have Makefiles in them. They are just used to structure the collection of modules that make up GHC, but all are managed by the single Makefile in ghc/compiler.
You will notice that ghc/ also contains a directory ghc/mk/. It contains GHC-specific Makefile boilerplate code. More precisely:
ghc/mk/boilerplate.mk is included at the top of ghc/Makefile, and of all the leaf Makefiles in the sub-directories. It in turn includes the main boilerplate file mk/boilerplate.mk.
ghc/mk/target.mk is included at the bottom of ghc/Makefile, and of all the leaf Makefiles in the sub-directories. It in turn includes the file mk/target.mk.
So these two files are the place to look for GHC-wide customisation of the standard boilerplate.
Every Makefile includes a boilerplate.mk file at the top, and target.mk file at the bottom. In this section we discuss what is in these files, and why there have to be two of them. In general:
boilerplate.mk consists of:
Definitions of millions of
make variables that
collectively specify the build configuration. Examples:
HC_OPTS
,
the options to feed to the Haskell compiler;
NoFibSubDirs
,
the sub-directories to enable within the
nofib project;
GhcWithHc
,
the name of the Haskell compiler to use when compiling
GHC in the ghc project.
Standard pattern rules that tell gmake how to construct one file from another.
boilerplate.mk needs to be included at the top of each Makefile, so that the user can replace the boilerplate definitions or pattern rules by simply giving a new definition or pattern rule in the Makefile. gmake simply takes the last definition as the definitive one.
Instead of replacing boilerplate definitions, it is also quite common to augment them. For example, a Makefile might say:
SRC_HC_OPTS += -O
thereby adding “-O
” to
the end of
SRC_HC_OPTS
.
target.mk contains make rules for the standard targets described in Section 7.8. These rules are selectively included, depending on the setting of certain make variables. These variables are usually set in the middle section of the Makefile between the two includes.
target.mk must be included at the end (rather than being part of boilerplate.mk) for several tiresome reasons:
gmake commits target and dependency lists earlier than it should. For example, target.mk has a rule that looks like this:
$(HS_PROG) : $(OBJS) $(HC) $(LD_OPTS) $< -o $@
If this rule was in
boilerplate.mk then
$(HS_PROG)
and
$(OBJS)
would not have their final values at the moment
gmake encountered the rule. Alas,
gmake takes a snapshot of their
current values, and wires that snapshot into the rule.
(In contrast, the commands executed when the rule
“fires” are only substituted at the moment
of firing.) So, the rule must follow the definitions
given in the Makefile itself.
Unlike pattern rules, ordinary rules cannot be overriden or replaced by subsequent rules for the same target (at least, not without an error message). Including ordinary rules in boilerplate.mk would prevent the user from writing rules for specific targets in specific cases.
There are a couple of other reasons I've forgotten, but it doesn't matter too much.
If you look at
$(FPTOOLS_TOP)
/mk/boilerplate.mk
you will find that it consists of the following sections, each
held in a separate file:
is the build configuration file we discussed at length in Section 7.4.
defines make variables for pathnames and file lists. This file contains code for automatically compiling lists of source files and deriving lists of object files from those. The results can be overriden in the Makefile, but in most cases the automatic setup should do the right thing.
The following variables may be set in the Makefile to affect how the automatic source file search is done:
Set to a list of directories to search in addition to the current directory for source files.
Set to a list of source files (relative to the current directory) to omit from the automatic search. The source searching machinery is clever enough to know that if you exclude a source file from which other sources are derived, then the derived sources should also be excluded. For example, if you set EXCLUDED_SRCS to include Foo.y, then Foo.hs will also be excluded.
Set to a list of extra source files (perhaps in directories not listed in ALL_DIRS) that should be considered.
The results of the automatic source file search are placed in the following make variables:
All source files found, sorted and without duplicates, including those which might not exist yet but will be derived from other existing sources. SRCS can be overriden if necessary, in which case the variables below will follow suit.
all Haskell source files in the current directory, including those derived from other source files (eg. Happy sources also give rise to Haskell sources).
Object files derived from HS_SRCS.
Interface files (.hi files) derived from HS_SRCS.
All C source files found.
Object files derived from C_SRCS.
All script source files found (.lprl files).
"object" files derived from SCRIPT_SRCS (.prl files).
All hsc2hs source files (.hsc files).
All happy source files (.y or .hy files).
the concatenation of $(HS_OBJS), $(C_OBJS), and $(SCRIPT_OBJS).
Any or all of these definitions can easily be overriden by giving new definitions in your Makefile.
What, exactly, does paths.mk consider a "source file" to be? It's based on the file's suffix (e.g. .hs, .lhs, .c, .hy, etc), but this is the kind of detail that changes, so rather than enumerate the source suffices here the best thing to do is to look in paths.mk.
defines make variables for option
strings to pass to each program. For example, it defines
HC_OPTS
,
the option strings to pass to the Haskell compiler. See
Section 8.6.
defines standard pattern rules—see Section 8.6.
Any of the variables and pattern rules defined by the boilerplate file can easily be overridden in any particular Makefile, because the boilerplate include comes first. Definitions after this include directive simply override the default ones in boilerplate.mk.
The file suffix.mk defines standard pattern rules that say how to build one kind of file from another, for example, how to build a .o file from a .c file. (GNU make's pattern rules are more powerful and easier to use than Unix make's suffix rules.)
Almost all the rules look something like this:
%.o : %.c $(RM) $@ $(CC) $(CC_OPTS) -c $< -o $@
Here's how to understand the rule. It says that
something.o (say
Foo.o) can be built from
something.c
(Foo.c), by invoking the C compiler (path
name held in $(CC)
), passing to it
the options $(CC_OPTS)
and
the rule's dependent file of the rule
$< (Foo.c in
this case), and putting the result in the rule's target
$@ (Foo.o in this
case).
Every program is held in a make
variable defined in mk/config.mk—look
in mk/config.mk for the complete list. One
important one is the Haskell compiler, which is called
$(HC)
.
Every program's options are are held in a
make variables called
<prog>_OPTS
. the
<prog>_OPTS
variables are
defined in mk/opts.mk. Almost all of them
are defined like this:
CC_OPTS = $(SRC_CC_OPTS) $(WAY$(_way)_CC_OPTS) $($*_CC_OPTS) $(EXTRA_CC_OPTS)
The four variables from which
CC_OPTS
is built have the following
meaning:
SRC_CC_OPTS
:options passed to all C compilations.
WAY_<way>_CC_OPTS
:options passed to C compilations for way
<way>. For example,
WAY_mp_CC_OPTS
gives options to pass to the C compiler when compiling way
mp. The variable
WAY_CC_OPTS
holds
options to pass to the C compiler when compiling the
standard way. (Section 8.9 dicusses
multi-way compilation.)
<module>_CC_OPTS
:options to pass to the C compiler that are specific
to module <module>. For example,
SMap_CC_OPTS
gives the
specific options to pass to the C compiler when compiling
SMap.c.
EXTRA_CC_OPTS
:extra options to pass to all C compilations. This is intended for command line use, thus:
gmake libHS.a EXTRA_CC_OPTS="-v"
target.mk contains canned rules for all the standard targets described in Section 7.8. It is complicated by the fact that you don't want all of these rules to be active in every Makefile. Rather than have a plethora of tiny files which you can include selectively, there is a single file, target.mk, which selectively includes rules based on whether you have defined certain variables in your Makefile. This section explains what rules you get, what variables control them, and what the rules do. Hopefully, you will also get enough of an idea of what is supposed to happen that you can read and understand any weird special cases yourself.
HS_PROG
.If HS_PROG
is defined,
you get rules with the following targets:
itself. This rule links
$(OBJS)
with the Haskell
runtime system to get an executable called
$(HS_PROG)
.
installs
$(HS_PROG)
in
$(bindir)
.
C_PROG
is similar to HS_PROG
,
except that the link step links
$(C_OBJS)
with the C
runtime system.
LIBRARY
is similar to HS_PROG
,
except that it links
$(LIB_OBJS)
to make the
library archive $(LIBRARY)
,
and install installs it in
$(libdir)
.
LIB_DATA
…
LIB_EXEC
…
HS_SRCS
, C_SRCS
.If HS_SRCS
is defined
and non-empty, a rule for the target
depend is included, which generates
dependency information for Haskell programs. Similarly
for C_SRCS
.
All of these rules are “double-colon” rules, thus
install :: $(HS_PROG) ...how to install it...
GNU make treats double-colon rules as
separate entities. If there are several double-colon rules for
the same target it takes each in turn and fires it if its
dependencies say to do so. This means that you can, for
example, define both HS_PROG
and
LIBRARY
, which will generate two rules for
install. When you type gmake
install both rules will be fired, and both the program
and the library will be installed, just as you wanted.
In leaf Makefiles the variable
SUBDIRS
is undefined. In non-leaf Makefiles,
SUBDIRS
is set to the list of
sub-directories that contain subordinate
Makefiles. It is up to you to
set SUBDIRS
in the
Makefile. There is no automation
here—SUBDIRS
is too important to
automate.
When SUBDIRS
is defined,
target.mk includes a rather neat rule for
the standard targets (Section 7.8 that
simply invokes make recursively in each of
the sub-directories.
These recursive invocations are guaranteed to
occur in the order in which the list of directories is specified
in SUBDIRS
. This guarantee can
be important. For example, when you say gmake
boot it can be important that the recursive invocation
of make boot is done in one sub-directory
(the include files, say) before another (the source files).
Generally, put the most independent sub-directory first, and the
most dependent last.
We sometimes want to build essentially the same system in several different “ways”. For example, we want to build GHC's Prelude libraries with and without profiling, so that there is an appropriately-built library archive to link with when the user compiles his program. It would be possible to have a completely separate build tree for each such “way”, but it would be horribly bureaucratic, especially since often only parts of the build tree need to be constructed in multiple ways.
Instead, the target.mk contains some clever magic to allow you to build several versions of a system; and to control locally how many versions are built and how they differ. This section explains the magic.
The files for a particular way are distinguished by munging the suffix. The "normal way" is always built, and its files have the standard suffices .o, .hi, and so on. In addition, you can build one or more extra ways, each distinguished by a way tag. The object files and interface files for one of these extra ways are distinguished by their suffix. For example, way mp has files .mp_o and .mp_hi. Library archives have their way tag the other side of the dot, for boring reasons; thus, libHS_mp.a.
A make variable called
way
holds the current way tag.
way
is only ever set on the
command line of gmake (usually in
a recursive invocation of gmake by the
system). It is never set inside a
Makefile. So it is a global constant for
any one invocation of gmake. Two other
make variables,
way_
and
_way
are immediately derived from
$(way)
and never altered. If
way
is not set, then neither are
way_
and
_way
, and the invocation of
make will build the "normal
way". If way
is set, then the other
two variables are set in sympathy. For example, if
$(way)
is “mp”,
then way_
is set to
“mp_” and
_way
is set to
“_mp”. These three variables are
then used when constructing file names.
So how does make ever get recursively
invoked with way
set? There are two ways
in which this happens:
For some (but not all) of the standard targets, when
in a leaf sub-directory, make is
recursively invoked for each way tag in
$(WAYS)
. You set
WAYS
in the
Makefile to the list of way tags you
want these targets built for. The mechanism here is very
much like the recursive invocation of
make in sub-directories (Section 8.8). It is up to you to set
WAYS
in your
Makefile; this is how you control what
ways will get built.
For a useful collection of targets (such as
libHS_mp.a,
Foo.mp_o) there is a rule which
recursively invokes make to make the
specified target, setting the way
variable. So if you say gmake
Foo.mp_o you should see a recursive
invocation gmake Foo.mp_o way=mp,
and in this recursive invocation the pattern rule
for compiling a Haskell file into a .o
file will match. The key pattern rules (in
suffix.mk) look like this:
%.$(way_)o : %.lhs $(HC) $(HC_OPTS) $< -o $@Neat, eh?
You can invoke make with a particular way setting yourself, in order to build files related to a particular way in the current directory. eg.
$ make way=pwill build files for the profiling way only in the current directory.
Sometimes the canned rule just doesn't do the right thing.
For example, in the nofib suite we want the
link step to print out timing information. The thing to do here
is not to define
HS_PROG
or
C_PROG
, and instead define a special
purpose rule in your own Makefile. By
using different variable names you will avoid the canned rules
being included, and conflicting with yours.
[1] | One of the most important features of GNU make that we use is the ability for a Makefile to include another named file, very like cpp's #include directive. |