This file is meant to give an overview of how the GAP build system works. It is targeted at people who need to work on the build system itself (to extend it, fixes bugs in it, etc.). It should not be necessary to read this if all you want to do is compile GAP and work on the GAP library and kernel.
Note that this really is just an overview; for details, please refer to the comments inside the various parts of the build system.
In order to work on the build system, you need at least the following:
- GNU autoconf (we recommend 2.69 or later)
- GNU make
Note that we extensively use features provided by GNU make, so in general another version of make, such as BSD make, is not suitable.
If you are working with a fresh clone of the GAP repository, you need to
run the autogen.sh
script first, which will generate the configure
script. Afterwards, or if you are using a release version of GAP, you
can follow the standard procedure:
./configure
make
== Overview of the files constituting the GAP build system
-
autogen.sh
: sets up the build system; typically the first thing to run in a fresh clone of the GAP repository. It runsautoreconf
which in turn runs several tools includeaclocal
,autoheader
,autoconf
,libtoolize
-
configure
: generated byautogen.sh
fromconfigure.ac
. -
configure.ac
: the GNU autoconf source of our configure script. -
GNUmakefile
,GNUmakefile.in
: The former file is generated from the latter byconfigure
. It is the primary Makefile (GNU make prefers it overMakefile
). It only contains variables and vpath settings, and includesMakefile.rules
for the actual build rules. -
Makefile
: This is a placeholder file, and serves two purposes:- If the user runs
make
beforeconfigure
, it prints a warning. - If
configure
did run, butmake
is not GNU make, it produces a corresponding error message.
- If the user runs
-
Makefile.rules
: This is the core of the build system. If you want to add or remove a kernel C source file, you need to add or remove its name here and only here. -
bin/
: This directory is created for compatibility mode (see below). -
cnf/
: All files in this directory are part of the build system. -
extern/
: External libraries we bundle with GAP (such as GMP) are put in here. -
gen/
: Generated code (such asconfig.h
andgap_version.c
) is put into this directory. -
obj/
: All*.o
files are placed into this directory.
-
.deps/
directories contain*.d
files generated by the build system, and which are used to track dependencies, e.g. of C source files on header files. -
.libs/
directories are created by libtool. Please refer to the libtool documentation for details.
The old GAP build system had a concept of "configs" and "CONFIGNAME", which allowed you to build GAP in different configurations from a single set of sources. This is gone in the current build system. However, a similar goal can be achieved by using so-called "out-of-tree builds".
In the following and also in the files that make up the build system, "srcdir" refers to the directory containing the GAP files, i.e. it contains this README, the src and lib directories and more.
To create a new out-of-tree build, create a new directory anywhere in your filesystem. A typical setup places the out-of-tree dirs into subdirectories of a "build" directory inside the srcdir. So you might have directories
srcdir/build/default
srcdir/build/default32
srcdir/build/hpcgap
srcdir/build/hpcgap32
...
We will refer to this directory from now on as the "builddir".
To initialize the out-of-tree build, change into the builddir and execute the configure script from the srcdir, like this:
cd $builddir
$srcdir/configure
You can pass any additional options you like to configure, e.g. ABI=32
or --enable-hpcgap
.
Once the configure script has completed, you can run make
as usual,
and all the object files and the gap executable will be placed inside
builddir. Your srcdir will remain untouched.
Compatibility mode emulates the build environment of the old GAP build system in order to allow packages with kernel extensions to be used with the new build system unmodified. As such, it mainly exists to ease the transition between new and old build system, and the plan is to remove it once all packages have been adapted to the new build system. However, that is still far off.
Compatibility mode does the following things:
- create
sysinfo.gap
file in the build dir - create a symlink
sysinfo.gap-default$ABI
pointing atsysinfo.gap
- create a
bin/$GAPARCH/config.h
symlink pointing atgen/config.h
- create a
bin/$GAPARCH/gac
symlink pointing atgac
- create a
bin/$GAPARCH/src
symlink pointing at either$srcdir/src
or (for HPC-GAP build) at$srcdir/hpcgap/src
- for out-of-tree builds, it creates a
${builddir}/src/compiled.h
file - ...
For now, using compatibility mode is required if one wants to build the
kernel extension for most packages which have one. In the future, we will
provide an alternative way to do this, and also will extend gac
to
cleanly supported building kernel extensions.
The build system tracks depdencies between files, such as between C source and
header files, via *.d
files in .deps
directories below obj/
and gen/
.
These files are mostly generated by the compiler; for this, the compiler needs
to support the relevant flags (gcc, clang, icc all do so).
For a detailed explanation of a very similar scheme, see here: http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
One of the main features of the new build system is that it optionally allows to build HPC-GAP instead of plain GAP. HPC-GAP is an experimental fork of GAP which implements concurrent programming, multi-threading, etc..
The HPC-GAP kernel and library were forked from the GAP kernel and library and
developed semi-independently for several years, with occasional merges between
the two. In order to recombine the two, we merged the HPC-GAP fork into a
subdirectory hpcgap
of the GAP repository. Then, all files inside hpcgap
which were identical to their counterparts in the GAP repository were deleted
(e.g. hpcgap/src/ariths.c
was deleted as it was identical to src/ariths.c
).
The new build system was then modified to optionally allow building HPC-GAP
(when the --enable-hpcgap
flag is passed to the configure
script). For
this work, and the resulting HPC-GAP binary to work, two tricks are used:
-
In order to get the library to work right, HPC-GAP mode employs multiple GAP root paths. Specifically, the GAP kernel function
SySetGapRootPath
was modified so that for every root directoryFOO
that gets added, we first addFOO/hpcgap
to the list of root directories. -
Ward (a tool for scanning our C sources and inserting guard statements into it) is integrated as follows: Normally,
src/FOO.c
gets compiled toobj/FOO.o
(or ratherobj/FOO.lo
, as we use GNU libtool). With ward in the mix,src/FOO.c
is first turned intogen/FOO.c
by ward, and then the compiler turns the latter intoobj/FOO.lo
. For each conversion, dependencies are tracked viagen/.deps/FOO.d
resp.obj/.deps/FOO.d
files. In particular, ifsrc/FOO.c
is modified, then the dependency rules ingen/.deps/FOO.d
will ensure thatgen/FOO.c
is regenerated fromsrc/FOO.c
; and the dependency rules inobj/.deps/FOO.d
ensure thatobj/FOO.lo
is regenerated fromgen/FOO.c
There are many things that still need to be done in the new build system. For an overview, see <https://github.com/gap-system/gap/issues?q=is:open+is:issue+label:"build system">
The main open task is to add support for make install
. There are some
pitfalls to that, esp. when it comes to handling packages with kernel
extensions.