User:Shmget/Sweetshark5

gbuild: Meet the new boss (Same as the old boss)

This is the fifth post in a short series of blog posts about the new GNU make based build system that was integrated into the m96 milestone of the DEV300  codeline. It is covering gbuild commands and usage.

Welcome back to the little blog series about the new GNU make build system. After showing off some ANSI color eyecandy for the new build system it is time to have a look at the new build system and how to command it in the usual usecases (all commands assume  the shell to be in root directory of the module in question, if no  explicit cd command is given):

Some things changed from the old build system. Here is an overview:

no local output tree
The GNU make build system does not use a "local module output directory". All modules use a $WORKDIR (by default a directory named "workdir" in the platform directory in the $OUTDIR/solver) for intermediate files. This makes the source tree read-only for the migrated modules.

cleaning up of modules
build --prepare will not clear the $WORKDIR of files by migrated modules. However, calling make -sr clean in the module or in the $SRC_ROOT will.

current directory when starting make
Other than build.pl, one can not call make in any subdirectory to build the module. Either, one has to cd to the module root before calling make, or one has to explicitly give the makefile in the module  root to the make command: cd sw/source/core && make -srf  ../../Makefile.

changes in parallelization
The old build system used one dmake process per directory, while the new one is hooked into build.pl as one make process per module for now. Big modules like sw only use the parallelization by the second -P switch given to the build --all  command. As more and more modules get migrated the second -P switch in a build --all -P4 -- -P4  command will get more important. In the end -- after getting rid of build.pl -- only one make process will be used for the whole build and  the maximum number of jobs will be given to via the -j switch, thus eliminating the need for guesswork on how to distribute the parallelization over the two old systems.

precompiled debug headers
On Windows support for precompiled headers is also available on debug builds, resulting for example in a speedup of ~40% for an build of  module sw for debug builds.

no seperation of build and deliver
The new build system does not separate the build and deliver steps of a module. Since libraries are always linked against the solver/$OUTDIR this means that in module framework, where the library fwk is linking  against the library fwi, the library fwi will be copied to the solver/$OUTDIR  before linking the library fwk. This lifts the artificial dependency barriers introduced by modules, but also results in that building a  module always modifies the solver/$OUTDIR. It also avoids the confusion of building a module, but forgetting to deliver it.

no local module builds
One can not simply copy a module to "anywhere" and build it there. The build system will notice this and will bail out. And even when it would not bail out, it would ignore the copied module for anything but the  makefiles. It would still look for the files to build and to compile in the directories given in the variable $gb_REPOS as described in the post about multiple repository support. To provide a workaround for the rare usecase that one wants to build only one module with some quick or risky changes without changing the  solver, there is a setuplocal target available in gbuild. For example to do experimental stuff on the tools module one would: export gb_LOCALBUILDDIR=/tmp/myoootoolsexperiment cd tools make setuplocal # this will create a copy of the tools module and the solver at $gb_LOCALBUILDDIR and tune the build system to that location cd /tmp/myoootoolsexperiment/srcdir/tools     # hack away cd $SRC_ROOT/tools make removelocal # clears $gb_LOCALBUILDDIR and allows work directly on the source

This is an extension to the gbuild system (because it relies on rsync, which the gbuild core itself should not do) and thus can be found  in the extensions directory of the build system.

full dependencies
Migrated modules always have full dependencies thus changing one header in a low level module will trigger a rebuild of all objects using  that header. On Windows that means all headers except compiler headers and headers from platform, directx and Java SDK, on the other platforms  it means all headers.

Faster no-op builds
Checking that nothing (or almost nothing) needs to be rebuild is faster. On a sample system (Notebook with Core2Duo, 2 GHz) on Windows XP (anti virus software installed), rechecking that nothing needs to be  done for module sw takes 7 sec with a warm cache. On the same machine build.pl/dmake took 210 sec with the same "full" header dependencies.