makefiles /

Filename Size Date modified Message
test
12 B
93 B
2.0 KB
16.8 KB
39.1 KB
6.3 KB
15.9 KB
3.0 KB
128 B
6.7 KB
1.9 KB
4.6 KB

Makefiles

A collection of Makefiles and related tools supporting the following features:

  • Simple conversion of README.rst files to README.html files using docutils.
  • A fairly complete automatic build system.

Note that subsets of these features are provided by various named branches in the repository.

Common Conventions

Each new component can provide the following (replacing * with the component name):

  • help.*: Component specific help (should simply be a list of targets and descriptions).
  • clean.*: Component specific clean commands.
  • debug.*: Component specific debug commands.

To hook these into the common functionality (provided by help.mk) these should be added to help.targets, clean.targets, and debug.targets. See rst2html.mk for a simple example.

RST2HTML

Requirements

Requires the rst2html.py script which is provided by the docutils package. (The script location can be customized by the RST2HTML variable.)

Usage

A simple use (I have started using this for all my projects) is to include a top-level Makefile of the form:

makefiles_dir = .makefiles
include $(makefiles_dir)/rst2html.mk

By default, this will convert README.rst files to README.html files in the same directory (you might like to ignore these in your .hgignore file for example). You can customize this behaviour with the following variables:

README (= README.rst):
Name of input files.
RST_HTML_INPUT:
Input directory. This will be searched using find "$(RST_HTML_INPUT)" -type f -name "$(README)".
RST_HTML_OUTPUT:
Output directory (will have the same directory structure as the input directory.)
RST2HTML (= rst2html.py):
Name of rst2html.py command.

Automatic Build System

Most of the files here are for an automatic build system using GNU make with the following behaviour:

  • Automatic location of source files and generation of prerequisites (using compiler flags).

  • All build products go in a separate folder (typically called build) that

    can be simply removed to clean up.

  • Optional flat or nested structure in this directory. (In the latter case,

    the executables are sym-linked to the top-level of the build directory.)

Typical usage is to install this package into a directory makefiles in your project, and then copy Makefile.example to the top level Makefile, editing as appropriate.

Components

makefiles/common.mk:
Factoring of common platform independent functionality such as generic compiler flags and common library locations (relative to $HOME for example.)
makefiles/platforms.mk:
Platform dependent overrides, library location etc. Right now this this is geared towards the computers I use, but should be easy to modify.
makefiles/rules.mk:
Customized rules to support output in the build directory, linking, etc. without using $(VPATH). Also has some Fortran 90 rules.
makefiles/distribute_setup.py:
Provided so that the top-level setup.py can use the new distribute project for distributing python.

These files should not have to be modified on a per-project basis (if you need to make a modification, try to make it general and let me know so I can include the modification here). All project-specific information should be specified in a Makefile at the top level of the project that is based on Makefile.example.

Limitations

  • Executable names must be the same as the source file containing main().

  • Executable names must not start with lib (we filter these out when generating the phony targets.)

  • There are no shortcuts for specifying library targets: you must include the full path of both the library and the object files such as:

    targets += $(build_dir)/libname.a($(build_dir)/obj/dirname/name.o)
    

Usage

My recommendation for using this project is to follow Rusty Klophaus' suggestions of Submodules and Subrepos Done Right: I clone this repository into my project, then version control all of the files with the project (since they are an integral part) but use his subhg command to keep the makefiles in sync with the bitbucket project. Thus, I do the following:

cd MyProject
hg clone ssh://hg@bitbucket.org/mforbes/makefiles .makefiles
cp .makefiles/Makefile.example Makefile
emacs Makefile    # Customize as required

# The next step is optional and only of use if you want to track changes in
# this project or contributed.  You will need to have the subhg script
# defined and your mercurial environment setup as described in Rusty's blog.
# If you do not want to do this, just remove the .hg directory:
#rm -rf .makefiles/.hg
cd .makefiles
subhg setup
cd ..

# Now add your changes, commit, etc.
hg add
hg commit -m "Added Makefile and .makefiles directory"

Simply place the three .mk files in a folder called makefiles (can be changed) and suitably modify Makefile.example as your Makefile in the top level directory. If you use the following structure, then all you need to change in the Makefile is the list of libraries required by defining the appropriate $(use_*) flags to true:

src/:
All code should go here.
src/mains/:
All code that defines main() should go here: one source file for each desired executable (of the same name).

Since I might update this, I would recommend that you use mercurial to checkout a version controlled copy of this project. It is probably a good idea to do this as a subrepository so that your project records the version being used. Here is how:

  • If you use http access to bitbucket then do something like this:

    cd project_dir
    hg clone https://bitbucket.org/mforbes/makefiles makefiles
    echo "makefiles = https://bitbucket.org/mforbes/makefiles" >> .hgsub
    hg add .hgsub
    hg com -m "Added makefiles as a subrepository."
    cp makefiles/Makefile.example Makefile
    emacs Makefile
    hg add Makefile
    hg com -m "Customized Makefile."
    
  • If you use ssh access to bitbucket (which has the advantage that you can use a key-pair for password-less log in) then do something like this:

    cd project_dir
    hg clone ssh://hg@bitbucket.org/mforbes/makefiles makefiles
    echo "makefiles = ssh://hg@bitbucket.org/mforbes/makefiles" >> .hgsub
    hg add .hgsub
    hg com -m "Added makefiles as a subrepository."
    cp makefiles/Makefile.example Makefile
    emacs Makefile
    hg add Makefile
    hg com -m "Customized Makefile."
    

Variables

Specifying your Build Structure

build_dir = build:
All output will be placed in this directory. You can simply remove this to clean up (and this is what make clean will do). Note: I sometimes allow environmental variables to change this so that I can build different versions of the project in different places (but I call all directories build* for easy cleaning).
exe_dir = $(build_dir):
Unless doing a flat build, all executables will be sym-linked here.
obj_dir = $(build_dir)/obj:
Unless doing a flat build, everything will be built in this directory with the same directory structure as in $(src_dir)
flat_build:
If true, then all source directories will be added to $(VPATH) and the directory names will be stripped from all object files and executable files. This means that the build directory will contain only files (no folders).

Specifying your Sources

src_dir:
If this is defined, then it will be assumed that all source files live under this directory. If not using a flat build, then this directory prefix will be stripped before placing files into the $(build_dir). If this is not defined, then it will be defined to be the directory from which make was run (and converted to an absolute path).
src_dirs:
These directories will be searched to find source files. If not defined, then all directories in $(src_dir) will be searched using find.
main_dirs:
Locations of sources containing main() functions. If this is defined, then all files with an extension in $(o_exts) will be added (without the extension) to $(targets).
targets:
List of targets: executable programs or libraries. Libraries should be specified as $(build_dir)/libname.a(file1.o, file2.o, ...) where the files in the parenthesis will be included in the library (archive). This will also be used to generate some automatic dependencies and in implicit rules, so it is important that the executable files have the same name as the corresponding .o file that contains the main function. (See also all_object_files below.)
makefiles_dir: (default makefiles)
Specifies the location of the *.mk files.

Flags

These are variables that are intended to be defined upon invocation, for example make DEBUG=true. This means that within the makefiles, the variable $(DEBUG) will be defined to the value true:

DEBUG:
Adds options like -g to the various flags and disables optimizations (see common.mk).
WALL:
If true, then no compiler warnings will be suppressed. Typically, I suppress some non-critical warnings in platform.mk. This will re-enable all warnings.

Any other variables that you define like make VAR=val will be treated as a macro to pass to the preprocessor. I.e. CPPFLAGS += -DVAR=val.

Libraries and Headers

The following may need to be modified if you use libraries that I have not explicitly coded (see Libraries below):

include_dirs:
List of directories containing header files that will be added to $(CPPFLAGS) with -I prepended.
lib_dirs:
List of directories containing libraries that will be added to $(LDLIBS) with -L prepended.
libraries:
List of libraries that will be added to $(LDLIBS) with the -l prefix. Note: if a library has name mylib then the library file will typically be called libmylib.a for a static library or libmylib.so for a dynamic library.

Compiler Options

warning_flags:

debug_flags:

optimize_flags:
Will be added to $(CFLAGS), $(CXXFLAGS), and $(FFLAGS). Also supported are cwarning_flags, cxxwarning_flags, and fwarning_flags; cdebug_flags, cxxdebug_flags, and fdebug_flags; and coptimize_flags, cxxoptimize_flags, and foptimize_flags which will be added to only the respective $(*FLAGS) variable.

Generated Lists

The following variables govern the automatic source-file and prerequisite generation functionality. These are typically set by common.mk based on a search of the files in the current source files, but these can be overridden if needed.

o_exts:
List of extensions used to generate object files. This is used to automatically generate the list of sources.
h_exts:
List of extensions used on header files. Used to determine $(include_dirs) automatically.
source_files:
List of source files. Typically this is generated automatically by finding all files with an extension in $(o_exts) found in the directories specified in $(src_dirs).
main_files:
List of object files containing main. Generated from $(main_dir) if defined. Used to generate $(targets).
all_object_files:
List of all object files obtained by combining $(main_files) and $(source_files) with the directories removed in the latter and the extensions replaced by .o. Note: since this includes all $(main_files), it should not be given to the linker. It is used for generating the dependencies only.
object_files:
List of object files to pass to the linker. Obtained from $(all_object_files) by removing $(main_files). These are treated as dependencies for all targets. Note that the associated object file in $(main_files) is implicitly included (and thus needs to have the same name as the executable) so that only one main symbol is defined per executable.
depend_files:
List of *.d files generated from $(all_object_files) by replacing .o with .d. These files will be included and are automatically generated by the compiler
generate_depends:
Should be set to the incantation (typically -MMD or -Wp,-MMD) to get your compilers to generate the local (no system) dependency *.d files automatically.

Libraries

I have provided some support for the following libraries. To use these, just set the appropriate flag to true:

use_blacs:

use_lapack:

use_scalapack:

use_fft:

use_pthread:

use_math:

use_armadillo:
C++ header library for linear algebra that uses the underlying LAPACK and BLAS operations. My fork of armadillo has a few extra functions.
use_gsl:
GNU Scientific Library.
use_boost:
BOOST C++ Libraries. I typically only use the headers.

Conventions

  • I use lowercase variable names. Names in CAPS refer to variables used by the implicit rules (see this by running make -p) or by compile options (such as DEBUG) meant to be defined in the make command line (such as make DEBUG=true).
  • Variables that end in _dirs are lists of directories. These do not contain -I or -L (which will be added using the $(addprefix) function).
  • Variables in the makefiles directory are generally set using ?= or += so that variables set by the user are maintained. (The ?= form only has effect if the variable was not previously defined.)
  • Variables starting with an underscore _* are "private". Don't use.

VPATH

One issue I struggled with was how to use $(VPATH) and the related vpath command. These allows make to locate dependencies (the former provides a general search path while the latter allows the user to specify specific paths for each file type). The main advantage of this approach is that you can use the implicit rules, and often do not need to add a new rule for a different compiler. The problem is that the implicit rules expect the source files and object files to be located in the same directory (or on $(VPATH)) which prevents us from organizing the $(build_dir). For example, if you have a program that is built from src/mains/prog.c, then the implicit rules will make build/prog.o and build/prog and you don't have much control. (There can also be simplifications in specifying perquisites, but as we will use the compilers to explicitly generate the dependencies, this is not relevant.)

Rules

To maintain control of the build directory and to simplify the structure of the makefiles, I have chosen to include a set of rules via rules.mk. I made these from the output of make -p by simply modifying the paths. Thus, the typical implicit C compiling rule looks like:

%.o: %.c
#  commands to execute (built-in):
      $(COMPILE.c) $(OUTPUT_OPTION) $<

I have modified this to have the form:

$(obj_dir)/%.o : $(src_dir)/%.c
      $(MKDIR) $(dir $@)
      $(COMPILE.c) $(OUTPUT_OPTION) $<
      $(SYMLINKEXE)

In this way, the source files in $(src_dir) are translated into object files in $(build_dir). The $(SYMLINKEXE) command will do the symlinks if needed for a non-flat build.

References

Although the final solution does not use the techniques described here, I leared much from Paul D. Smith's approach, (which might be useful for allowing target dependent output) and the gpwiki.

Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.