Overview

Compiler Wrapper
****************

The purpose of the compiler wrapper is to change compiler and linker
options before passing them on. This allows us to enforce consistent
options for all third-party C/C++ libraries even if they are not
correctly handling ``CFLAGS`` / ``LDFLAGS``.

Note: In principle, other compilers can be supported. However, currently
  only GNU compiler (gcc) can be used to build Sage. The compiler
  wrapper will not wrap anything for other compilers. Also, strictly
  speaking, we are wrapping both the GNU compiler (gcc) as well as
  binutils (ld).


Linker Options
==============

The following changes are made to linker options:

* Any rpath/runpath options that point outside of ``$SAGE_LOCAL/lib``
  are removed. Those binaries would not be relocatable to other
  machines. Also, system libraries ought to be found by the shared
  library search path defined in ``/etc/ld.so.conf``.

* The rpath is set to ``$ORIGIN/../lib``. For all sage binaries (in
  ``$SAGE_LOCAL/bin``), this expands to ``$SAGE_LOCAL/lib``. Thus, the
  Sage binaries find their libraries without having to resort to
  ``LD_LIBRARY_PATH``.

* An autoconf test detects whether the linker understands the ELF-
  specific ``--hash-style=both`` option. If yes, it is always added to
  the linker. This forces both gnu and sysv hashes in the ELF binary.


Compiler Options
================

All linker options discussed above are also passed to gcc and wrapped
in ``-Wl,<...>``. The ``-Wl`` instructs gcc to pass it through to the
linker. But gcc can be configured to call a ``ld`` that is not in the
path, so we cannot rely on the linker wrapper being called.

In addition, the following changes are made to compiler options:

* The ``cc`` wrapper actually calls ``gcc``. This should already be
  the case as cc just installs as symlink ``cc`` -> ``gcc``. But
  sometimes this link is dropped in manual gcc installs.

* On Itanium systems, gcc optimizations are limited to ``-O2`` and
  lower. Also, ``--funroll-loops`` is removed. This is done to work
  around bugs in the gcc optimizer on IA64.

* On gcc-4.4.1, optimizations are limited to ``-O1``. If you are using
  this compiler release, please consider updating to a more recent
  version. Gcc-4.4.5 was released on 2010-10-03.


Environment Variables
=====================

The following enviroment variables can be used to change the behaviour
of the wrapper:

* ``COMPILERWRAPPER_DEBUG=<yes>``

  If you set the ``COMPILERWRAPPER_DEBUG`` environment variable to any
  value, the wrapper will output the rewritten arguments to stderr
  before executing gcc. For example, here the wrapper on Itanium
  systems where is configured to remove ``-funroll-loops`` and limit
  the optimization to ``-O2``:

     [~]$ COMPILERWRAPPER_DEBUG=yes gcc -O3 -funroll-loops main.c
     ------- rewritten arguments --------------------
     Argument 0: /usr/bin/gcc
     Argument 1: -O2
     Argument 2: main.c

  Note how ``-O3`` was changed into ``-O2`` and ``-funroll-loops`` was
  removed. Also, note that the extra output will break configure
  scripts.

* ``COMPILERWRAPPER_LDFLAGS=<comma-separated list>``

  The comma-separated list of shared libraries will be added to the
  front of the linker command line. So any symbols defined in these
  libraries will take precedence over any other library that the
  application is linking against.

  So, for example, ``COMPILERWRAPPER_LDFLAGS=-La/b,-lY,-lZ`` will make
  sure that all binaries link to *libY*, *libZ*, and search for
  libraries in *./a/b* in addition to the usual search path.

     [~]$ COMPILERWRAPPER_LDFLAGS=-lm COMPILERWRAPPER_DEBUG=yes gcc main.c
     ------- rewritten arguments --------------------
     Argument 0: /usr/bin/gcc
     Argument 1: -lm
     Argument 2: main.c

* ``COMPILERWRAPPER_LD_LIBRARY_PATH=<pathlist>``

  If this environment variable is set, the compilerwrapper will
  overwrite the ``LD_LIBRARY_PATH`` (or ``DYLD_LIBRARY_PATH`` on OSX)
  environment variable with the value of
  ``COMPILERWRAPPER_LD_LIBRARY_PATH`` before executing any wrapped
  binary. This is useful for configure scripts that need a special
  library path set to run the compiled binaries, but where the library
  path would conflict with the system compiler/binutils.

     [~]$ export LD_LIBRARY_PATH=/somepath
     [~]$ export COMPILERWRAPPER_LD_LIBRARY_PATH=/otherpath
     [~]$ export COMPILERWRAPPER_DEBUG=yes
     [~]$ gcc main.c
     ------- Changed library path --------------------
     LD_LIBRARY_PATH now is /otherpath


Switching it On And Off
=======================

Using the ``wrapper <on|off>`` command, you can switch the compiler
wrapper on and off. This works by creating and removing, respectively,
symlinks ``$SAGE_LOCAL/bin/gcc`` -> ``$SAGE_LOCAL/bin/wrapper`` (and
similarly for other GCC/binutils executables).

      [~]$ which gcc
      [~]$ wrapper on
      [~]$ which gcc


Installation
============

The compiler wrapper is built automatically when you compile Sage.
However, you can also compile it manually and use it for any other
project. Or you could recompile it to temporarily use a different
compiler:

   [~]$ tar xzf compilerwrapper-1.1.tar.gz
   [~]$ cd compilerwrapper-1.1/
   [compilerwrapper-1.1]$ ./configure --prefix=/home/fred/local --with-ccpath=/usr/local/bin
   [compilerwrapper-1.1]$ make
   [compilerwrapper-1.1]$ make install

This will install the wrappers into ``/home/fred/local/bin``. The
wrappers will change some options and then call the original compiler
executable in ``/usr/local/bin``.

Important configure options are

* ``--prefix=<path>``: Install the wrapper and symlinks into
  ``<path>/bin``.

* ``--with-ccpath=<gccpath>``: Use the compiler executables in
  ``<ccpath>``. So, for example, the ``gcc`` wrapper calls
  ``<ccpath>/gcc`` internally. Defaults to ``/usr/bin``.

* ``--with-ldpath=<ldpath>``: Use the binutils binaries in
  ``<ldpath>``. For example, the ``ld`` wrapper calls ``<ldpath>/ld``
  internally. Defaults to ``<ccpath>``.

By default, the wrapper will call the gcc binaries ``gcc``, ``cc``,
``c99``, ``c89``, ``c++``, and ``g++`` through a wrapper of the same
name. You can, however, change the wrapper binary names as well as the
underlying wrapped gcc binary names. The wrapper names are controlled
by the standard program name transformations that autoconf offers:

* ``--program-prefix=PREFIX``: prepend ``PREFIX`` to the compiler
  wrapper names.

* ``--program-suffix=SUFFIX``: append ``SUFFIX`` to the compiler
  wrapper names.

* ``--program-transform-name=PROGRAM``: run sed ``PROGRAM`` on the
  compiler wrapper names.

The underlying gcc binary names can be modified by the analogous

* ``--with-gcc-transform-name=PROGRAM``: run sed ``PROGRAM`` on the
  underlying gcc binary names.

Here, ``PROGRAM`` always refers to a sed regex patterm seach/replace.
For example, ``--with-gcc-transform-name='s/^/i386-/'`` will make the
compiler wrapper call the underlying gcc binary named ``i386-gcc``.
The compiler wrapper will still call itself ``gcc`` unless you change
it with one of the ``--program-.*`` options.

There is also a fine manual which you are reading right now. Building
it requires ``sphinx-build`` and ``latexmk`` (and a working PDFLaTeX
install). By default, the documentation is not built. You can enable
it with the configure option

* ``--enable-doc``: Enable building of documentation


Techncial Details
*****************

Consider the following two simple source files. Our ``main.c`` file is

   int main(void)
   {
     int x;
     libraryfunction(x);
     return int(x==1);
   }

and ``library.c`` contains

   void libraryfunction(int *x)
   {
      x = 1;
   }

We want to build a shared library containing ``libraryfunction()`` and
dynamically link it with the executable, with the following twist: The
main program will be installed in the subdirectory ``./bin``, and the
shared library will be installed in the subdirectory ``./lib``. If we
do not tell the main executable where the library is, then it cannot
work. We could set ``LD_LIBRARY_PATH`` (or ``DYLD_LIBRARY_PATH`` on
OSX), but that will then force every program to search in that path
first, potentially breaking existing programs. Here is a more
targetted approach that will affect the executable we are building.

First, we compile the source files into object files:

   gcc -fPIC -c -o main.o main.c
   gcc -fPIC -c -o library.o library.c

This produces a ``main.o`` and a ``library.o`` file. Note that we
compile with ``-fPIC`` to produce position-independent code. How do we
turn them into an executable?


Dynamic Linking on Linux, Solaris, ...
======================================

On systems using the ELF object file format, the solution is to
compile the executable with an *rpath*. Then the dynamic linker will
first search there for shared libraries.

   gcc -shared -o libexample.so library.o
   gcc main.o -L. -lexample -o example -Wl,-rpath,'$ORIGIN/../lib'

The ``-shared`` argument tells gcc, unsurprisingly, that we build a
shared library. On the second line, we build the main executable
(called ``example``) and link it (``-l``) with the shared library.

Only the magical incantation ``-Wl,-rpath,'$ORIGIN/../lib'`` seem
suspicious. But all that it means is that ``-rpath $ORIGIN/../lib`` is
passed to the linker *ld*, setting the rpath.


Dynamic Linking on Mac OSX
==========================

On OSX, oddly enough, the library seach path of the executable is
specified in the shared library and not while linking the executable
itself:

   gcc -dynamiclib -Wl,-install_name,@executable_path/../lib/libexample.dylib -o libexample.dylib library.o
   gcc main.o -L. -lexample -o example


Execution
=========

Now we copy

* ``main`` -> ``./bin/main``

* ``libexample.{so,dylib}`` -> ``./lib/libexample.{so,dylib}``
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.