Cactus should not interpret environment variables as configuration options

Create issue
Issue #332 open
Ian Hinder created an issue

Currently, if you have the variable HDF5_DIR set in your shell startup file, Cactus will interpret this as a configuration option and will override whatever is set in your optionlist.

This behaviour occurs because the "make" utility allows you to specify options on the command line which are then converted into environment variables. This is a method for overriding options on the command line.

However, the fact that variables from the user's environment are also overriding the optionlist is very undesirable. It would be good if this could be fixed somehow.

This ticket is in response to #100.

Keyword:

Comments (5)

  1. Erik Schnetter
    • removed comment

    One good way around this would be to rewrite the top-level Cactus makefile in Python. This top-level makefile does not use any real make facilities; it only parses the command and then calls the corresponding "real" makefile in the lib/make directory.

    Rewriting this makefile should be straightforward. I began to do this some time ago; I attach the current state of my efforts.

  2. anonymous
    • removed comment

    Replacing the Makefile is not a bad idea, but I am not so sure using python would be the best option. Given how many problems we see with simfactory and different versions of python being installed (or not) one various systems, one would have to be very careful and test a lot. The logic isn't all that complicated, and the rest of the build system is in perl anyway - IHMO it would make most sense to have the global "Makefile" in perl as well.

  3. Roland Haas

    Rewriting the top level Makefile will not actually prevent Cactus from interpreting (and giving precedence to) ENV variables over what is in option lists. That behaviour is explicitly coded in the ProcessConfiguration subroutine in ib/sbin/ProcessConfiguration.pl (and more specifically in AmalgamateOptions in CSTUtils.pl).

    It is related to make indeed passes variables specified on its command line as ENV variables to its children, ie:

    make FOO=bar
    

    makes a variable FOO available to sub-shells. In particular he ExternalLibaries' use of configure.sh scripts relies then on (a subset those in @allowed_options in CST and ProcessConfiguration.pl) of those being passed as ENV variables which is where ENV{HDF5_DIR} messing up HDF5_DIR in an option list actually comes from.

    If one wants to use only command line options instead of all ENV variables as a source for those @allowed_options then one can parse MAKEFLAGS the way that lib/make/setup_configuration.pl does:

    # parse make command line options FOO=BAR
    # Set variables from makefile command line first
    my %MAKEFLAGS;
    my $commandline = $ENV{"MAKEFLAGS"};
    $commandline =~ s/^.*? -- //;
    while ($commandline =~ /^(.*)\s*\b(\w+)\s*=\s*(.*?)\s*$/)
    {
      $MAKEFLAG{$2} = $3;
      $commandline=$1;
    }
    

    and use MAKEFLAGS instead of ENV. Before calling any sub-shell in CST one then should remove all @allowed_options not in MAKEFLAGS from ENV.

    The ExternalLibraries' build.sh scripts (and the Makefile) gets HDF5_INSTALL_DIR and similar actually from bindings/Configuration/Capabilities/make.HDF5.defn where a make variable HDF5_INSTALL_DIR is set (based on configure.sh's output) which make will prefer over any HDF5_INSTALL_DIR in ENV.

    This may fix the issue, though its a bit fragile since make will still use ENV variables for variables not otherwise defined (or eg stored in config-data/make.config.defn).

    Branch rhaas/envopts implements what is described above.

  4. Log in to comment