When an external library is rebuilt, its dependencies should also be rebuilt

Create issue
Issue #41 open
Erik Schnetter created an issue

When an external library (such as e.g. BLAS) is rebuilt, then all its dependencies (such as e.g. LAPACK) should also be rebuilt. Cactus knows these dependencies via the configuration.ccl declarations, but these are not checked in the scripts that determine whether to rebuild.

Keyword:

Comments (2)

  1. Roland Haas
    • edited description
    • changed status to open

    There are three cases of dependencies:

    1. the library file itself, which does not require recompilation of the client since the linker will take care of using the new library automatically
    2. any header files generated, these are already tracked in the .d files created by the various XXX_DEPEND tools in lib/make/make.config.rules.in
    3. Fortran module files .mod. These are not currently tracked by anything since the F_DEPEND_MODULES functionality is limited to modules in the same thorn and sets up a dependency assuming that a module foo is provided by a source file foo.F90 which is not sufficient.

    For this last case one can however make use of the fact that gfortran accepts the same -M -MF options that gcc (or cpp) do to write out dependency files. These do track modules wherever they are but assume that the module must exist when gfortran is first called. For ExternalLibraries this will be the case since Cactus already ensures that thorns REQUIREing each other are build in the correct order (exactly for the reason to provide .mod files).

    An example usage looks like this:

    mkdir bing
    cat <<EOF >bing/foo.F90
    #include "bar.h"
    subroutine foo
      use bar
    
      implicit none
    
    end subroutine
    EOF
    cat >bar.F90 <<EOF
    module bar
    end module
    EOF
    gfortran -c bar.F90
    gfortran -J$PWD/.. -M -MF foo.F90.d foo.F90
    

    Thus the best way to achieve the goal in this ticket seems to be to extend F_DEPEND or F_DEPEND_MODULES to call F90 as well as FPP (since FPP may have a different search path for include files and / or may not be cpp but cpp.pl for example). Note that (as in the example) gfortran is happy with #include statements in F90 files but will balk at them in f90 files (which is the same as in Cactus). Alternatively one can call F90 on the preprocessed Fortran sources to avoid having both F90 and FPP set up (possibly conflicting) dependencies on header files (though the worst case scenario is too many compiles if a file that is not actually depended on is listed as a dependency).

  2. Roland Haas

    Not so easy after all. It seems as if gfortran ends up in a chicken and egg situation where in order to generate the dependency file for a F90 file which uses modules, those very modules need to already exist: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49149 . The gfortran devs do not seem to consider this a bug. The workaround mentioned in that ticket is some clever use of makefile targets that has make try and generate the dependency files again and again until it eventually succeeds. This makefiles (which generates its own sources) demonstrates the method:

    SOURCES := foo.f90 bar.f90 baz.f90
    
    .PHONY: all clean
    
    all: $(patsubst %.f90,%.o,$(SOURCES))
    
    clean:
        rm -f *.mod *.o *.d *.f90
    
    %.d: %.f90
        gfortran -cpp -M -MF $@ $<
    
    %.o: %.f90 %.d
        gfortran -c $<
    
    # include any gnerated dependency files, don't worry about ones you did not
    # find yet
    ifneq ($(MAKECMDGOALS),clean)
    -include $(patsubst %.f90,%.d,$(SOURCES))
    endif
    
    # demo files:
    define FOO_F90
    subroutine foosub
      use baz
    end subroutine
    endef
    foo.f90:
        $(file >$@, $(FOO_F90))
    
    define BAR_F90
    module bar
    end module
    endef
    bar.f90:
        $(file >$@, $(BAR_F90))
    
    define BAZ_F90
    module baz
    end module
    
    subroutine bazsub
      use bar
    end subroutine
    endef
    baz.f90:
        $(file >$@, $(BAZ_F90))
    

    with the important bit being the -include line.

    Running

    make clean ; make foo.o
    

    gives me:

    > make clean ; make foo.o
    rm -f *.mod *.o *.d *.f90
    gfortran -cpp -M -MF baz.d baz.f90
    baz.f90:5:6:
    
        5 |   use bar
          |      1
    Fatal Error: Cannot open module file ‘bar.mod’ for reading at (1): No such file or directory
    compilation terminated.
    gfortran -cpp -M -MF bar.d bar.f90
    gfortran -cpp -M -MF foo.d foo.f90
    gfortran -cpp -M -MF baz.d baz.f90
    gfortran -c foo.f90
    

    showing that make first tried to make baz.d, failed (b/c bar.mod does not yet exist) then tries the others and eventually succeeds building foo.o for me.

    More unfortunately the Cray compiler does not have a similar -M option it seems and using GNU to build dependencies while building with Cray seems dicey (in particular given that -M still generates the .mod files).

  3. Log in to comment