attempting to build KadathThorn assumes a git repository

[Roland Haas]

Attenmpting to build KadathThorn fails when no git repository is present:

cd ./ ; /Applications/Xcode.app/Contents/Developer/usr/bin/make CCTK_TARGET=make.checked TOP=/Users/rhaas/ET_Next/configs/sim CONFIG=/Users/rhaas/ET_Next/configs/sim/config-data SRCDIR=/Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src THORN=KadathThorn USESTHORNS="Boost FFTW3 GSL LAPACK" -f /Users/rhaas/ET_Next/lib/make/make.subdir
/Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src/build.sh
+ set -e
++ dirname /Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src/build.sh
+ SRCDIR=/Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src
+ echo 'KadathThorn - BUILD srcdir: /Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src'
KadathThorn - BUILD srcdir: /Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src
+ pushd /Users/rhaas/ET_Next/arrangements/Fuka/KadathThorn/src
~/ET_Next/arrangements/Fuka/KadathThorn/src ~/ET_Next/configs/sim/build/KadathThorn
++ realpath ../
+ BASEDIR_ABS=/Users/rhaas/ET_Next/repos/KadathThorn
++ basename /Users/rhaas/ET_Next/repos/KadathThorn
+ THORN=KadathThorn
+ echo 'KadathThorn - BUILD thornname: KadathThorn'
KadathThorn - BUILD thornname: KadathThorn
+ NAME=fuka
+ git submodule init

simfactory does not copy git repositories to remote hosts when used with sim sync.

Building thorns cannot depend on the full git repository being present since this will break with sim sync, which is the officially supported workflow when compiling on clusters and also when trying to build from Formaline tarballs. It also means that the source code is not self contained, which we would like to avoid if possible.

Unfortunately I had originally assumed that GetComponents would check out the submodule automatically, since Kranc also has a submodule (for its TestThorns submodule). This does however not seem to be the case. So the only fix, right now, is to add fuka as an explicit checkout to the thornlist. Something like this is needed:

!TARGET = $ROOT/repos/KadathThorn/src/fuka
!TYPE   = git
!URL    = https://bitbucket.org/fukaws/fuka
!CHECKOUT = Cmake build_debug build_release codes eos include install_par.sh install_seq.sh src src_par src_seq

or, more typical, a tar file with the source could be included in KadathThorn. Which also helps avoid the need to create a ET_YYYY_MM branch in fuka (since it needs to be an identifiable version)

It may be also worthwhile consider puttin KadathThorn, which really is no thorn with any Cactus code, but an ExternalLibrary like wrapper thorn, into ExternalLibraries (same as LORENE).

[tootle]

@Roland Haas I’m still working on a solution to this. The issue with the proposed (though not advised) solution of having a separate checkout to populate the src/fuka directory results in a race condition on the filesystem since a working tree becomes locked when being used by GetComponents even without --parallel.

I also verified, at least on my system, that the submodules of Kranc are not automatically updated via GetComponents so possibly this is just a remnant on your system.

Is it the case that sim-sync only downloads the repos from the internet and pushes them to a remote system and only on the remote system are the build scripts executed? Within the confines of the use case, I don’t see much of an alternative than to switch to a tarball. I’ll think on it more though.

[Roland Haas]

GetComponents has code to avoid race conditions. If that does not trigger then there is something wrong.

[Roland Haas]

Alright, something is wrong. I will need to see what it actually uses to check dependency. Likely the TARGET/CHECKOUT and not the NAME data.

[Roland Haas]

Ok, there is definitely (at least) one bug in GetComponents. Basically it does not handle symbolic links in paths correctly. I do not want to touch GetComponents this close to the release, so here’s a workaround using a crazy path:

# FUKA initial data thorns
# the crazy path works round a bug in GetComponents that does not handly
# symbolic links and ".." correctly
!TARGET = $ARR/Fuka/KadathThorn/src/fuka/../../../../../repos/KadathThorn/src/fuka
!TYPE   = git
!URL    = https://bitbucket.org/fukaws/fuka
!CHECKOUT = Cmake build_debug build_release codes eos include install_par.sh install_seq.sh src src_par src_seq

‌

[tootle]

@Roland Haas Thanks for finding the work around! Based on these changes, I needed to modify the build script for the `KadathThorn`.

In 4cca4c8 I removed the manual git pull that took place before and updated copying files correctly to the scratch directory based on the new symbolic links

In cc91146 I removed the submodule since this is no longer needed.

I have since tested the build procedure and everything worked on my end. Let me know if there are any further issues.