1. Einstein Toolkit Avatar Einstein Toolkit
  2. Einstein Toolkit
  3. tickets
  4. Issues

include Ellipitca (reader) in Einstein Toolkit

Issue #2619 open
Roland Haas created an issue

Elliptica is solver for compact binary systems.

It has been presented in the ET 2022 school at UIdaho: https://einsteintoolkit.github.io/et2022uidaho/lectures/20-Talk4/index.html (recording) and in a paper: https://arxiv-export1.library.cornell.edu/abs/2109.14511

Comments (37)

  7. Samuel Cupp

    Hey @Alireza R. . Do you plan for this to be included in the upcoming release? If so, please let us know. If so, it would be great if you or someone else working on this could attend the weekly call this Thursday, as we’ll be deciding what to include in the release during that call.

  8. Alireza R.

    Hey @Samuel Cupp .Yes, I’ll plan to do so. Sure, me and @Aviral Prakash will be attending. Thanks!

  10. Alireza R.

    Hi @Roland Haas ,

    I have a question: we need to link a library for the reader. I don’t know how where and how to specify the path and name to the library. Please give me an example file. Thanks!

  11. Roland Haas reporter

    Ideally the library would become an ExternalLibrary (same as say LORENE https://github.com/einsteinToolkit/externalLibraries-lorene) and the reader thorn would REQUIRE it (like Meudon_Bin_NS https://bitbucket.org/einsteintoolkit/einsteininitialdata/src/master/Meudon_Bin_NS/ in file configuration.ccl).

    This has the advantage that multiple different ID readers can use the same ExternalLibrarry eg for BBH or BNS or BHNS initial data. Those should not be all handled by a single thorn since eg the BNS or BHNS reader will need to provide data in thorn HydroBase which would not be active for a pure vacuum run.

    If the library cannot be incorporated as an ExternalLibrary (eg b/c it requires use of C++ features not allowed in C++11 or Fortran newer than Fortran 2008) then you could have the ID thorns take FOO_DIR etc option list variables the way an ExternalLibrary thorn would do. However this would mean that a user who would like to compile in multiple different ID thorns all using Elliptica’s library would have to, potentially, set multiple option list variables with identical values.

    On the technical level, you can tell the build system which library to link against and the path to find is using configuration.ccl by adding a (http://einsteintoolkit.org/usersguide/UsersGuide.html#x1-53000C1.2.3) block

    PROVIDES <My_Capability>
{
  SCRIPT <My_ConfigScript>
  LANG <My_Language>
}

    where My_ConfigScript has to produce output that adheres to the syntax described here: http://einsteintoolkit.org/usersguide/UsersGuide.html#x1-194000D2.5.1

    For a library that cannot be compiled from source you can pattern this after what is done in ExternalLbraries/pthreads (this is fine even if you have actual Cactus code in src/) which looks like what you can find here (https://github.com/EinsteinToolkit/ExternalLibraries-pthreads): https://github.com/EinsteinToolkit/ExternalLibraries-pthreads/blob/master/src/detect.sh The important bit are the MAKE_DEFINITION (to set Makefile variables that other thorns can use) and INCLUDE_DIRECTORY, LIBRARY_DIRECTORY, LIBRARY (to tell Cactus what -L, -I and -l options to produce) blocks:

    echo "BEGIN MAKE_DEFINITION"
echo "PTHREADS_DIR            = ${PTHREADS_DIR}"
echo "PTHREADS_INC_DIRS       = ${PTHREADS_INC_DIRS}"
echo "PTHREADS_LIB_DIRS       = ${PTHREADS_LIB_DIRS}"
echo "PTHREADS_LIBS           = ${PTHREADS_LIBS}"
echo "END MAKE_DEFINITION"

echo 'INCLUDE_DIRECTORY $(PTHREADS_INC_DIRS)'
echo 'LIBRARY_DIRECTORY $(PTHREADS_LIB_DIRS)'
echo 'LIBRARY           $(PTHREADS_LIBS)'

    Hope this helps.

  12. Pedro Espino

    Hi Roland, thank you for the guidance. We were able to write an ExternalLibraries thorn for the library in question and to compile the toolkit with this thorn by specifying the aforementioned options (INC_DIRS, LIB_DIRS, and LIBS). However, when we try to compile the separate reader thorn along with this ExternalLibrary, the compilation is facing an issue in the linking stage. Specifically, we would compile the reader code (currently being ported to a thorn) using this library with the following command:

    gcc -o idr idr.c -fopenmp -I/path/to/INC_DIRS -L/path/to/LIB_DIRS -lelliptica_id_reader -Wl,-rpath,/path/to/LIB_DIRS -lm

    We have tried including the linker options (-lelliptica_id_reader -Wl,-rpath,/path/to/LIB_DIRS -lm) in the LDFLAGS and LIBS environment variables during compilation, but we are still running into a compilation error during the linking stage. Is there some way to have the toolkit use these linking options to compile the reader thorn specifically?

  14. Pedro Espino

    optionlist compilation file used in compiling the Elliptica_BHNS (to read and interpolate Elliptica BHNS) and ExternalLibraries-EIDR (to distribute Elliptica external library) thorns

  17. Pedro Espino

    I tried to compile with the above options in the optionlist, but am still not able to compile successfully. I’ve attached my optionlist (expanse.cfg) and compilation log output (make.log) for reference, along with the configuration.ccl and associated detect.sh script for the ExternalLibrary. Just to be clear, I am able to compile successfully when just including the ExternalLibrary (so the paths are apparently set correctly), but the compilation fails with an

    undefined reference to `elliptica_id_reader_init(char const*, char const*)'

    when trying to call functions that are defined in the ExternalLibrary (such as elliptica_id_reader_init). Do I need to specify the above options (LIBDIRS and LIBS) in the configuration.ccl and associated script of the separate reader thorn which relies on the ExternalLibrary?

  19. Beyhan Karakaş

    I can confirm that ExternalLibrary for Elliptica provided by @Alireza R. does no harm and is under review. Elliptica ID reader thorn are currently missing, was informed to be provided until the weekend.

    ‌

  20. Roland Haas reporter

    When this was discussed in the ET calls then the argument for inclusion was that there would be a library of initial data files available. Right now I cannot find those anywhere. Are they available?

    An alternative would be to have the elliptica initial data solver (not just the reader) available, though right now it is unclear where it is located or if it is available.

    Having either one available was a condition for inclusion in the ET (open source astro code): http://lists.einsteintoolkit.org/pipermail/users/2022-July/008615.html

  21. Roland Haas reporter

    Looking at thornlist fragment included in in manifest:

    # Elliptic ID reader
!TARGET   = $ARR
!TYPE     = git
!URL      = https://github.com/aup1075/EllipticaID.git
!REPO_BRANCH = BHNS_reader
!REPO_PATH = $2
!CHECKOUT =
EllipticaID/Elliptica_BHNS

# required lib for Elliptica BHNS ID reader
!TARGET   = $ARR
!TYPE     = git
!URL      = https://github.com/rashti-alireza/Elliptica_ID_Reader.git
!REPO_BRANCH = Cactus
!REPO_PATH = ../$2
!CHECKOUT =
ExternalLibraries/Elliptica_ID_Reader

    then both use a non-default branch (not just not named “master” but actually not the default branch). This is ok as long as those branches are (in the “master” branch manifest) are the ones that will continue to be the ones that releases will be spun off from and are the ones that are actually used by the authors. That is they should not be branches created solely for this ticket.

    https://github.com/rashti-alireza/Elliptica_ID_Reader.git seems to be ok in that respect since it is the branch containing the Cactus ExternalLibrary like thorn and includes a tar file of the source (though it could have also been included in a subdirectory of the main branch [and include its own tar-file there] in the way RePrimAnd does).

    However https://github.com/aup1075/EllipticaID/tree/BHNS_reader has what looks like the same Elliptica_BHNS thorn also available in main. If the copy of the thorn is BHNS_reader is just a copy of the one in main then it would be preferable to track main where presumably the actual updates will happen. In particular in anticipation of a future inclusion of other types of readers. There is no harm in having extra thorns (or other code) in the repo and branch that is used with Cactus. The extra ones are just ignored as long as they are not actually listed in the thornlist (and are not subject to review).
    The added simplicity of not having to maintain multiple branches that need to be kept up to date with bugfixes and (possibly) improvements may well be worth the larger size on disk when checking out the repo.

  22. Alireza R.

    Thanks, Roland, for the comments. Regarding the Elliptica_ID_Reader repo., as you mentioned, there is a dedicated Cactus branch; hence, all future updates and features for Cactus will remain in this branch. For the EllipticaID repo. , following your comments, we merged everything into the main branch. Thanks!

  24. Roland Haas reporter

    To update the thornlist in manifest? Sure. I just gave you commit rights so you can also directly commit to manifest (just be a bit careful to not change more than the thing you would like to change, eg running a git diff origin/master..master before you push).

  25. Roland Haas reporter

    I changed manifest/master’s einsteintoolkit.th to refer to the default branch for EllipticaID (and verified that the BHNS reader in there agrees with the one in Elliptica_BHNS) and pushed the change (mostly so that those are used for testing on clusters).

  28. Alireza R.

    @Roland Haas

    ”Attached, please find the orbit comparisons between BAM and ETK codes. As far as the loading of the initial data goes, they match, i.e., the metric and hydro fields populated on the correct region of the grid and, as a result, orbiting in the expected direction. There is an offset for NS trajectories, and I believe the difference is related to the grid structure that we set in the parameter file.
    The deviation of BH trajectories is something that we are investigating now and requires more time to find out the best configuration for this new BHNS system. Please let me know what you think and if this is sufficient to include Elliptica in this release. Thanks!”

    ‌

    ‌

    ‌

  29. Samuel Cupp

    @Alireza R. why does the neutron star trajectory not start at the same point for both? That seems to be an error, is it not?

  30. Pedro Espino

    I can confirm that both objects begin at the correct locations (i.e., the same as BAM) on the grid. Below is a view of the trajectories on the equatorial plane for each evolution code (ET is EinsteinToolkit and BA is BAM). We’ve tried this test with both the VolumeIntegrals_GRMHD and NSTracker thorns for tracking the NS, and they result in similar results. I suspect the differences in trajectories are due to different gauge conditions. It is difficult to get a 1-to-1 overlap in the gauge conditions between BAM and ML_BSSN.

    ‌

  31. Alireza R.

    Additionally, the resolutions and grid structures of these testing runs between ETK and BAM are different, so we expect some variation at a later time. We plan to perform some gauge-independent tests after completing an entire evolution.

  33. Leonardo Werneck

    We recommend that the following tasks be completed for inclusion in a future release:

    1. Compare Einstein Toolkit simulation results that use LORENE and Elliptica initial data, as comparing results against those from a different infrastructure can be tricky and the results hard to interpret. There are at least two tests that could/should be done:
      1.a Read in both initial data onto the Carpet grid and show that the Hamiltonian and momentum constraints are initially comparable;
      1.b Perform two simulations in the Einstein Toolkit that differ only in the way the initial data is read, and show that e.g., the NS trajectories and the gravitational waves extracted from both simulations overlap. You can use the gallery example parameter file as a guide.
    2. Create a reasonably large (>10) initial data catalog and make it publicly available. The catalog should include, for example, initial data with different equations of state, initial separations, masses, and/or mass ratios. Without such a catalog, people would not be able to use the Elliptica reader.

    ‌

  37. Alireza R.

    I got sidetracked by other projects, I’ll try to get back to it by the end of this summer. Thanks!

  38. Log in to comment
Assignee
Type
enhancement
Priority
major
Status
open
Component
–
Milestone
–
Version
–
Votes
0
Watchers
6
Jira: the preferred issue tracker for Bitbucket. Join the team!