#294 Merged at 6472d1d
  1. John Wise

This PR contains many bugfixes, optimizations, and additions to the ray tracing components.

  • New Feature: Calculating the optically-thin Lyman-Werner radiation field with a tree instead of a direct summation. I have run into a few instances that result in a crash, so I have embedded this new feature in the define LWTREE, which is specified at compile-time. The tree calculation is only used when there are >10 sources.

  • New Feature: Limit the ray tracing timestep by a sound crossing time of a cell on a user-defined level, RadiativeTransferTimestepVelocityLevel. Default: off.

  • New Feature: Added the distributed stellar feedback model from So et al. (2014), which is based on the distributed feedback model already in Enzo. This is method 13.

  • New Feature: Added the ability for star particles that are created by any method to become radiation sources. This is turned on with the parameter StarParticleRadiativeFeedback. Uses the existing parameter StarEnergyToStellarUV to convert stellar mass into UV emissivity. Default: OFF.

  • Improvement: Various MPI optimizations in the ray communication.

  • Improvement: Rays are deleted when its associated photo-ionization timescale drops below RadiativeTransferHubbleTimeFraction * t_Hubble, when considering the case where all photons are absorbed in a single cell. Below it was fixed to be tiny_number, resulting in wasted computations when the flux is low. This should only be used with ray merging.

  • Improvement: When calculating the ray tracing timestep, only use grids on the maximum level with radiation, instead of the overall maximum level.

  • Improvement: Restructured the looping, stored the photo-ionization cross-section, and eliminated some unnecessary calculations in the WalkPhotonPackage routine for a 10-15% speedup.

  • Improvement: Added performance timer fields to the ray tracing routines.

  • BUGFIX: In the ray tracing restart, do not consider sources that were created during the time interval, time_0-L_box/c -> time_0.

  • BUGFIX: Corrected the Lyman-Werner luminosities for radiating star cluster (Pop II) particles.

  • BUGFIX: Corrected an incorrect calculation of the destination grid, which resulted in "orphaned" photons.

  • BUGFIX: Eliminated some minor memory leaks.

  • BUGFIX: Do not interpolate the RaySegments fields, used with ray tracing load balancing.

Comments (16)

        1. John Wise author

          Thanks. I'll look at these failures to see if they're expected with all of these changes to the RT.

        2. John Wise author

          Just an update. It seems like the PhotonTest* tests weren't running on Jenkins. There were no outputs made at all. It runs fine on my laptop. I'll have to test the exact setup (i.e. show-config) that Jenkins uses.

          1. Brian OShea

            Following up on this: In running the test suite, I have several failed tests. The errors were:

            Tests that failed:

            PhotonShadowing__test_standard.test_standard: FAILURE (<type 'exceptions.AssertionError'>, AssertionError(' Items are not equal: Number of outputs not equal.

            PhotonTest__test_standard.test_standard: FAILURE (<type 'exceptions.AssertionError'>, AssertionError(' Items are not equal: Number of outputs not equal.

            PhotonTestAMR__test_standard.test_standard: FAILURE (<type 'exceptions.AssertionError'>, AssertionError(' Items are not equal: Number of outputs not equal.

            1. Brian OShea

              Followup on my followup: I dug into this further (as a result of testing the other PRs) and found that what seems to be happening is that the test simulations are all seg faulting on my laptop. I'm not sure why that is - we should talk about it tomorrow morning.

              1. John Wise author

                It only segfaults with all of the options in 64-bit in ReadPhotonSources. I'm currently tracking it down. There seems to be some sort of memory corruption that gdb or valgrind can't identify. I'm doing an "hg bisect" to search for the offender.

                Update I found that 5e0cea8ad378 in my fork was identified by the bisect.

                Update 2 The #define MAX_SOURCES was increased from 3k to 100k that was causing stack overflows. I'm reverting that change since 3k should be good for most cases.

  1. John Wise author

    Thanks, @john_regan, for reviewing my changes. I'll update the PR today with your suggested changes.

  2. James Larrue

    I think it just needs a comment added to the code to explain why the equals condition does not need to be handled in src/enzo/Grid_CheckSubgridMarker.C, as per the comments below.

  3. Brian OShea

    All tests pass; some minor differences are observed, but they are due to improvements in the code. I will approve and merge momentarily.