Wiki

Clone wiki

Sunrise / SunriseOverview

Introduction

The original purpose of Sunrise was to be able to calculate images and spectra of simulated galaxies in a hydrodynamic simulation. Since this represents a standard work flow that must be repeated hundreds of times to follow a simulation in its evolution, this process has been fairly streamlined. (The alternative is if you want to use Sunrise to do something arbitrary, in which case you need to write code, see SunriseSpecialOccasions.) After reading this overview, you can check out Fabio's SunriseBootCamp, which features a more tutorial-style example they use at UW.

When using Sunrise to process Gadget or Gasoline snapshots, three "standard executables" are invoked in order:

  • sfrhist loads the snapshot and calculates the source spectra of the stellar (and AGN) particles in the simulation based on their age, metallicity and potentially other environmental factors. It also generates the adaptive mesh refinement grid that is used to store the dust density distribution. The output from sfrhist is an intermediate FITS file, whose format is described in SfrhistConfigAndOutputFileFormat.
  • mcrx performs the core radiation-transfer calculations. It reads the file created by sfrhist and creates a (usually quite large) output FITS file that contains the output of the RT calculation. The full format of the output file is described in McrxConfigAndOutputFileFormat, but it basically contains a number of data cubes describing the radiation from the simulated that reaches the defined "cameras". For each camera, there are data cubes for "unattenuated" radiation (just the source distribution in the absence of any dust), for the "attenuated" source radiation that emerges after passing through the dust, and for radiation emitted by the heated dust grains. There are also auxiliary data cubes containing data from the snapshots like surface densities of gas, metals, stars, star-formation rate, and bolometric source luminosity. Since these are projected identically to the radiation data cubes, it makes it easy to correlate radiation quantities with things like mass and metallicity.
  • broadband finally takes the data cubes output by mcrx and calculates the fluxes in specific filter bands, potentially putting the source at a specified redshift. Because the number of filter bands desired usually is far smaller than the number of wavelengths in the full spectrum, the FITS files output by broadband (described in BroadbandConfigAndOutputFileFormat are much smaller.

Each of these executables take a configuration file as the argument. These configuration files contain keyword-value pairs that completely specify the problem. The keywords recognized in these files are described in the wiki pages above (SfrhistConfigAndOutputFileFormat, McrxConfigAndOutputFileFormat, and BroadbandConfigAndOutputFileFormat). Because one generally runs a series of snapshots from the same run, it's useful to collect the settings that are common to all snapshots in a configuration "stub" file, which then can be included by all the config files for the specific snapshots. In this way, the config files pretty much only contain the names of the input and output files. For example, a typical sfrhist config file might look like:

include_file    sfrhist.stub
snapshot_file   snapshot_010
output_file     grid_010.fits
use_counters    false
The parameters for all the snapshots can then be collected in the sfrhist.stub file. (In addition, there are scripts to manage the submission of all these to the batch system, see below.)

sfrhist details

There are three main sets of parameters for sfrhist: Two are general: stellar population model and grid creation ones. The third only pertains to Gadget simulations that are started with initial galaxies (i.e., not cosmological runs) and determine the age and metallicity distributions in the initial galaxies.

The stellar population model keywords determine which SED grid is used for the stellar populations and also parameters that determine the use of the MAPPINGS subresolution models for star-forming regions.

The grid creation keywords set the size of the base grid, the maximum level of refinement, and the criteria used to determine whether a grid cell should be further refined or unified. For an overview of the heuristic used for this, see Jonsson 06 and Jonsson, Groves & Cox 10.

The initial conditions parameters are only used when the initial conditions contain stars or nonzero metallicity gas, and determine metallicity gradients (which are applied equally to both stars and gas), and the star-formation histories of the pre-existing bulge and disk stars. In order for sfrhist to be able to determine where the stars started out, you need to also specify snapshots to the isolated initial galaxy (or galaxies, if there are more than one).

For detailed info about these keywords, see SfrhistConfigAndOutputFileFormat.

mcrx details

The radiation transfer calculation proceeds in several stages and has many tunable parameters, so there are many keywords in the mcrx parameter file. For detailed info, see McrxConfigAndOutputFileFormat:

  • Camera setup parameters determine the number and positions of the cameras and their field of view and resolution.

  • Dust model parameters determine the dust opacities and cross sections used. These can be specified in a variety of ways from just a simple ascii file (if all you care about is absorption and scattering) to a set of full models of dust grain compositions and sizes based on the Weingartner & Draine 01/Draine & Li 07 papers.

  • Dust density parameters determine the conversion from the density field of gas and metals (from the Gadget simulation) into dust. They also potentially specify the parameters for the Springel & Hernquist 03 "multiphase" model used to determine how much of the dust is in small dense clumps and how much is in diffuse form.

  • Monte Carlo parameters determine things like intensity thresholds for splitting and dropping rays, for using forced scattering, and for the reference wavelengths used in the "polychromatic" formulation of the Monte Carlo (see Jonsson 06 for a detailed description of how this works). They also determine the number of rays to shoot for the different stages of the calculation.

  • Infrared emission parameters determine how dust emission is calculated and the accuracy with which self-absorption is iterated to convergence.

  • Runtime parameters determine things like the number of threads to use, the random number seed, and things pertaining to batch job resubmission.

broadband details

Broadband is pretty simple. The only parameters specify the filter bands you want the magnitudes in and where to find the filter definition files.

Parameter sets

As a starting point, you can use the parameters from the "Sbc-example" available in the downloads. Note however that this can only be a starting point, as different simulations have different resolutions and characteristics that may require different parameters. You are strongly encouraged to read ConvergenceTests at this point.

Performance

Unlike most of the hydro codes, Sunrise does not use MPI. It uses threads to parallelize the calculation. This means that you can only run it on one node, so a good machine to run Sunrise on has many processors and a lot of memory on one node. The memory issue in particular makes many of the supercomputers unusable, because they have too little memory on the nodes. (Note that system descriptions often quote the memory per core, which is an irrelevant quantity. What you care about is total addressable memory from one process.)

To be able to run reasonably large calculations, you need 32GB of total memory. For high-density, bright starbursts, you need so many grid cells to be converged that you really need 64 or more GB, though this depends strongly on the wavelength resolution you want.

Disclaimer

When Sunrise starts, it prints: "This program comes with ABSOLUTELY NO WARRANTY." This is not a joke. While I am relatively confident that there are no major bugs in the radiation transfer calculation, the outputs from Sunrise are only as good as the parameters used. It is the responsibility of each and every user to perform adequate testing to convince themselves that their results are correct in their situation.

Updated