1. The Enzo Project
  2. Untitled project
  3. enzo-dev


yua...@login3.ranger.tacc.utexas.edu  committed 2c1a825 Merge

merge with enzo-dev

  • Participants
  • Parent commits 072c9db, cab01f8
  • Branches week-of-code

Comments (0)

Files changed (775)

File .hgignore

File contents unchanged.

File .hgtags

View file
 bbf0a2ffbd22c4fbecf946c9c96e6c4fac5cbdae woc_pre_fld_merge
 48b4e9d9d6b90f703e48e621b488136be2a0e9cf woc_fld_merge
 b86d8ba026d6a0ec30f15d8134add1e55fae2958 Wise10_GalaxyBirth
+2d90aa38e06f00a531db45a43225cde1faf093f2 enzo-2.2


View file
-Enzo is developed in the open at:
+Enzo's main webpage is:
- * http://enzo.googlecode.com/
+ * http://enzo-project.org
+Enzo is developed in the open on bitbucket.org:
+ * Stable Version: https://bitbucket.org/enzo/enzo-stable
+ * Development Version: https://bitbucket.org/enzo/enzo-dev
 Documentation, including instructions for compilation, can be found at:
- * http://docs.enzo.googlecode.com/hg/index.html
+ * http://enzo-project.org/docs/2.2/
 Please subscribe to the Enzo Users' mailing list at:
  * https://mailman.ucsd.edu/mailman/listinfo/enzo-users-l
+You can also follow the Enzo Project on google+ at:
+ * https://plus.google.com/115923030596894217717
 If you have received this source code through an archive, rather than the
 mercurial version control system, we highly encourage you to upgrade to the
 version controlled source, as no support can be provided for archived
 Many people have contributed to the development of Enzo -- here's just a short
-list of the people who have recently contributed.
+list of the people who have recently contributed, in alphabetical order:
-   * Greg Bryan      gbryan@astro.columbia.edu
-   * Tom Abel        tabel@stanford.edu
-   * Michael Norman  mlnorman@ucsd.edu
-   * John Wise       jwise@physics.gatech.edu
-   * Dan Reynolds    reynolds@smu.edu
-   * Michael Kuhlen  mqk@astro.berkeley.edu
-   * Matthew Turk    matthewturk@gmail.com
-   * Brian O'Shea    oshea@msu.edu
-   * Robert Harkness harkness@sdsc.edu
-   * Alexei Kritsuk   akritsuk@ucsd.edu
-   * Elizabeth Tasker taskere@mcmaster.ca
-   * Dave Collins    dcollins@physics.ucsd.edu
-   * Britton Smith   brittonsmith@gmail.com
+   * Tom Abel               tabel@stanford.edu
+   * James Bordner          jobordner@ucsd.edu
+   * Greg Bryan             gbryan@astro.columbia.edu
+   * Renyue Cen             cen@astro.princeton.edu
+   * Dave Collins           dcollins@physics.ucsd.edu
+   * Nathan Goldbaum        goldbaum@ucolick.org
+   * Robert Harkness        harkness@sdsc.edu
    * Elizabeth Harper-Clark h-clark@astro.utoronto.ca
-   * Peng Wang       pengw@slac.stanford.edu
-   * Fen Zhao        fenzhao@stanford.edu
-   * James Bordner   jobordner@ucsd.edu
-   * Pascal Paschos  ppaschos@minbari.ucsd.edu
-   * Stephen Skory   sskory@physics.ucsd.edu
-   * Rick Wagner     rwagner@physics.ucsd.edu
-   * Renyue Cen      cen@astro.princeton.edu
-   * Alex Razoumov   razoumov@gmail.com
-   * Cameron Hummels chummels@astro.columbia.edu
-   * JS Oishi        jsoishi@gmail.com
-   * Christine Simpson csimpson@astro.columbia.edu
+   * Cameron Hummels        chummels@gmail.com
+   * Alexei Kritsuk         akritsuk@ucsd.edu
+   * Michael Kuhlen         mqk@astro.berkeley.edu
+   * Eve Lee                elee@cita.utoronto.ca
+   * Yuan Li                yuan@astro.columbia.edu
+   * Michael Norman         mlnorman@ucsd.edu
+   * JS Oishi               jsoishi@gmail.com
+   * Brian O'Shea           oshea@msu.edu
+   * Pascal Paschos         ppaschos@minbari.ucsd.edu
+   * Alex Razoumov          razoumov@gmail.com
+   * Dan Reynolds           reynolds@smu.edu
+   * Christine Simpson      csimpson@astro.columbia.edu
+   * Samuel Skillman        samskillman@gmail.com 
+   * Stephen Skory          s@skory.us
+   * Britton Smith          brittonsmith@gmail.com
+   * Elizabeth Tasker       tasker@astro1.sci.hokudai.ac.jp
+   * Matthew Turk           matthewturk@gmail.com
+   * Rick Wagner            rwagner@physics.ucsd.edu
+   * Peng Wang              pengw@slac.stanford.edu
+   * John Wise              jwise@physics.gatech.edu
+   * Fen Zhao               fenzhao@stanford.edu

File bin/pr_testing

View file
+### Author: Michael Kuhlen (mqk@astro.berkeley.edy)
+### Date: 06/12/2012
+function do_help {
+    echo
+    printf "Usage: pr_testing REPO [REV] [--help] [--make_opts MAKE_OPTS] [--machine MACH_NAME] [--build BUILD_DIR] [--gold_standard GOLD_STD]\n"
+    echo
+    printf "This script checks out revision REV (defaults to tip) of the Enzo code from \nthe repository REPO and builds Enzo in the directory BUILD_DIR for the \nmachine MACH_NAME and with build options given by MAKE_OPTS. It then launches \nEnzo's test_runner.py, which performs the quicksuite tests and compares the \nresults to a gold standard (downloaded from \nhttp://enzo-project.org/tests/gold_standard_quick.tar.gz). An alternative \ngold standard file (tar.gz) can be specified with GOLD_STD.\n"
+    echo
+    printf "Only MACH_NAME is mandatory; BUILD_DIR, MAKE_OPTS, and GOLD_STD are \noptional. You can specify these variables as environment variables, set \nthem in ~/.enzo/enzo_testing_vars.sh, or manually by using the command line \noptions.\n"
+    echo
+    printf 'Example: ./PR_testing.sh https://bitbucket.org/enzo/enzo-dev 0e97196b30f3 --machine linux-gnu --make_opts "-j 8"\n'
+    echo
+    exit 0
+function do_exit {
+    echo $*
+    exit 1
+### First source ~/.enzo/enzo_testing_vars.sh, if it exists
+if [ -e ~/.enzo/enzo_testing_vars.sh ]
+    . ~/.enzo/enzo_testing_vars.sh
+### Now parse command line arguments
+while (( "$#" ))
+    case "$1" in
+	--help | -h)
+	    do_help
+	    ;;
+	--make_opts)
+	    shift
+	    MAKE_OPTS="$1"
+	    ;;
+	--machine)
+	    shift
+	    MACH_NAME="$1"
+	    ;;
+	--build)
+	    shift
+	    BUILD_DIR="$1"
+	    ;;
+	--gold_standard)
+	    shift
+	    GOLD_STD="$1"
+	    ;;
+	-*)
+	    echo
+	    echo "Unknown command line option $1."
+	    do_help
+	    ;;
+	*)
+	    if [ -z "$REPO" ]; then
+		REPO="$1"
+	    elif [ -z "$REV" ]; then
+		REV="$1"
+	    fi
+    esac
+    shift
+### If revision wasn't specified, set it to "tip".
+if [ -z "$REV" ]; then
+    REV="tip"
+### If build dir wasn't specified, set it to $PWD.
+### Sanity check. Ensure that everything has been properly set.
+if [ -z "$REPO" ]
+    echo
+    echo "ERROR! You must specify a repository."
+    do_help
+if [ -z "$MACH_NAME" ]
+    echo
+    echo "ERROR! MACH_NAME must be set."
+    do_help
+### Set log file
+echo "Your Configuration:"
+echo "        REPO = $REPO"
+echo "         REV = $REV"
+echo "   MACH_NAME = $MACH_NAME"
+echo "   MAKE_OPTS = $MAKE_OPTS"
+echo "  BUILD_DIR = $BUILD_DIR"
+echo "    GOLD_STD = $GOLD_STD"
+echo "     LOGFILE = $LOGFILE"
+### Download gold standard, if URL was provided. Otherwise simply use
+### the file provided in GOLD_STD.
+if [ ${GOLD_STD:0:4} == "http" ]; then
+if [ -n "$GOLD_STD_IS_URL" ]; then
+    GOLD_STD_FILE="$BUILD_DIR/$(basename $GOLD_STD)"
+    ## only download, if it doesn't already exist.
+    if [ ! -e "$GOLD_STD_FILE" ]; then
+	echo "Downloading gold standard from $GOLD_STD."
+	( wget -nv "$GOLD_STD" 2>&1 ) \
+	    1>> "$LOGFILE" || do_exit "Could not download the gold standard, $GOLD_STD."
+    fi
+echo "Using gold standard file $GOLD_STD."
+### Clone Revision
+echo "Cloning revision $REV from repository $REPO into $BUILD_DIR/enzo-$REV."
+( hg clone -r "$REV" "$REPO" enzo-"$REV" 2>&1 ) 1>> "$LOGFILE" || \
+    do_exit "Could not clone."
+cd enzo-"$REV"/
+### Check that the correct revision has been checked out
+THIS_REV=$(hg identify -i)
+## must handle 'REV==tip' differently
+if [ "$REV" == "tip" ]; then
+    THIS_REV=$(hg identify -t)
+if [ "$REV" != "$THIS_REV" ]; then
+    do_exit "Incorrect revision checked out."
+### Build Enzo
+echo "Building Enzo."
+printf "   "
+cd src/enzo
+echo "   Running 'make machine-$MACH_NAME'."
+( make machine-${MACH_NAME} 2>&1 ) 1>> "$LOGFILE" || \
+    do_exit "Couldn't make machine correctly."
+echo "   Running 'make $MAKE_OPTS'."
+( make ${MAKE_OPTS} 2>&1 ) 1>> "$LOGFILE" || \
+    do_exit "Could not build."
+cd ../../run/
+### Extract gold standard
+echo "Extracting gold standard."
+( tar xfz "$GOLD_STD" 2>&1 ) 1>> "$LOGFILE" || \
+    do_exit "Couldn't extract gold standard from $BUILD_DIR/$GOLD_STD."
+### Run test suite
+echo "Running test suite."
+( python test_runner.py -c gold_standard -o PR --quicksuite=True 2>&1 ) \
+    1>> "$LOGFILE" || do_exit "Test runner died."
+### Print results
+echo "RESULTS:"
+cat PR/"$REV"/test_results.txt

File doc/manual/source/EnzoParameters.rst

-.. _parameters:
-Enzo Parameter List
-The following is a largely complete list of the parameters that Enzo
-understands, and a brief description of what they mean. They are grouped
-roughly by meaning; an alphabetical list is also available. Parameters for
-individual test problems are also listed here.
-This parameter list has two purposes. The first is to describe and explain the
-parameters that can be put into the initial parameter file that begins a run.
-The second is to provide a comprehensive list of all parameters that the code
-uses, including those that go into an output file (which contains a complete
-list of all parameters), so that users can better understand these output
-The parameters fall into a number of categories:
-    These are user parameters in the sense that they can be set in the
-    parameter file, and provide the primary means of communication
-    between Enzo and the user.
-    These are mostly not set in the parameter file (although strictly
-    speaking they can be) and are generally used for program to
-    communicate with itself (via the restart of output files).
-    No longer used.
-    To be used later.
-Generally the external parameters are the only ones that are modified or set,
-but the internal parameters can provide useful information and can sometimes be
-modified so I list them here as well. Some parameters are true/false or on/off
-boolean flags.  Eventually, these may be parsed, but in the meantime, we use the
-common convention of 0 meaning false or off and 1 for true or on.
-This list includes parameters for the Enzo 2.0 release.
-.. highlight:: none
-Stopping Parameters
-``StopTime`` (external)
-    This parameter specifies the time (in code units) when the
-    calculation will halt. For cosmology simulations, this variable is
-    automatically set by ``CosmologyFinalRedshift``. *No default.*
-``StopCycle`` (external)
-    The cycle (top grid timestep) at which the calculation stops. A
-    value of zero indicates that this criterion is not be used.
-    *Default: 100,000*
-``StopFirstTimeAtLevel`` (external)
-    Causes the simulation to immediately stop when a specified level is
-    reached. Default value 0 (off), possible values are levels 1
-    through maximum number of levels in a given simulation.
-``NumberOfOutputsBeforeExit`` (external)
-    After this many datadumps have been written, the code will exit.  If 
-    set to 0 (default), this option will not be used.  Default: 0.
-``StopCPUTime`` (external)
-    Causes the simulation to stop if the wall time exceeds ``StopCPUTime``.
-    The simulation will output if the wall time after the next
-    top-level timestep will exceed ``StopCPUTime``, assuming that the wall
-    time elapsed during a top-level timestep the same as the previous
-    timestep. In units of seconds. Default: 2.592e6 (30 days)
-``ResubmitOn`` (external)
-    If set to 1, the simulation will stop if the wall time will exceed
-    ``StopCPUTime`` within the next top-level timestep and run a shell
-    script defined in ``ResubmitCommand`` that should resubmit the job
-    for the user. Default: 0.
-``ResubmitCommand`` (external)
-    Filename of a shell script that creates a queuing (e.g. PBS)
-    script from two arguments, the number of processors and parameter
-    file.  This script is run by the root processor when stopping with
-    ``ResubmitOn``. An example script can be found in
-    input/resubmit.sh. Default: (null)
-Initialization Parameters
-``ProblemType`` (external)
-    This integer specifies the type of problem to be run. Its value
-    causes the correct problem initializer to be called to set up the
-    grid, and also may trigger certain boundary conditions or other
-    problem-dependent routines to be called. The possible values are
-    listed below. Default: none. 
-For other problem-specific parameters follow the links below.  The problems
-marked with "hydro_rk" originate from the MUSCL solver package in the enzo installation directory
-``src/enzo/hydro_rk``.  For the 4xx radiation hydrodynamics problem types, see
-the user guides in the installation directory ``doc/implicit_fld`` and ``doc/split_fld``.
-============ ====================================
-Problem Type Description and Parameter List
-============ ====================================
-1 	     :ref:`shocktube_param`
-2	     :ref:`wavepool_param`
-3 	     :ref:`shockpool_param`
-4 	     :ref:`doublemach_param`
-5 	     :ref:`shockinabox_param`
-6 	     Implosion
-7 	     SedovBlast
-8 	     KH Instability
-9 	     2D/3D Noh Problem
-10 	     :ref:`rotatingcylinder_param`
-11 	     :ref:`radiatingshock_param`
-12 	     :ref:`freeexpansion_param`
-20 	     :ref:`zeldovichpancake_param`
-21 	     :ref:`pressurelesscollapse_param`
-22 	     :ref:`adiabaticexpansion_param`
-23 	     :ref:`testgravity_param`
-24 	     :ref:`sphericalinfall_param`
-25 	     :ref:`testgravitysphere_param`
-26 	     :ref:`gravityequilibriumtest_param`
-27 	     :ref:`collapsetest_param`
-28 	     TestGravityMotion
-29 	     TestOrbit
-30 	     :ref:`cosmologysimulation_param`
-31 	     :ref:`galaxysimulation_param`
-35 	     :ref:`shearingbox_param`
-40 	     :ref:`supernovarestart_param`
-50 	     :ref:`photontest_param`
-60 	     Turbulence Simulation
-61 	     Protostellar Collapse
-62 	     :ref:`coolingtest_param`
-101	     3D Collapse Test (hydro_rk)
-102	     1D Spherical Collapse Test (hydro_rk)
-106	     Hydro and MHD Turbulence Simulation (hydro_rk)
-107 	     Put Sink from restart
-200	     1D MHD Test
-201	     2D MHD Test
-202	     3D MHD Collapse Test
-203	     MHD Turbulent Collapse Test
-207	     Galaxy disk
-208	     AGN disk
-300	     Poisson solver test
-400 	     Radiation-Hydrodynamics test 1 -- constant fields
-401 	     Radiation-Hydrodynamics test 2 -- stream test
-402 	     Radiation-Hydrodynamics test 3 -- pulse test
-403 	     Radiation-Hydrodynamics test 4 -- grey Marshak test
-404/405	     Radiation-Hydrodynamics test 5 -- radiating shock test
-410/411	     Radiation-Hydrodynamics test 10/11 -- Static HI ionization
-412 	     Radiation-Hydrodynamics test 12 -- HI ionization of a clump
-413 	     Radiation-Hydrodynamics test 13 -- HI ionization of a steep region
-414/415	     Radiation-Hydrodynamics test 14/15 -- Cosmological HI ionization
-450-452	     Free-streaming radiation tests
-============ ====================================
-.. raw:: html
-   <p></p>
-``TopGridRank`` (external)
-    This specifies the dimensionality of the root grid and by extension
-    the entire hierarchy. It should be 1,2 or 3. Default: none
-``TopGridDimensions`` (external)
-    This is the dimension of the top or root grid. It should consist of
-    1, 2 or 3 integers separated by spaces. For those familiar with the
-    KRONOS or ZEUS method of specifying dimensions, these values do not
-    include ghost or boundary zones. A dimension cannot be less than 3
-    zones wide and more than ``MAX_ANY_SINGLE_DIRECTION`` -
-    ``NumberOfGhostZones``\*2. ``MAX_ANY_SINGLE_DIRECTION`` is defined in
-    ``fortran.def``. Default: none
-``DomainLeftEdge``, ``DomainRightEdge`` (external)
-    These float values specify the two corners of the problem domain
-    (in code units). The defaults are: 0 0 0 for the left edge and 1 1
-    1 for the right edge.
-``LeftFaceBoundaryCondition``, ``RightFaceBoundaryCondition`` (external)
-    These two parameters each consist of vectors of integers (of length
-    ``TopGridRank``). They specify the boundary conditions for the top grid
-    (and hence the entire hierarchy). The first integer corresponds to
-    the x-direction, the second to the y-direction and the third, the
-    z-direction. The possible values are: 0 - reflecting, 1 - outflow,
-    2 - inflow, 3 - periodic, 4 - shearing. For inflow, the inflow
-    values can be set through the next parameter, or more commonly are
-    controlled by problem-specific code triggered by the ``ProblemType``.
-    For shearing boundaries, the boundary pair in another direction
-    must be periodic. Note that self gravity will not be consistent
-    with shearing boundary conditions. Default: 0 0 0
-``ShearingVelocityDirection`` (external)
-    Select direction of shearing boundary. Default is x direction. Changing this is probably not a good idea.
-``AngularVelocity`` (external)
-    The value of the angular velocity in the shearing boundary.
-    Default: 0.001
-``VelocityGradient`` (external)
-    The value of the per code length gradient in the angular velocity
-    in the shearing boundary. Default: 1.0
-``BoundaryConditionName`` (external)
-    While the above parameters provide an easy way to set an entire
-    side of grid to a given boundary value, the possibility exists to
-    set the boundary conditions on an individual cell basis. This is
-    most often done with problem specific code, but it can also be set
-    by specifying a file which contains the information in the
-    appropriate format. This is too involved to go into here. Default:
-    none
-``InitialTime`` (internal)
-    The time, in code units, of the current step. For cosmology the
-    units are in free-fall times at the initial epoch (see :ref:`EnzoOutputFormats`). Default: generally 0, depending on problem
-``Initialdt`` (internal)
-    The timestep, in code units, for the current step. For cosmology
-    the units are in free-fall times at the initial epoch (see :ref:`EnzoOutputFormats`). Default: generally 0, depending on problem
-Simulation Identifiers and UUIDs
-These parameters help to track, identify and group datasets. For reference,
-`Universally Unique Identifiers
-<http://en.wikipedia.org/wiki/Universally_Unique_Identifier>`_ (UUIDs) are
-opaque identifiers using random 128-bit numbers, with an extremely low chance
-of collision. (See :ref:`SimulationNamesAndIdentifiers` for a longer
-description of these parameters.)
-``MetaDataIdentifier`` (external)
-    This is a character string without spaces (specifically, something
-    that can be picked by "%s"), that can be defined in a parameter
-    file, and will be written out in every following output, if it is
-    found.
-``MetaDataSimulationUUID`` (internal)
-    A UUID that will be written out in all of the following outputs.
-    Like ``MetaDataIdentifier``, an existing UUID will be kept, but if one
-    is not found, and new one will be generated.
-``MetaDataDatasetUUID`` (internal)
-    A UUID created for each specific output.
-``MetaDataRestartDatasetUUID`` (internal)
-    If a ``MetaDataDatasetUUID`` UUID is found when the parameter file is
-    read in, it will written to the following datasets. This is used to
-    track simulations across restarts and parameter adjustments.
-``MetaDataInitialConditionsUUID`` (internal)
-    This is similar to ``MetaDataRestartDatasetUUID``, except it's used to
-    track which initial conditions were used.
-I/O Parameters
-There are three ways to specify the frequency of outputs:
-time-based, cycle-based (a cycle is a top-grid timestep), and, for
-cosmology simulations, redshift-based. There is also a shortened
-output format intended for visualization (movie format). Please
-have a look at :ref:`controlling_data_output` for more information.
-``dtDataDump`` (external)
-    The time interval, in code units, between time-based outputs. A
-    value of 0 turns off the time-based outputs. Default: 0
-``CycleSkipDataDump`` (external)
-    The number of cycles (top grid timesteps) between cycle-based
-    outputs. Zero turns off the cycle-based outputs. Default: 0
-``DataDumpName`` (external)
-    The base file name used for both time and cycle based outputs.
-    Default: data
-``RedshiftDumpName`` (external)
-    The base file name used for redshift-based outputs (this can be
-    overridden by the ``CosmologyOutputRedshiftName`` parameter). Normally
-    a four digit identification number is appended to the end of this
-    name, starting from 0000 and incrementing by one for every output.
-    This can be over-ridden by including four consecutive R's in the
-    name (e.g. RedshiftRRRR) in which case the an identification number
-    will not be appended but the four R's will be converted to a
-    redshift with an implied decimal point in the middle (i.e. z=1.24
-    becomes 0124). Default: RedshiftOutput
-``CosmologyOutputRedshift[NNNN]`` (external)
-    The time and cycle-based outputs occur regularly at constant
-    intervals, but the redshift outputs are specified individually.
-    This is done by the use of this statement, which sets the output
-    redshift for a specific identification number (this integer is
-    between 0000 and 9999 and is used in forming the name). So the
-    statement ``CosmologyOutputRedshift[1] = 4.0`` will cause an output to
-    be written out at z=4 with the name RedshiftOutput0001 (unless the
-    base name is changed either with the previous parameter or the next
-    one). This parameter can be repeated with different values for the
-    number (NNNN) Default: none
-``CosmologyOutputRedshiftName[NNNN]`` (external)
-    This parameter overrides the parameter ``RedshiftOutputName`` for this
-    (only only this) redshift output. Can be used repeatedly in the
-    same manner as the previous parameter. Default: none
-``OutputFirstTimeAtLevel`` (external)
-    This forces Enzo to output when a given level is reached, and at
-    every level thereafter. Default is 0 (off). User can usefully
-    specify anything up to the maximum number of levels in a given
-    simulation.
-    If this parameter is set to 1, whenever the finest level has finished
-    evolving Enzo will check for new signal files to output.  (See
-    :ref:`force_output_now`.)  Default 1.
-``XrayLowerCutoffkeV``, ``XrayUpperCutoffkeV``, ``XrayTableFileName`` (external)
-    These parameters are used in 2D projections (``enzo -p ...``). The
-    first two specify the X-ray band (observed at z=0) to be used, and
-    the last gives the name of an ascii file that contains the X-ray
-    spectral information. A gzipped version of this file good for
-    bands within the 0.1 - 20 keV range is provided in the
-    distribution in ``input/lookup_metal0.3.data``. If these
-    parameters are specified, then the second field is replaced with
-    integrated emissivity along the line of sight in units of 10\
-    :sup:`-23` erg/cm\ :sup:`2`/s. Default: ``XrayLowerCutoffkeV =
-    0.5``, ``XrayUpperCutoffkeV = 2.5``.
-``ExtractFieldsOnly`` (external)
-    Used for extractions (enzo -x ...) when only field data are needed
-    instead of field + particle data. Default is 1 (TRUE).
-    Reserved for future use.
-    Reserved for future use.
-    Reserved for future use.
-    Reserved for future use.
-    Reserved for future use.
-    Reserved for future use.
-``ParallelRootGridIO`` (external)
-    Normally for the mpi version, the root grid is read into the root
-    processor and then partitioned to separate processors using communication.
-    However, for
-    very large root grids (e.g. 512\ :sup:`3`\ ), the root processor
-    may not have enough memory. If this toggle switch is set on (i.e.
-    to the value 1), then each processor reads its own section of the
-    root grid. More I/O is required (to split up the grids and
-    particles), but it is more balanced in terms of memory.
-    ``ParallelRootGridIO`` and ``ParallelParticleIO`` MUST be set to 1 (TRUE)
-    for runs involving > 64 cpus! Default: 0 (FALSE). See also ``Unigrid``
-    below.
-``Unigrid`` (external)
-    This parameter should be set to 1 (TRUE) for large cases--AMR as
-    well as non-AMR--where the root grid is 512\ :sup:`3`\  or larger.
-    This prevents initialization under subgrids at start up, which is
-    unnecessary in cases with simple non-nested initial conditions.
-    Unigrid must be set to 0 (FALSE) for cases with nested initial
-    conditions. Default: 0 (FALSE). See also ``ParallelRootGridIO`` above.
-``UnigridTranspose`` (external)
-    This parameter governs the fast FFT bookkeeping for Unigrid runs.
-    Does not work with isolated gravity. Default: 0 (FALSE). See also
-    ``Unigrid`` above.
-``OutputTemperature`` (external)
-    Set to 1 if you want to output a temperature field in the datasets.
-    Always 1 for cosmology simulations. Default: 0.
-``OutputCoolingTime`` (external)
-    Set to 1 if you want to output the cooling time in the datasets.
-    Default: 0.
-``OutputSmoothedDarkMatter`` (external)
-    Set to 1 if you want to output a dark matter density field,
-    smoothed by an SPH kernel. Set to 2 to also output smoothed dark
-    matter velocities and velocity dispersion. Set to 0 to turn off.
-    Default: 0.
-``OutputGriddedStarParticle`` (external)
-    Set to 1 or 2 to write out star particle data gridded onto mesh.
-    This will be useful e.g. if you have lots of star particles in a
-    galactic scale simulation. 1 will output just
-    ``star_particle_density``; and 2 will dump
-    ``actively_forming_stellar_mass_density``, ``SFR_density``, etc.
-    Default: 0.
-``VelAnyl`` (external)
-    Set to 1 if you want to output the divergence and vorticity of
-    velocity. Works in 2D and 3D.
-``BAnyl`` (external)
-    Set to 1 if you want to output the divergence and vorticity of
-    ``Bfield``. Works in 2D and 3D.
-``SmoothedDarkMatterNeighbors`` (external)
-    Number of nearest neighbors to smooth dark matter quantities over.
-    Default: 32.
-.. _streaming_param:
-Streaming Data Format
-``NewMovieLeftEdge``, ``NewMovieRightEdge`` (external)
-    These two parameters control the region for which the streaming
-    data are written. Default: ``DomainLeftEdge`` and ``DomainRightEdge``.
-``MovieSkipTimestep`` (external)
-    Controls how many timesteps on a level are skipped between outputs
-    in the streaming data. Streaming format is off if this equals
-    ``INT_UNDEFINED``. Default: ``INT_UNDEFINED``
-``Movie3DVolume`` (external)
-    Set to 1 to write streaming data as 3-D arrays. This should always
-    be set to 1 if using the streaming format. A previous version had
-    2D maximum intensity projections, which now defunct. Default: 0.
-``MovieVertexCentered`` (external)
-    Set to 1 to write the streaming data interpolated to vertices. Set
-    to 0 for cell-centered data. Default: 0.
-``NewMovieDumpNumber`` (internal)
-    Counter for streaming data files. This should equal the cycle
-    number.
-``MovieTimestepCounter`` (internal)
-    Timestep counter for the streaming data files.
-``MovieDataField`` (external)
-    A maximum of 6 data fields can be written in the streaming format.
-    The data fields are specified by the array element of
-    BaryonField, i.e. 0 = Density, 7 = HII
-    Density. For writing temperature, a special value of 1000 is used.
-    This should be improved to be more transparent in which fields will
-    be written. Any element that equals ``INT_UNDEFINED`` indicates no
-    field will be written. Default: ``INT_UNDEFINED`` x 6
-``NewMovieParticleOn`` (external)
-    Set to 1 to write all particles in the grids. Set to 2 to write
-    ONLY particles that aren't dark matter, e.g. stars. Set to 3/4 to
-    write ONLY particles that aren't dark matter into a file separate
-    from the grid info. (For example, ``MoviePackParticle_P000.hdf5``,
-    etc. will be the file name; this will be very helpful in speeding
-    up the access to the star particle data, especially for the
-    visualization or for the star particle. See ``AMRH5writer.C``) Set to 0
-    for no particle output. Default: 0.
-Hierarchy Control Parameters
-``StaticHierarchy`` (external)
-    A flag which indicates if the hierarchy is static (1) or dynamic
-    (0). In other words, a value of 1 takes the A out of AMR. Default:
-    1
-``RefineBy`` (external)
-    This is the refinement factor between a grid and its subgrid. For
-    cosmology simulations, we have found a ratio of 2 to be most useful.
-    Default: 4
-``MaximumRefinementLevel`` (external)
-    This is the lowest (most refined) depth that the code will produce.
-    It is zero based, so the total number of levels (including the root
-    grid) is one more than this value. Default: 2
-``CellFlaggingMethod`` (external)
-    The method(s) used to specify when a cell should be refined. This
-    is a list of integers, up to 9, as described by the following
-    table. The methods combine in an "OR" fashion: if any of them
-    indicate that a cell should be refined, then it is flagged. For
-    cosmology simulations, methods 2 and 4 are probably most useful.
-    Note that some methods have additional parameters which are
-    described below. Default: 1
-    :: 
-       1 - refine by slope		       6  - refine by Jeans length
-       2 - refine by baryon mass	       7  - refine if (cooling time < cell width/sound speed)
-       3 - refine by shocks 		       11 - refine by resistive length
-       4 - refine by particle mass	       12 - refine by defined region "MustRefineRegion"
-       5 - refine by baryon overdensity	       13 - refine by metallicity
-       	  (currently disabled)
-       101 - avoid refinement in regions
-             defined in "AvoidRefineRegion"
-``RefineRegionLeftEdge``, ``RefineRegionRightEdge`` (external)
-    These two parameters control the region in which refinement is
-    permitted. Each is a vector of floats (of length given by the
-    problem rank) and they specify the two corners of a volume.
-    Default: set equal to ``DomainLeftEdge`` and ``DomainRightEdge``.
-``RefineRegionAutoAdjust`` (external)
-    This is useful for multiresolution simulations with particles in
-    which the particles have varying mass. Set to 1 to automatically
-    adjust the refine region at root grid timesteps to only contain
-    high-resolution particles. This makes sure that the fine regions do
-    not contain more massive particles which may lead to small
-    particles orbiting them or other undesired outcomes. Setting to any
-    integer (for example, 3) will make AdjustRefineRegion to work at
-    (RefineRegionAutoAdjust-1)th level timesteps because sometimes the
-    heavy particles are coming into the fine regions too fast that you
-    need more frequent protection. Default: 0.
-``RefineRegionTimeType`` (external)
-    If set, this controls how the first column of a refinement region
-    evolution file (see below) is interpreted, 0 for code time, 1 for
-    redshift. Default: -1, which is equivalent to 'off'.
-``RefineRegionFile`` (external)
-    The name of a text file containing the corners of the time-evolving
-    refinement region. The lines in the file change the values of
-    ``RefineRegionLeft/RightEdge`` during the course of the simulation, and
-    the lines are ordered in the file from early times to late times.
-    The first column of data is the time index (in code units or
-    redshift, see the parameter above) for the next six columns, which
-    are the values of ``RefineRegionLeft/RightEdge``. For example, this
-    might be two lines from the text file when time is indexed by
-    redshift:
-    ::
-        0.60 0.530 0.612 0.185 0.591 0.667 0.208
-        0.55 0.520 0.607 0.181 0.584 0.653 0.201
-    In this case, the refinement region stays at the z=0.60 value
-    until z=0.55, when the box moves slightly closer to the (0,0,0)
-    corner. There is a maximum of 300 lines in the file and there is no
-    comment header line. Default: None.
-``MinimumOverDensityForRefinement`` (external)
-    These float values (up to 9) are used if the
-    ``CellFlaggingMethod`` is 2, 4 or 5. For method 2 and 4, the value is the density (baryon or particle), in code units, above which refinement occurs. When using method 5, it becomes rho [code] - 1. The elements in this array must match those in ``CellFlaggingMethod``. Therefore, if ``CellFlaggingMethod`` = 1 4 9 10, ``MinimumOverDensityForRefinement`` = 0 8.0 0 0.
-    In practice, this value is converted into a mass by
-    multiplying it by the volume of the top grid cell. The result is
-    then stored in the next parameter (unless that is set directly in
-    which case this parameter is ignored), and this defines the mass
-    resolution of the simulation. Note that the volume is of a top grid
-    cell, so if you are doing a multi-grid initialization, you must
-    divide this number by r\ :sup:`(d\*l)`\  where r is the refinement
-    factor, d is the dimensionality and l is the (zero-based) lowest
-    level. For example, for a two grid cosmology setup where a cell should be
-    refined whenever the mass exceeds 4 times the mean density of the
-    subgrid, this value should be 4 / (2\ :sup:`(3\*1)`\ ) = 4 / 8 =
-    0.5. Keep in mind that this parameter has no effect if it is
-    changed in a restart output; if you want to change the refinement
-    mid-run you will have to modify the next parameter. Up to 9
-    numbers may be specified here, each corresponding to the respective
-    ``CellFlaggingMethod``. Default: 1.5
-``MinimumMassForRefinement`` (internal)
-    This float is usually set by the parameter above and so is labeled
-    internal, but it can be set by hand. For non-cosmological simulations, it can be the easier refinement criteria to specify. It is the mass above
-    which a refinement occurs if the ``CellFlaggingMethod`` is
-    appropriately set. For cosmological simulations, it is specified in units such
-    that the entire mass in the computational volume is 1.0, otherwise it is in code units. There are 9 numbers here again, as per the
-    above parameter. Default: none
-``MinimumMassForRefinementLevelExponent`` (external).
-    This parameter modifies the behaviour of the above parameter. As it
-    stands, the refinement based on the ``MinimumMassForRefinement``
-    (hereafter Mmin) parameter is complete Lagrangian. However, this
-    can be modified. The actual mass used is
-    Mmin\*r\ :sup:`(l\*alpha)`\  where r is the refinement factor, l is
-    the level and alpha is the value of this parameter
-    (``MinimumMassForRefinementLevelExponent``). Therefore a negative value
-    makes the refinement super-Lagrangian, while positive values are
-    sub-Lagrangian. There are up to 9 values specified here, as per
-    the above two parameters. Default: 0.0
-``SlopeFlaggingFields[#]`` (external)
-    If ``CellFlaggingMethod`` is 1, and you only want to refine on the
-    slopes of certain fields then you can enter the number IDs of the
-    fields. Default: Refine on slopes of all fields.
-``MinimumSlopeForRefinement`` (external)
-    If ``CellFlaggingMethod`` is 1, then local gradients are used as the
-    refinement criteria. All variables are examined and the relative
-    slope is computed: abs(q(i+1)-q(i-1))/q(i). Where this value
-    exceeds this parameter, the cell is marked for refinement. This
-    causes problems if q(i) is near zero. This is a single integer (as
-    opposed to the list of five for the above parameters). Entering
-    multiple numbers here correspond to the fields listed in
-    ``SlopeFlaggingFields``. Default: 0.3
-``MinimumPressureJumpForRefinement`` (external)
-    If refinement is done by shocks, then this is the minimum
-    (relative) pressure jump in one-dimension to qualify for a shock.
-    The definition is rather standard (see Colella and Woodward's PPM
-    paper for example) Default: 0.33
-``MinimumEnergyRatioForRefinement`` (external)
-    For the dual energy formalism, and cell flagging by
-    shock-detection, this is an extra filter which removes weak shocks
-    (or noise in the dual energy fields) from triggering the shock
-    detection. Default: 0.1
-``MetallicityRefinementMinLevel`` (external)
-    Sets the minimum level (maximum cell size) to which a cell enriched
-    with metal above a level set by ``MetallicityRefinementMinMetallicity``
-    will be refined. This can be set to any level up to and including
-    ``MaximumRefinementLevel``. (No default setting)
-``MetallicityRefinementMinMetallicity`` (external)
-    This is the threshold metallicity (in units of solar metallicity)
-    above which cells must be refined to a minimum level of
-    ``MetallicityRefinementMinLevel``. Default: 1.0e-5
-``MustRefineRegionMinRefinementLevel`` (external)
-    Minimum level to which the rectangular solid volume defined by
-    ``MustRefineRegionLeftEdge`` and ``MustRefineRegionRightEdge`` will be
-    refined to at all times. (No default setting)
-``MustRefineRegionLeftEdge`` (external)
-    Bottom-left corner of refinement region. Must be within the overall
-    refinement region. Default: 0.0 0.0 0.0
-``MustRefineRegionRightEdge`` (external)
-    Top-right corner of refinement region. Must be within the overall
-    refinement region. Default: 1.0 1.0 1.0
-``MustRefineParticlesRefineToLevel`` (external)
-    The maximum level on which ``MustRefineParticles`` are required to
-    refine to. Currently sink particles and MBH particles are required
-    to be sitting at this level at all times. Default: 0
-``MustRefineParticlesRefineToLevelAutoAdjust`` (external)
-    The parameter above might not be handy in cosmological simulations
-    if you want your ``MustRefineParticles`` to be refined to a certain
-    physical length, not to a level whose cell size keeps changing.
-    This parameter (positive integer in pc) allows you to do just that.
-    For example, if you set ``MustRefineParticlesRefineToLevelAutoAdjust``
-    = 128 (pc), then the code will automatically calculate
-    ``MustRefineParticlesRefineToLevel`` using the boxsize and redshift
-    information. Default: 0 (FALSE)
-``FluxCorrection`` (external)
-    This flag indicates if the flux fix-up step should be carried out
-    around the boundaries of the sub-grid to preserve conservation (1 -
-    on, 0 - off). Strictly speaking this should always be used, but we
-    have found it to lead to a less accurate solution for cosmological
-    simulations because of the relatively sharp density gradients
-    involved. However, it does appear to be important when radiative
-    cooling is turned on and very dense structures are created.
-    It does work with the ZEUS
-    hydro method, but since velocity is face-centered, momentum flux is
-    not corrected. Species quantities are not flux corrected directly
-    but are modified to keep the fraction constant based on the density
-    change. Default: 1
-``InterpolationMethod`` (external)
-    There should be a whole section devoted to the interpolation
-    method, which is used to generate new sub-grids and to fill in the
-    boundary zones of old sub-grids, but a brief summary must suffice.
-    The possible values of this integer flag are shown in the table
-    below. The names specify (in at least a rough sense) the order of
-    the leading error term for a spatial Taylor expansion, as well as a
-    letter for possible variants within that order. The basic problem
-    is that you would like your interpolation method to be:
-    multi-dimensional, accurate, monotonic and conservative. There
-    doesn't appear to be much literature on this, so I've had to
-    experiment. The first one (ThirdOrderA) is time-consuming and
-    probably not all that accurate. The second one (SecondOrderA) is
-    the workhorse: it's only problem is that it is not always
-    symmetric. The next one (SecondOrderB) is a failed experiment, and
-    SecondOrderC is not conservative. FirstOrderA is everything except
-    for accurate. If HydroMethod = 2 (ZEUS), this flag is ignored, and
-    the code automatically uses SecondOrderC for velocities and
-    FirstOrderA for cell-centered quantities. Default: 1
-    ::
-              0 - ThirdOrderA     3 - SecondOrderC
-              1 - SecondOrderA    4 - FirstOrderA
-              2 - SecondOrderB  
-``ConservativeInterpolation`` (external)
-    This flag (1 - on, 0 - off) indicates if the interpolation should
-    be done in the conserved quantities (e.g. momentum rather than
-    velocity). Ideally, this should be done, but it can cause problems
-    when strong density gradients occur. This must(!) be set off for
-    ZEUS hydro (the code does it automatically). Default: 1
-``MinimumEfficiency`` (external)
-    When new grids are created during the rebuilding process, each grid
-    is split up by a recursive bisection process that continues until a
-    subgrid is either of a minimum size or has an efficiency higher
-    than this value. The efficiency is the ratio of flagged zones
-    (those requiring refinement) to the total number of zones in the
-    grid. This is a number between 0 and 1 and should probably by
-    around 0.4 for standard three-dimensional runs. Default: 0.2
-``NumberOfBufferZones`` (external)
-    Each flagged cell, during the regridding process, is surrounded by
-    a number of zones to prevent the phenomenon of interest from
-    leaving the refined region before the next regrid. This integer
-    parameter controls the number required, which should almost always
-    be one. Default: 1
-``RefineByJeansLengthSafetyFactor`` (external)
-    If the Jeans length refinement criterion (see ``CellFlaggingMethod``)
-    is being used, then this parameter specifies the number of cells
-    which must cover one Jeans length. Default: 4
-``JeansRefinementColdTemperature`` (external)
-    If the Jeans length refinement criterion (see ``CellFlaggingMethod``)
-    is being used, and this parameter is greater than zero, it will be
-    used in place of the temperature in all cells. Default: -1.0
-``StaticRefineRegionLevel[#]`` (external)
-    This parameter is used to specify regions of the problem that are
-    to be statically refined, regardless of other parameters. This is mostly
-    used as an internal mechanism to keep the initial grid hierarchy in
-    place, but can be specified by the user. Up to 20 static regions
-    may be defined (this number set in ``macros_and_parameters.h``), and
-    each static region is labeled starting from zero. For each static
-    refined region, two pieces of information are required: (1) the
-    region (see the next two parameters), and (2) the level at which
-    the refinement is to occurs (0 implies a level 1 region will always
-    exist). Default: none
-``StaticRefineRegionLeftEdge[#]``, ``StaticRefineRegionRightEdge[#]`` (external)
-    These two parameters specify the two corners of a statically
-    refined region (see the previous parameter). Default: none
-``AvoidRefineRegionLevel[#]`` (external)
-    This parameter is used to limit the refinement to this level in a
-    rectangular region.  Up to MAX_STATIC_REGIONS regions can be used.
-``AvoidRefineRegionLeftEdge[#]``, ``AvoidRefineRegionRightEdge[#]`` (external) 
-    These two parameters specify the two corners of a region that
-    limits refinement to a certain level (see the previous
-    parameter). Default: none
-``RefineByResistiveLength`` (external)
-    Resistive length is defined as the curl of the magnetic field over
-    the magnitude of the magnetic field. We make sure this length is
-    covered by this number of cells. Default: 2
-``LoadBalancing`` (external)
-    Set to 0 to keep child grids on the same processor as their
-    parents. Set to 1 to balance the work on one level over all
-    processors. Set to 2 or 3 to load balance the grids but keep them
-    on the same node. Option 2 assumes grouped scheduling, i.e. proc #
-    = (01234567) reside on node (00112233) if there are 4 nodes. Option
-    3 assumes round-robin scheduling (proc