TPFileConvert.C - a binary to HDF5 converter for tracer particle data.

Author: Devin W. Silvia Date: February 12, 2009.  Affiliation:
CASA/University of CO, Boulder Homepage:


As it currently stands, in Enzo runs that make use of tracer particles
all of the tracer particle data is output as streaming binary files.
While this is the fastest and cheapest way to output the particle data
(by avoiding extra HDF5 writes and minimizing file size), it is not
terribly useful from an analysis standpoint.  To remedy this
situation, TPFileConvert.C reads in the standard tracer particle
files, parses the binary stream, and then outputs HDF5 files loaded
with the data from the simulation in the standard Enzo practice of
having an HDF5 file for each data dump.  The data can then be read and
analyzed using the simulator's code of choice.

This code was built off of a chunk of core code first written by Brian
W. O'Shea.


TPFileConvert.C is a C++ code and as such must be compiled before it
can be run.  An example of how to compile the code is the following:

g++ -I/opt/local/include -L/opt/local/lib -lhdf5 TPFileConvert.C -o tp.exe

Note the -I and -L flags in the call to the g++ compiler.  Since
TPFileConvert.C creates and writes to HDF5 files, you must have HDF5
installed on your machine and you must tell the compiler where the
code can find the HDF5 libraries.  Note that you may need to add the
compiler flag "-DH5_USE_16_API" if you are using a newer (v1.8+)
version of HDF5. In addition, g++ can be substituted with whatever C++
compiler your machine uses.

Once the code has been compiled you will find that a 'tp.exe'
executable has been created.  At this point, all that needs to be done
is the following:


This will run the executable and carry out the binary to HDF5


It is important to mention that as it stands currently,
TPFileConvert.C requires several flags to be preset by the user before
compiling to ensure successful conversion.

   1) ENZO_INTERNAL_FLOAT and HDF5_MEM_FLOAT -- these two flags (which
   need to agree) should correspond to Enzo's internal floating-point
   precision during simulation runtime (i.e., the simulation that generated
   the tracer particle data you're using).  If this is not set correctly,
   TPFileConvert.C will incorrectly step through the files and exceedingly
   puzzling results will occur.

   2) ENZO_INTERNAL_INT and HDF5_MEM_INT -- as with the floating-point flags,
   these correspond to Enzo's internal integer precision during simulation 

   3) SIMULATION_GRIDRANK -- this flag tells TPFileConvert.C what
   the dimensionality of the Enzo simulation is.  This should be
   equal to TopGridRank in the Enzo parameter file.

   4) TP_VELOCITY -- this flag tells TPFileConvert.C whether or not
   the user wants tracer particle velocity data to be included in the
   HDF5 files.  TP_VELOCITY can only be used if the binary data files
   actually contain tracer particle velocity data.  If the user
   doesn't wish to have the velocity information included in the
   created HDF5 files or there is no velocity information included
   within the binary data files then this flag should be set to

   5) TOTALPROCS -- this defines the number of processors that the
   simulation was run with as this corresponds to the number of TP
   binary files that need to be read.  Set this to be the same as the
   number of processors that were used to ensure successful

If these variables are not set correctly, then you will undoubtedly
get spurious results -- check these first if the code errors.

IV. TPFileConvert.C OUTPUT

When the executable is run, information about the progress of the file
conversion will print to the screen.  This will begin with which
binary file the code is currently operating on as well as the names of
the files it creates for that processor and ends with the names of the
processor-combined HDF5 files for each data dump.

Note that the files which the code creates for each processor are only
needed as an intermediate step and are deleted when the code finishes.
The final output consists of files that follow the format of
TPDD####.h5 where #### will indicate which data dump that file
corresponds to.

Each HDF5 file will contain arrays of particle data that should be as
long as the number of tracer particles used in the simulation.  These
arrays are named densvals, tempvals, xvals, yvals, and zvals which
correspond to density, temperature, x-position, y-position,
z-position.  If the user has also included velocity data then these
will be named vxvals, vyvals, and vzvals which are, not surprisingly,
x-velocity, y-velocity, and z-velocity.

The output files should be easily read by whatever analysis language
is desired provided the correct HDF5 file reading commands are used.