Wiki

Clone wiki

Simflowny / UserGuide_SAMRAIParameters

SAMRAI parameter file configuration

Simflowny generates code for SAMRAI framework for mesh, particles and mixed (particles and mesh) problems. All of them have parameters in common, but there are some differences.

The following sections explain all the parameters. If the parameter does not exist in the sample file generated by Simflowny means that the option is not available for that kind of simulation.

First of all, SAMRAI parameter files are structured by what is called database. A database in no more than a group of parameters (and also other databases) that have something in common. An example may be:

CartesianGeometry {
    domain_boxes = [ (0, 0) , (99, 99) ]
    x_lo = 0.0, 0.0
    x_up = 10.0, 10.0
    periodic_dimension = 0, 0
}

Problem database

In this database, the first parameters are the problem ones, those specified by the user during the problem creation.

In some problems, random number generation is used, in order to allow result reproducibility there is a parameter to set the seed of the generator. This parameter is random_seed.

The subcycling parameter allows AMR simulations to shorten execution time. Currently, the available options are as follows:

  • DISABLED. This is the default option, in which the simulation step is the same for every level in the AMR hierarchy. So, the coarsest level time step is the same lenght as the finest one.
  • BERGER-OLIGER. This is the classical subcycling algorithm and fastest implemented in Simflowny. The time step for every level in the AMR hierarchy is proportional to the refinement ratio, so coarser level steps covers more time than finer ones. Therefore, sophisticated synchronization is used between levels.

This database may contain two databases depending on the simulation type.

Particles database

There are some common parameters:

  • print_average is a boolean parameter to indicate if the average of the particle field values are calculated in the underlying mesh.
  • influenceRadius is the parameter to indicate the radius in which the particles have influence. The value must be set in domain units.

For each species there is a specific database with the following parameters:

  • number_of_particles is used to set the number of particles. The total number of particles is a product of those values.
  • domain_offset_factor sets the offset for the initial particle regarding the domain. The parameter is a factor of a cell, so values must be between 0 and 1.
  • box_min (Only for BOX particle distribution) to set the minimum coordinates on in a box distribution
  • box_max (Only for BOX particle distribution) to set the maximum coordinates on in a box distribution
  • normal_mean (Only for NORMAL particle distribution) Sets the mean value of the distribution in each dimension.
  • normal_stddev (Only for NORMAL particle distribution) Sets the standard deviation value of the distribution in each dimension.
  • particle_distribution indicates how the particles are set in the domain. Four options are available. REGULAR will set the particles in as a regular mesh, with a constant separation in every axis for all the particles. STAGGERED similar to regular distribution but with a particle displacement in consecutive rows. BOX distribution sets a regular distribution of particles inside a box subdomain. NORMAL will set particles following a normal distribution with parameters normal_mean and normal_stddev. RANDOM will simply set a random position for all the cells using an uniform distribution function.

Regridding database

SAMRAI provides Adaptive Mesh Refinement (and also Fixed Mesh Refinement). A programmer can provide any algorithm to tag the mesh cells to refine. In the current version, Simflowny provides two tagging algorithms that are configured in this database. In the example file automatically created when generatig code, the two regridding configurations are available.

Independently from the regridding tagging algorithm, the user may also change the interpolation type to use setting the following parameters:

  • interpolator. The value chosen must be one of these three: "LINEAR_REFINE", "CUBIC_REFINE", "QUINTIC_REFINE".

There are currently three regridding strategies, Gradient, Function and Shadow. They all have the following common parameter:

  • regridding_buffer indicates the number of extra cells surrounding tags marked for regridding. It makes the finer levels bigger.
  • regridding_min_level indicates the first level for AMR regridding.
  • regridding_max_level indicates the last level for AMR regridding.
  • regridding_type used to select the specific regridding criteria.

Regarding the specific regridding parameters:

  • Gradient. A cell will be refined when the gradient of the field on any neighbour cell is greater than the next nearest cell gradient with a compression factor and an offset. The conditional formula is: \(\nabla(f_i) > C * MIN(\nabla(f_i+1), \nabla(f_i-1)) + O * dx^2\). The parameters are:
    • regridding_field is the field in which to check gradient.
    • regridding_compressionFactor is the compression factor for the tagging algorithm (C in the formula).
    • regridding_mOffset is the offset for the tagging algorithm (O in the formula).
  • Function. The cell is refined if the provided field is greater than a given threshold. The parameters are
    • regridding_field is the field in which to check the threshold.
    • regridding_threshold is the threshold for the field.
  • Shadow. Two equivalent fields are compared, if the relative difference is greater than a given error, the cell is to be refined.
    • regridding_fields used to set the two fields to compare.
    • regridding_error is the error relative threshold.

Note: The fields used in the refinement criteria cannot be from analysis.

Example

Problem {
    particles {
        water {
            //Number of particles by dimension
            number_of_particles = 20, 20
            //Domain offset factor (In particle separation units [0, 1])
            domain_offset_factor = 0.5, 0.5
            normal_mean = 1, 2
            normal_stddev = 0.7, 0.7
            //Particle distribution
            particle_distribution = "NORMAL"  //REGULAR or RANDOM
        }
        //Print particles average in cells
        print_average = FALSE
        //Influence radius (in domain units, recommendend value = 1.3 * particle separation)
        influenceRadius = 0.26
    }
    lambda = 0.000001625
    mu = 0.0009
    gamma = 7
    pfx = 0
    pfy = -9.81
    rho0 = 1028
    c0 = 65
    Paux = 0.0
    tend = 10
    random_seed = 0
    subcycling = "BERGER-OLIGER" // "BERGER-OLIGER", "DISABLED"
    //Regridding options
    regridding {
       refine_interpolator = "LINEAR_REFINE"
       //Common regridding parameters
       regridding_buffer = 3        //Number of extra cells surrounding regridding criteria
       regridding_min_level = 1     //First level number to check regridding from
       regridding_max_level = 10    //Last level number to check regridding from
       //regridding_type = "GRADIENT"
       //regridding_field = "rho"
       //regridding_compressionFactor = 3
       //regridding_mOffset = 50
       regridding_type = "FUNCTION"
       regridding_function_field = "E"
       regridding_threshold = 4000.0
       //regridding_type = "SHADOW"
       //regridding_fields = "Sfd_x" "rk3Sfd_x"
       //regridding_error = 1e-5
    }
}

Main database

In this database the user can configure the simulation common evolution parameters.

  • output_interval is the interval for screen output for simulation current state.
  • timer_output_interval is the interval for screen output of the simulation timers.

SAMRAI provides the capability to stop a simulation and restart it from stored snapshots. This feature is configured through the following parameters:

  • start_from_restart it indicates if the simulation is restating from a stored snapshot.
  • restart_interval if this value is greater than zero it will store snapshots for a future restart with the providen frequency.
  • restart_iteration if the simulation is restarting from snapshot, this parameter indicates from which snapshot number it restarts.
  • restart_dirname sets the name of the directory for the restart snapshots.
  • rebalance_processors sets if the processors must force a rebalance of patches after restarting from a checkpoint.

The simulation time step is set through:

  • dt. Indicates the time increment between each step in the simulation.

There is the possibility to change clustering and load balancing strategies. If not sure, the following parameters must be left as default values:

  • clustering_type indicates the way to partition the domain.
  • partitioner_type indicates how the partitions are distributed in processors.

Example

Main {
            // output configuration
            output_interval = 1            // zero to turn off (In coarser level cycle units)
            timer_output_interval = 1            // zero to turn off
            // restart configuration
            start_from_restart = TRUE
            restart_interval = 5000           // zero to turn off
            restart_iteration = 25000          // iteration to restart from (if start_from_restart = TRUE)
            restart_dirname = "checkpoint.restart" //To store the checkpoints and to take the checkpoint for a restart
            rebalance_processors = FALSE
            // dt
            dt = 0.001
            //Clustering type
            clustering_type = "BergerRigoutsos" // TileClustering or BergerRigoutsos
            //Load Balancer
            partitioner_type = "CascadePartitioner" // CascadePartitioner, TreeLoadBalancer or ChopAndPackLoadBalancer
}

In this example, restart capability is active, creating snapshots every 5000 time steps. Furthermore, this simulation already restarts from snapshot 25000 (providing it exists), continuing the simulation from that time on. In this case regridding is inactive (regridding_interval = 0).

FileWriter database

Simflowny provides different output formats depending on the problem characteristics.

All simulations provides at least one full_dump_mesh/particles output, where the whole domain is writen to disk for cells or particle species.

  • hdf5_dump_interval. To configure the interval for the output files with data.
  • hdf5_dump_dirname. The directory name for the files.
  • variables. The variables to write to disk

When available, there will be a slicing capability for writing 2D slices of 3D data. The user can set as many slices as desired. Each slice has its own properties and must be defined inside its own section slice_x, being x the number of the slice, beginning with 0 and being correlative. The properties of a slice are the following:

  • plane_normal_axis. To select the axis used as the normal of the slice plane. Allowed values are 1, 2, or 3.
  • distance_to_origin. Used to set the location of the plane. The user must set the value in domain units.
  • variables. The list of variables to dump for the slice.
  • hdf5_dump_interval. To configure the interval for the output files with data.
  • hdf5_dump_dirname. The directory name for the files.
  • activate_analysis. If set to true, the analysis variables are calculated in the interval of the current slice, so the user can print them out if selected.

Another extra capability is providen to extract a spherical surface from 3D problems. The user can set as many spheres as desired. Each sphere has its own properties and must be defined inside its own section sphere_x, being x the number of the sphere, beginning with 0 and being correlative. The properties of a sphere are the following:

  • center. The center of the sphere in domain units
  • radius. The radius of the sphere in domain units.
  • resolution. The number of cells for the 2D projection. Projection domain is \([2 * \Pi, \Pi]\), corresponding to angles theta and phi (Description in https://en.wikipedia.org/wiki/Sphere#Equations_in_three-dimensional_space).
  • variables. The list of variables to dump for the sphere.
  • hdf5_dump_interval. To configure the interval for the output files with data.
  • hdf5_dump_dirname. The directory name for the files.
  • activate_analysis. If set to true, the analysis variables are calculated in the interval of the current sphere, so the user can print them out if selected.

It is also posible to output point data. The user can set as many points as desired. Each point has its own properties and must be defined inside its own section point_x, being x the number of the point, beginning with 0 and being correlative. The properties of a point are the following:

  • variables. The list of variables to dump for the point.
  • coordinates. Coordinate of the point in domain untis
  • ascii_dump_interval. To configure the interval for the output files with data.
  • ascii_dump_dirname. The directory name for the files.
  • activate_analysis. If set to true, the analysis variables are calculated in the interval of the current point, so the user can print them out if selected.

The last output capability is integration from the whole domain cells. The user can set as many integrations as desired. Each integration has its own properties and must be defined inside its own section integration_x, being x the number of the sphere, beginning with 0 and being correlative. The properties of integration are the following:

  • variables. The list of variables to integrate.
  • calculation. The list of calculations to apply to variables. INTEGRAL, L2NORM, MIN, MAX, ABSMIN and ABSMAX are available
  • ascii_dump_interval. To configure the interval for the output files with data.
  • ascii_dump_dirname. The directory name for the files.
  • activate_analysis. If set to true, the analysis variables are calculated in the interval of the current integration, so the user can print them out if selected.

TimeRefinementIntegrator database

In this database, the time refinement algorithm parameters are set (when available).

  • tag_buffer. Array of integer values (one for each level that may be refined) representing the number of cells by which tagged cells are buffered before clustering into boxes.
  • regridding_interval in case of adaptive regridding it indicates when the regridding must be reset.
  • read_on_restart indicates if those parameters must be read from the parameters file in case of restarting the simulation from a checkpoint.

CartesianGeometry database

In this database the simulation domain is set. Simflowny automatically sets the physical domain with the data set in the problem, but the size of the mesh should be set by the user.

  • domain_boxes sets the mesh size, from the initial values to the final ones. The ghost cells needed for calculations are automatically added to the mesh afterwards.
  • x_lo, x_up indicates the domain size in problem units. Simflowny sets the problem values, but it can be also modified by the user.
  • periodic_dimension are already set by Simflowny depending on the problem boundary conditions. Indicates the dimensions that have periodical boundary. It should not be changed by a user since boundaries that are not periodic are implemented within the code. If the user wants to change the boundary conditions is strongly advised to modify the problem and generate the code again.

Example

CartesianGeometry {
            domain_boxes = [ (0, 0) , (399, 399) ]
            x_lo = -5.0, 0.0     // lower end of computational domain.
            x_up = 5.0, 60.0  // upper end of computational domain.
            periodic_dimension = 1, 0
}

In the previous example, a 2D simulation is set to use 400x400 mesh in a \([-5, 5]x[0, 60]\) domain. Notice the first dimension has periodic boundaries, but not the second one, whose boundaries must be specified inside the source code.

StandardTagAndInitialize database

This database is used for regridding purposes. By default, there is an example of a combination for fixed and adaptive regridding. Multiple criteria can be set, and even it can be activated from certain cycle. Therefore, the subsection at_0 defines the start cycle parameter for this regridding configuration to 0. Once defined the starting time for regridding, the criteria is defined. "REFINE_BOXES" stands for fixed regridding, while "GRADIENT_DETECTOR" stands for adaptive.

The box refinement is configured at each level, indicating the begin and end of each coordinate of the domain mesh to regrid.

  • cycle is used to set the cycle from which the regridding is active.
  • tagging_method is the criteria for tagging cells. REFINE_BOXES for a fixed regridding based on boxes. GRADIENT_DETECTOR for adaptive mesh refinement, whose parameters are explained and configured in database problem->regridding
  • boxes defines the boxes for regridding at a given level. The values are in domain units.

Example

StandardTagAndInitialize{
            at_0{
                    cycle=0      //Active from cycle 0
                    tag_0{       //FMR
                            tagging_method = "REFINE_BOXES"
                            level_0 {
                                box_0 {
                                            x_lo = -0.5, -0.5     // lower end of refinement box.
                                            x_up = 0.5, 0.5  // upper end of refinement box.
                                    }
                            }
                            level_1 {
                                box_0 {
                                            x_lo = -0.25, -0.25     // lower end of refinement box.
                                            x_up = 0.25, 0.25  // upper end of refinement box.
                                    }
                            }
                    }
                    tag_1{       //AMR
                            tagging_method = "GRADIENT_DETECTOR"
                    }
            }
}

In the previous example, a combination of AMR + FMR is set. The regridding is active from cycle 0. For the fixed mesh refinement, the mesh can reach up to 3 levels: the original one, a sub-mesh between \([-0.5, 0.5]^2\), and another in \([-0.25, 0.25]^2\). The adaptive mesh refinement is also active, in tag_1, but its parameters must be defined in the problem->regridding database.

PatchHierarchy database

This database allows the set up of patches, which are SARMAI internal mesh divisions.

  • max_levels indicates the maximum number of mesh levels. A value of 1 indicates that regridding is disabled. When a value is 2 or greater regridding is active.
  • ratio_to_coarser is used to set the regridding factor for the next level of refinement. More than one line is allowed to provide a different ratio for the levels.
  • largest_patch_size, smallest_patch_size set up the maximum and minimum size of patches in each level. For performance purposes it is recommended that the maximum size for level_0 was greater than the mesh size.
  • proper_nesting_buffer indicates the minimum number of cells between regridding levels.

Example

PatchHierarchy {
            max_levels = 3        // Maximum number of levels in hierarchy.
            ratio_to_coarser {            // vector ratio to next coarser level
                level_1 = 2, 2
                level_2 = 3, 2
            }

            largest_patch_size {
                level_0 = 1000, 1000  // largest patch allowed in hierarchy
            }
            smallest_patch_size {
                level_0 = 10, 10  // smallest patch allowed in hierarchy
            }
            proper_nesting_buffer = 2 //minimum de number of cells between regridding levels
}

In the previous example, the regridding can reach up to 3 levels (if regridding is active in main database). For the first level, the ratio for the new sub-mesh is 2x2, but for the next one is 3x2. For instance, if a base mesh of 100x100 is used, the first level sub-meshes are defined in a 200x200 mesh domain and the last level sub-meshes in a 600x400 mesh domain.

GriddingAlgorithm database

This is a database used in SAMRAI to configure regridding related configurations. However, a regular user may not need to change the default values. Therefore, this database is empty.

LoadBalancer database

In current version, Simflowny uses a tree-based load balancer, whose default parameters does not need to be modified by a regular user.

TimerManager database

This database includes some time printing for profiling purposes. It details the time the simulation spends writing to disk, regridding and performing the calculations.

  • timer_list lists the active timers. The user can remove any not desired.
  • print_threshold. Timers that use up less than (print_threshold) percent of the overall run time are not printed.
  • print_processor. Print times measured on individual processors.
  • print_max. Print maximum time spent on any processor, and the processor ID that incurred the time.

Example

TimerManager{
            timer_list = "OutputGeneration", "Regridding", "Step"
            print_threshold = 5
}

The timers, OutputGeneration, Regridding, and Step are active and show results when their time is more than the 5% of the whole simulation time.

Updated