It would be nice to avoid having both an mdb, and three separate files (option list, submit script, run script) for each machine. These really belong together.
The following may be possible: 1. An option list consists of a set of key-value pairs. It should be straightforward to put these into the mdb. 2. The submit script consists essentially of a single command, and various options that are set via comments. These options could also be set via command line options to qsub, and thus be stored in the mdb. In fact, Orca (Sharcnet) already requires this, and only has a trivial submit script. 3. All run scripts follow the same pattern: some commands to change the environment, some commands to mangle MPI node lists, and an mpirun command. Interspersed is a lot of boilerplate. These could be abstracted out into two or three MDB entries specifying these commands.
The result would be that all information is stored in the mdb, and no additional files need to be tracked.
To protect against mdb modifications while a job is waiting, we could/should store a copy of the mdb entry with each simulation.
If the current mdb format is not convenient for storing this information (but why should it not?), then we could examine other options, e.g. based on XML or SqLite. (There are e.g. XML-like formats that have the same semantics as XML, but use a slightly different syntax that is friendlier for humans, but is as easy to parse for machines.) XML and friends would give us a hierarchy, which INI doesn't.