PyModules is an alternative implementation of the
Environment Modules system
for managing software environments on research computing clusters.
It provides several new features:
- Uses simple, INI-style configuration files that share common values across
multiple versions of a package.
- Module configuations are cached in a SQLite database, and editing/publishing
are separate actions.
- Modules can be categorized. The
module availcommand lists by
category or by fulltext search on the package name or version.
- Modules can be localized by CPU architecture, for either a specific vendor
(AMD vs. Intel), SSE instruction set, or CPU model identifier.
module bincommand lists all binaries provided by a module.
PyModules is developed by the
Center for Computation and Visualization
at Brown University, and distributed freely for academic use under a
non-commercial license (see the LICENSE file for full details).
Zip archives are available from the downloads
page, or you can clone the
Bitbucket repo for the latest development version.
For installation instructions, see the INSTALL file.
The system has included an auditing system to record the module loading/unloading
by users. Although this system supported SQLite and file logging and has the related
adapters, we have decided to hard-code the system to use MySQL, since the former two
do not work well with concurrency.
When a user loads or unloads modules, the auditing system keeps track with the module
name, module version, timestamp, and whether or not it is due to a global/user-defined
module loading script. The actually logging work is done by a separate process so that
the user won't be bothered.
By default, the system suppresses all the error messages and logs them to a failout
log file at user-directory (~/.modulelog.log). To debug the system, the suppression
should be released by editing
modulelog_config['suppress-error'] in modulecfg.py.
modulelog_config['suppress-error'] defines the timeout for the auditing
system to connect to the database.
This auditing system has three "red-buttons". Unsetting
in the environment disables the failout logging in case of read-only filesystems.
ENABLE_MODULELOG in the environment or setting
to False in modulecfg.py disables the entire logging system including the failout
logging and the actual module logging.
Howison M, Shen A, Loomis A. 2013. Building Software Environments for Research
Computing Clusters. In Proceedings of the 27th Large Installation System
Administration Conference (LISA '13), Washington, DC, USA.
Please report any issues using the issue tracker at Bitbucket:
Hongkai (George) Sun