pythongrid - a high level front end to DRMAA-python
The module pythongrid.py provides wrappers that simplify
submission and collection of jobs, in a more 'pythonic' fashion.
It also uses the same interface for running jobs locally on the
same machine using threads. This may be useful for debugging purposes,
and may also be faster if the cluster is busy and the jobs are small.
- `DRMAA-python`_ (>=0.4)
- `easysvm`_ (>=0.2) [optional, used in example]
.. _`DRMAA-python`: http://code.google.com/p/drmaa-python/
.. _`easysvm` : http://www.easysvm.org
DRMAA-python is required. To check that it is available
>>> import drmaa
Untar the archive, and put pythongrid.py somewhere in $PYTHONPATH
$ tar xzvf pythongrid-0.x.tar.gz
Edit pythongrid.py to set the following two variables at the top.
PYGRID = '/absolute/path/to/pythongrid.py'
TEMPDIR = '/absolute/path/to/inputoutputdir/'
PYGRID is used on the cluster as a shell command.
TEMPDIR should be readable and writable from both submission
and cluster systems.
If you have any settings of $PYTHONPATH, set it in PYTHONPATH in
Edit your environment such that the drmaa shared library for the grid
engine is in the shared path. For example
To check that everything has installed properly
>>> import pythongrid
Four functions wrap the complexity:
- submit_jobs takes a list of jobs and sends them to the cluster
- get_status checks the jobs in a given session
- collect_jobs takes a list of job ids and collects their results
- process_jobs provides a shortcut that does both in series. It also
provides an option to run locally.
Please look at the examples for usage details.
To use a function within the module itself, one has to import the
module explicitly from within the module. This is to resolve
namespaces in python
.. _GPLv3: http://gplv3.fsf.org/
All programs in this collection are free software:
you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Copyright 2008 Cheng Soon Ong and Christian Widmer