Source

pythongrid /

Filename Size Date modified Message
test
2.7 KB
486 B
3.4 KB
1.8 KB
2.4 KB
14.6 KB
14.5 KB
22.3 KB
256 B
===================================================
pythongrid - a high level front end to DRMAA-python
===================================================

.. contents::

Introduction
============
The module pythongrid.py provides wrappers that simplify 
submission and collection of jobs, in a more 'pythonic' fashion.
It also uses the same interface for running jobs locally on the
same machine using threads. This may be useful for debugging purposes,
and may also be faster if the cluster is busy and the jobs are small.

Installation
============

Dependencies
------------

- `DRMAA-python`_ (>=0.4)
- `easysvm`_  (>=0.2) [optional, used in example]

.. _`DRMAA-python`: http://code.google.com/p/drmaa-python/
.. _`easysvm` : http://www.easysvm.org


Install
-------

DRMAA-python is required. To check that it is available

    >>> import drmaa

Untar the archive, and put pythongrid.py somewhere in $PYTHONPATH

    $ tar xzvf pythongrid-0.x.tar.gz

Edit pythongrid.py to set the following two variables at the top.

    PYGRID = '/absolute/path/to/pythongrid.py'
    TEMPDIR = '/absolute/path/to/inputoutputdir/'

PYGRID is used on the cluster as a shell command.
TEMPDIR should be readable and writable from both submission
and cluster systems.

Edit your environment such that the drmaa shared library for the grid
engine is in the shared path. For example

    export LD_LIBRARY_PATH=/usr/local/sge/lib/lx26-amd64/:$LD_LIBRARY_PATH

To check that everything has installed properly

    >>> import pythongrid



Usage
=====

Four functions wrap the complexity:

- submit_jobs takes a list of jobs and sends them to the cluster
- get_status checks the jobs in a given session
- collect_jobs takes a list of job ids and collects their results
- process_jobs provides a shortcut that does both in series. It also
  provides an option to run locally.

Please look at the examples for usage details.

To use a function within the module itself, one has to import the
module explicitly from within the module. This is to resolve
namespaces in python

License
=======

GPLv3_

.. _GPLv3: http://gplv3.fsf.org/

All programs in this collection are free software: 
you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

Copyright 2008 Cheng Soon Ong and Christian Widmer
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.