Commits

Cheng Soon Ong committed b93f8be

Rough draft of startup documentation.

Comments (0)

Files changed (1)

-== Welcome ==
-
-Welcome to your wiki! This is the default page we've installed for your convenience. Go ahead and edit it.
-
-=== Wiki features ===
-
-This wiki uses the [[http://www.wikicreole.org/|Creole]] syntax, and is fully compatible with the 1.0 specification.
-
-The wiki itself is actually a hg repository, which means you can clone it, edit it locally/offline, add images or any other file type, and push it back to us. It will be live immediately.
-
-Go ahead and try:
-
-{{{
-$ hg clone http://bitbucket.org/chengsoon.ong/pythongrid/wiki/
-}}}
-
-Wiki pages are normal files, with the .wiki extension. You can edit them locally, as well as creating new ones.
-
-=== Syntax highlighting ===
-
-You can also highlight snippets of text, we use the excellent [[http://www.pygments.org/|Pygments]] library.
-
-Here's an example of some Python code:
-
-{{{
-#!python
-
-def wiki_rocks(text):
-	formatter = lambda t: "funky"+t
-	return formatter(text)
-}}}
-
-You can check out the source of this page to see how that's done, and make sure to bookmark [[http://pygments.org/docs/lexers/|the vast library of Pygment lexers]], we accept the 'short name' or the 'mimetype' of anything in there.
-
-Have fun!
+== Why ==
+
+We focus on what is usually known as [[http://en.wikipedia.org/wiki/Embarrassingly_parallel|embarassingly parallel]] problems, where there is no communication between the different tasks. These problems are particularly suited to cluster computing, since there is essentially no overhead for parallelisation. We provide a python interface for submitting and collecting cluster jobs via the [[http://www.drmaa.org/|DRMAA]] framework, instead of using the standard command line interface (e.g. qsub or bsub). This means that the user can write all the control flow in python.
+
+== Implementation notes ==
+
+This project uses [[http://code.google.com/p/drmaa-python/|drmaa-python]] to communicate with the queue master and depends on the file system (via cPickle) to transmit variables between the master node and the compute nodes.
+
+== Quick start example ==
+
+{{{
+#!python
+
+def computeFactorial(n):
+    """
+    computes factorial of n
+    """
+    ret=1
+    for ix in range(n):
+        ret=ret*(ix+1)
+    return ret
+
+inputvec = [[3], [5], [10], [15]]
+print 'Computing the factorials of %s' % str(inputvec)
+jobs=[]
+
+for input in inputvec:
+    job = Job(computeFactorial, input)
+    job.h_vmem="300M"
+    jobs.append(job)
+
+# All the work is done here
+processed = process_jobs(jobs)
+
+for (ix, job) in enumerate(processed):
+    print "Job #", ix, "- ret: ", str(job.ret)[0:10]
+
+}}}
+