Commits

Eric Rochester committed 3f11e12

Added initial notes.

  • Participants
  • Parent commits 234e84a

Comments (0)

Files changed (1)

+
+# dist-futures
+
+This is a library for Python 3.2 providing a `concurrent.futures` Executor for
+distributed processing. For more information on the interface, see [the
+documentation for
+concurrent.future](http://docs.python.org/py3k/library/concurrent.futures.html).
+
+# Installing
+
+You can install this using standard Python tools: easy\_install, pip, or
+setup.py install.
+
+# Workers
+
+## Authorization/Security
+
+# File Sharing
+
+# Notes
+
+ * authorization /security
+ * serialization
+ * look at process pools
+ * moving data around (file sandbox)
+ * executor
+   * takes worker machines as inputs
+   * distributes files
+   * queues jobs
+   * either starts workers with ssh or talks to existing processes
+   * serializes inputs
+   * heartbeat, handles failures
+   * listens for and accepts outputs
+ * worker
+   * listens for tasks
+   * executes them
+   * returns results
+   * takes new files
+   * manages and deletes unused files (expires)
+ * look at:
+   * http://docs.python.org/py3k/library/concurrent.futures.html
+   * http://hg.python.org/cpython/file/fecf9e6d7630/Lib/concurrent
+   * http://hg.python.org/cpython/file/default/Lib/concurrent/futures/thread.py
+   * http://hg.python.org/cpython/file/default/Lib/concurrent/futures/process.py
+