asyncplot /

Filename Size Date modified Message
.makefiles
asyncplot
51 B
96 B
4.4 KB
973 B

Async Plotter

Simple two-process client/server plotting with the following features:

  • Plotting does not slow down calculations.
  • User maintains control of the calculation (i.e. KeyboardInterrupts work).
  • Auto-launch client from the server.
    • The user defines the plotting code in a (picklable) class, and provides the class to the Server() constructor. The Server passes the pickled Plotter to the client when the client process is launched in a new python interpreter. The python interpreter command line options for the client are passed to the Server constructor at the time of instantiation. Default values are: (ex. ["ipython", "--pylab=osx", "-c"])

How to Use

  1. Comment out #import mmf on line 10 in server.py.

2. Install asyncplot to place the libraries on your path. Go to the root asyncplot hg directory and enter this on the commandline: > pip install --upgrade -e . 3. Open two terminal windows. In the first one do this: > python >>> import example1 as e >>> e.run()

The plot window will open.

4. In the second terminal enter this: > python >>> import example1 as e >>> e.run_client()

  1. Go to the server terminal and press Enter. The plot window will animate.

Note: the run_client_example() function in client.py and run_server_example() function in server.py do not work together as a running pair yet.

To Do

Prototype for asynchronous plotting with separate processes using sockets

Still Needs:

  • Comprehensive testing
  • Logging (print debug messages in debug mode for checking network problems)
  • Profile performance.
  • Auto-launch client from the server.
    • The user defines the plotting code in a (picklable) class, and provides the class to the Server() constructor. The Server passes the pickled Plotter to the client when the client process is launched in a new python interpreter. The python interpreter command line options for the client are passed to the Server constructor at the time of instantiation. Default values are: (ex. ["ipython", "--pylab=osx", "-c"])
  • Configuration (hostname, port specification etc.)

I think that these have been dealt with, but they need testing:

  • socket buffer overflow (recv)
  • cleanup thread and socket command line termination
  • error handling. ex. when a client disconnects then server listen continues, network errors
  • multiple clients (plotting)

Other Design Approaches

Threading Solution

The simplest approach is a multi-thread approach. In principle, one can run the computations in the main thread and plotting in a separate thread. This solution is sketched in thread.py but fails with most matplotlib backends due to their requirement of running in the main thread. A quick work-around is to run the computation in a secondary thread, but this precludes the user being able to interrupt the computation.

A nice feature of the python GIL is that one can be fairly confident about sharing data (a careful solution would require locks etc.)

Multiprocessing Solution

This same solution should work with multiprocessing, but this fails on my development platform (Mac OS X 10.5) with the following error:

The process has forked and you cannot use this CoreFoundation
functionality safely. You MUST exec().
Break on
__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Separate Processes

It seems that the most robust solution is to have the calculation and plotters run in completely separate processes. This has an added benefit:

  • User can plot remotely.

One issue that needs to be addressed here (and in the multiprocessing solution) is the copying of data. One common use-case is that the plotter may be slower than the computation. Thus, intermediate data may be discarded and should not be sent across the network.

Additional Approaches

It seems that one should be able to use IPython to do this, but I have not found a simple way to do this yet.

Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.