execnet / CHANGELOG

Full commit

- group.remote_exec now supports kwargs as well

- support per channel string coercion configuration

- now reads correct amounts of bytes from nonblocking fd's

- added a ``dont_write_bytecode`` option to Popen gateways, this sets the
  ``sys.dont_write_bytecode`` flag on the spawned process, this only works on
  CPython 2.6 and higher.  Thanks to Alex Gaynor.

- added a pytest --broken-isp option to skip tests that assume
  DNS queries for unknown hosts actually are resolved as such (Thanks
  Alex Gaynor)

- fix issue 1 - decouple string coercion of channels and gateway

- fix issue #2 - propperly reconfigure the channels string coercion for rsync,
  so it can send from python2 to python3

- refactor socketserver, so it can be directly remote_exec'd for starting a socket gateway on a remote

- include recent version


- add gw.reconfigure() to configure per gateway options.  Currently supported:
  py2str_as_py3str and py3str_as_py2str to configure string deserialization

- channel.makefile() objects now have a isatty() returning False

- group.allocate_id(spec) allows to early-determine an (automatic) id

- internal refactorings and cleanups (thanks Ronny Pfannschmidt):
  - refactor message types into received handler functions
  - refactor b(chr(opcode)) to bchr(opcode)
  - reorder Message ctor args, rename msgtype to msgcode
  - refactor gateway.send to take message's init args instead of a message
  - inline and remove Message.writeto/readfrom
  - refactor collection loading to avoid the indirection over tuple
  - remove the unused NamedThreadPool


- new ``gateway.remote_exec(func, **kwargs)`` style fo executing
  a pure function with parameters.  The function on the remote
  side also needs to accept a ``channel`` which allows it to
  communicate back and forth.  Thanks to Ronny Pfannschmidt
  for implementing it with special kudos to Maciej Fijalkowski
  for writing a "pure-function" checker so that on Python2.6
  onwards non-pure functions will be rejected.

- enhance rsyncing to also sync permissions (stat().st_mode)
  of directories and files.
  (should also resolve

- fix rsyncing of symlinks, thanks to Charles Solar
  (should also resolve

- update internal usage of apipkg to 1.0b6

- remote_exec(module) now makes sure that the linecache is updated
  before reading and sending the source.  thanks Ronny, Matt.

- removed all trailing whitespace from source files


- try to avoid a random KeyboardInterrupt Error when threads
  are ending.

- extend xspec syntax to allow for one or multiple "env:NAME=value"
  environment variable settings which will be set on the remote side.
  (thanks Jakub Gustak)


- fix jython/windows interactions
- fix waitclose/callback-with-endmarker race condition
- fix race condition where multiple threads sending data over channels
  would crash the serializer and process


- more care during receiver-thread finalization during interp-shutdown,
  should get rid of annoying and meaningless exceptions
- fix glitch in ssh-fileserver example
- experimentally add " test" support - will run py.test


- try to deal more cleanly with interpreter shutdown setting globals to
  None. this avoids (hopefully) some bogus tracebacks at process exit.


- refine termination some more: CTRL-C and gateway.exit will
  now try harder to interrupt remote execution.  this
  helps to avoid left-over ssh-processes.
- fix read-on-non-blocking-files issue probably related to jython only:
  the low-level read on subprocess pipes may be non-blocking, returning
  less bytes than requested - so we now loop.
- Windows/python2.4: fix bug that killing subprocesses would fail
- make RemoteError and TimeoutError available directly on execnet namespace

- fix some doc and test issues (thanks thm and ronny), add ssh_fileserver example
- update internal copy of apipkg
- always skip remote tests if no ssh specs given


- generalize channel-over-channel sending: you can now have channels
  anywhere in a data structure (i.e. as an item of a container type).
  Add according examples.

- automatically close a channel when a remote callback raises
  an exception, makes communication more robust because until
  now failing callbacks rendered the receiverthread unuseable
  leaving the remote side in-accessible.

- internally split socket gateways, speeds up popen-gateways
  by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine)

- fix bug in channel.receive() that would wrongly raise a TimeoutError
  after 1000 seconds (thanks Ronny Pfannschmidt)


- revamp and better structure documentation

- new method: gateway.hasreceiver() returns True
  if the gateway is still receive-active. remote_status
  now only carries information about remote execution status.

- new: execnet.MultiChannel provides basic iteration/contain interface

- new: execnet.Group can be indexed by integer

- new: group.makegateway() uses group.default_spec if no spec is given
  and the execnet.default_group uses ``popen`` as a default spec.

- have popen-gateways use imports instead of source-strings,
  also improves debugging/tracebacks, as a side effect
  popen-gateway startup can be substantially faster (>30%)

- refine internal gateway exit/termination procedure
  and introduce group.terminate(timeout) which will
  attempt to kill all subprocesses that did not terminate
  within time.

- EOFError on channel.receive/waitclose if the other
  side unexpectedly went away.  When a gateway exits
  it now internally sends an explicit termination message
  instead of abruptly closing.

- introduce a timeout parameter to channel.receive()
  and default to periodically internally wake up
  to let KeyboardInterrupts pass through.

- EXECNET_DEBUG=2 will cause tracing to go to stderr,
  which with popen slave gateways will relay back
  tracing to the instantiator process.


* introduce execnet.Group for managing gateway creation
  and termination.  Introduce execnet.default_group through which
  all "global" calls are routed.  cleanup gateway termination.
  All Gateways get an id through which they can be
  retrieved from a group object.

* deprecate execnet.XYZGateway in favour of direct makegateway() calls.

* refine socketserver-examples, experimentally introduce a
  way to indirectly setup a socket server ("installvia")
  through a gateway url.

* refine and automatically test documentation examples


* fix EXECNET_DEBUG to work with win32
* add support for serializing longs, sets and frozensets  (thanks
  Benjamin Peterson)
* introduce remote_status() method which on the low level gives
  information about the remote side of a gateway
* disallow explicit close in remote_exec situation
* perform some more detailed tracing with EXECNET_DEBUG


* make internal protocols more robust against serialization failures

* fix a seralization bug with nested tuples containing empty tuples
  (thanks to ronny for discovering it)

* setting the environment variable EXECNET_DEBUG will generate per
  process trace-files for debugging


* added new examples for NumPy, Jython, IronPython
* improved documentation
* include for lazy-importing
* integrated new serializer code from Benjamin Peterson
* improved support for Jython-2.5.1


* improve documentation, new website

* use sphinx for documentation, added boilerplate files and

* fixes for standalone usage, adding boilerplate files

* imported py/execnet and made it work standalone