Issue #9 resolved

Threading exception on Python 3.2

Walter Dörwald
created an issue

I am using the following test script:

{{{

!python

import datetime import execnet

i = 0 while True: print(datetime.datetime.now(), i) group = execnet.Group() gw = group.makegateway("ssh=user@host") ch = gw.remote_exec("import time ; time.sleep(2.0)") group.terminate(timeout=1.0) i += 1 }}}

After about 250 runs of the loop body I get the following output:

{{{ 2012-09-03 18:04:10.822562 253 2012-09-03 18:04:11.205508 254 Traceback (most recent call last): File "/var/home/walter/.local/lib/python3.2/site-packages/execnet-1.1-py3.2.egg/execnet/threadpool.py", line 130, in dispatch thread, _ = self._ready.popitem() KeyError: 'popitem(): dictionary is empty'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "gurk.py", line 10, in <module> group.terminate(timeout=1.0) File "/var/home/walter/.local/lib/python3.2/site-packages/execnet-1.1-py3.2.egg/execnet/multi.py", line 168, in terminate for gw in self._gateways_to_join]) File "/var/home/walter/.local/lib/python3.2/site-packages/execnet-1.1-py3.2.egg/execnet/multi.py", line 251, in safe_terminate reply = workerpool.dispatch(termkill, termfunc, killfunc) File "/var/home/walter/.local/lib/python3.2/site-packages/execnet-1.1-py3.2.egg/execnet/threadpool.py", line 135, in dispatch thread = self._newthread() File "/var/home/walter/.local/lib/python3.2/site-packages/execnet-1.1-py3.2.egg/execnet/threadpool.py", line 141, in _newthread thread.start() File "/var/home/walter/.local/lib/python3.2/threading.py", line 683, in start _start_new_thread(self._bootstrap, ()) _thread.error: can't start new thread }}}

Comments (5)

  1. Ronny Pfannschmidt

    indeed, i checked the script a bit and we leak threads `threading.active_count()` increases

    the updated script is

    from __future__ import print_function
    import datetime
    import execnet
    import threading
    import gc
    i = 0
    while True:
        print(datetime.datetime.now(), i, threading.active_count())
        group = execnet.Group()
        gw = group.makegateway("popen")
        ch = gw.remote_exec("import time ; time.sleep(2.0)")
        group.terminate(timeout=1.0)
        gc.collect()
        i += 1
    
  2. Thomas Moschny

    As already discussed on #pylib, applying this patch to execnet 1.1 causes testing/test_channel.py hang forever for me (platform linux -- Python 3.3.0 -- pytest-2.3.4).

  3. Log in to comment