Issue #71 resolved

atexit handler results in exceptions from half-torn-down-modules

Zooko O'Whielacronx
created an issue

This problem is exercised by the trialcoverage unit tests:

http://tahoe-lafs.org/trac/trialcoverage

Get trialcoverage and run "python setup.py test" to run trialcoverage's own self-tests which (on my machine) result in ...

Oh darn it now I can't reproduce it. It was giving me exceptions from code that was being run by the atexit handler when the module had been partially torn down so that, for example, the name "sorted" in coverage/data.py was None. I worked-around it by editing my copy of coverage/data.py to say "if sorted is None:" then just return and don't attempt to write out line_data.

Oh look! One of the Tahoe-LAFS buildbots is demonstrating the problem right now:

http://tahoe-lafs.org/buildbot/builders/Kyle%20OpenBSD-4.6%20amd64/builds/210/steps/test-coverage/logs/stdio

That buildbot is currently using these versions of coverage.py and trialcoverage:

http://tahoe-lafs.org/buildbot/builders/Kyle%20OpenBSD-4.6%20amd64/builds/210/steps/show-tool-versions/logs/stdio

That build ends with this stack trace: {{{ Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs func(*targs, kargs) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/control.py", line 302, in save self.data.write(suffix=data_suffix) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 83, in write self.write_file(filename) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 111, in write_file data['lines'] = self.line_data() File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 96, in line_data [(f, sorted(lmap.keys())) for f, lmap in self.lines.items()] TypeError: 'NoneType' object is not callable Error in sys.exitfunc: Traceback (most recent call last): File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs func(*targs, kargs) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/control.py", line 302, in save self.data.write(suffix=data_suffix) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 83, in write self.write_file(filename) File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 111, in write_file data['lines'] = self.line_data() File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 96, in line_data [(f, sorted(lmap.keys())) for f, lmap in self.lines.items()] TypeError: 'NoneType' object is not callable }}}

coverage: [coverage 3.3.2a1z11 (/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg)]

trialcoverage: [trialcoverage 0.3.10 (/usr/local/lib/python2.6/site-packages/trialcoverage-0.3.10-py2.6.egg), setuptools 0.6c11 (/usr/local/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg), pyutil 1.7.5 (/usr/local/lib/python2.6/site-packages/pyutil-1.7.5-py2.6.egg), coverage 3.3.2a1z11 (/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg), simplejson 2.1.1 (/usr/local/lib/python2.6/site-packages/simplejson-2.1.1-py2.6-openbsd-4.6-amd64.egg), zbase32 1.1.1 (/usr/local/lib/python2.6/site-packages/zbase32-1.1.1-py2.6.egg), argparse 0.9.1 (/usr/local/lib/python2.6/site-packages/argparse-0.9.1-py2.6.egg)]

I'll bet the reason that I can't reproduce it is that the trialcoverage self-tests currently pass and that this problem is triggered only when the tests are exiting in the failure state, as they are in that buildbot example.

Comments (6)

  1. Zooko O'Whielacronx reporter

    Oh look the atexit hook causes these problems even on success tests in some cases:

    http://tahoe-lafs.org/buildbot/builders/Kyle%20OpenBSD-4.6%20amd64/builds/227/steps/test-coverage/logs/stdio

    Ends with:

    Ran 795 tests in 717.507s
    
    PASSED (skips=15, expectedFailures=1, successes=779)
    Error in atexit._run_exitfuncs:
    Traceback (most recent call last):
      File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
        func(*targs, **kargs)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/control.py", line 302, in save
        self.data.write(suffix=data_suffix)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 83, in write
        self.write_file(filename)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 111, in write_file
        data['lines'] = self.line_data()
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 96, in line_data
        [(f, sorted(lmap.keys())) for f, lmap in self.lines.items()]
    TypeError: 'NoneType' object is not callable
    Error in sys.exitfunc:
    Traceback (most recent call last):
      File "/usr/local/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
        func(*targs, **kargs)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/control.py", line 302, in save
        self.data.write(suffix=data_suffix)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 83, in write
        self.write_file(filename)
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 111, in write_file
        data['lines'] = self.line_data()
      File "/usr/local/lib/python2.6/site-packages/coverage-3.3.2a1z11-py2.6-openbsd-4.6-amd64.egg/coverage/data.py", line 96, in line_data
        [(f, sorted(lmap.keys())) for f, lmap in self.lines.items()]
    TypeError: 'NoneType' object is not callable
    

    I've already asked the owner of that buildslave to install trialcoverage-0.3.11 which has by aforementioned kludge to disable the atexit handlers.

  2. Zooko O'Whielacronx reporter

    This happened again on a different buildslave:

    http://tahoe-lafs.org/buildbot/builders/Zooko%20zomp%20Mac-amd64%2010.6%20py2.6/builds/0/steps/test-coverage/logs/stdio

    Unfortunately I need to make it stop doing this so that this won't interfere with the imminent release of Tahoe-LAFS v1.7.0β, so I'm going to upgrade that buildslave from trialcoverage-1.3.8-2 to trialcoverage-1.3.12, which will have the "disable atexit hook" work-around in it...

  3. Zooko O'Whielacronx reporter
  4. Log in to comment