Issue #310 open

Coverage fails with os.fork and os._exit

Marc Schlaich
created an issue

It is a common pattern to exit a child process with os._exit after a fork (ref). However, in this case coverage for the child process fails.

Example:

import os

def child():
    a = 1 + 1
    assert a == 2


def main():
    pid = os.fork()
    if pid:
        _, systemstatus = os.waitpid(pid, 0)
        assert not systemstatus
    else:
        child()
        os._exit(0)


if __name__ == '__main__':
    main()

Test run:

$ coverage run -p cov.py
$ coverage combine && coverage report -m
Name    Stmts   Miss  Cover   Missing
-------------------------------------
cov        13      4    69%   6-7, 16-17

Comments (15)

  1. Ned Batchelder repo owner

    Your test run data is incorrect, it actually is:

    Name     Stmts   Miss  Cover   Missing
    --------------------------------------
    bug310      13      4    69%   4-5, 14-15
    

    I'm not sure what I can do to fix this. os._exit skips any further work in the process, including the work coverage.py needs to do to write out its measured data.

  2. Marc Schlaich reporter

    Your test run data is incorrect, it actually is:

    Bitbucket has stripped some new lines ;)

    I'm not sure what I can do to fix this. os._exit skips any further work in the process

    Yes, I feared this cannot be handled automatically. Anyway, it would be enough if I can do the clean up manually before os._exit but AFAIS there is no useful entry point in the API. Something like coverage.shutdown would be nice.

  3. Ryan Stuart

    Here is a dirty hack to work around this issue:

        def _coverage_hack(cls):
            """God awful hack to make coverage and py.test work together."""
            class Cov(object):
                stop = lambda self: None
                save = lambda self: None
            cov = Cov()
            if "coverage" in sys.modules:
                import coverage
                try:
                    raise ZeroDivisionError
                except ZeroDivisionError:
                    f = sys.exc_info()[2].tb_frame
                tb = []
                while f:
                    tb.append(f)
                    f = f.f_back
                t = tb[-3]
                if 'self' in t.f_locals:
                    slf = t.f_locals['self']
                    if hasattr(slf, "coverage"):
                        if isinstance(slf.coverage, coverage.coverage):
                            cov = slf.coverage
            return cov
        _coverage_hack = classmethod(_coverage_hack)
    

    It can be used as follows:

    self.__coverage = self._coverage_hack()
    child_pid = os.fork()
    if child_pid == 0:
        # Do work
        ((self.__coverage.stop(),) and (self.__coverage.save(),) and os._exit(0))
    
  4. Ned Batchelder repo owner

    Ryan Stuart If I read this code right, it's a way to get access to the coverage instance (if any), and then calling it explicitly from the product code. If the product code can be changed to accommodate this problem, then there's lots of possibilities, but people generally don't want to do that.

  5. Ned Batchelder repo owner

    Marc Schlaich The method you want is currently called coverage._atexit. I could rename that to coverage.shutdown. Try using coverage._atexit() now and tell me if it works well. Or you could use coverage.stop(); coverage.save().

  6. Ryan Stuart

    Marc Schlaich Just to be clear, the problem of it not capturing lines seems to be fixed but it certainly doesn't capture the lines in the subprocesses no matter if I exit with os._exit(), sys.exit() or my hack above.

  7. Log in to comment