python 2.7 inconsistent behavior upon SIGPIPE when using coverage

Issue #568 wontfix
vaab created an issue

All version tested of python (2.7, 3.4, 3.5, 3.6) and platforms (linux, windows) I tested show same behavior when confronted to unhandled SIGPIPE : some pythonic errors on stderr.

But this is not anymore the case if you go through coverage (using python 2.7 only, on windows or linux) as I intended to actually trigger that behavior in unittest. I nailed down the issue to the "isolate_module(..)" function in this example::

cat <<EOF > /tmp/test.py
from __future__ import print_function

import os
import types
import sys


ISOLATED_MODULES = {}

def isolate_module(mod):
    """Copy a module so that we are isolated from aggressive mocking.

    If a test suite mocks os.path.exists (for example), and then we need to use
    it during the test, everything will get tangled up if we use their mock.
    Making a copy of the module when we import it will isolate coverage.py from
    those complications.
    """
    if mod not in ISOLATED_MODULES:
        new_mod = types.ModuleType(mod.__name__)
        ISOLATED_MODULES[mod] = new_mod
        for name in dir(mod):
            value = getattr(mod, name)
            if isinstance(value, types.ModuleType):
                value = isolate_module(value)
            setattr(new_mod, name, value)
    return ISOLATED_MODULES[mod]


os = isolate_module(os)

print("output on stdout")
print("output on stderr", file=sys.stderr)
EOF
python /tmp/test.py | :

The output I'm expecting is:

output on stderr
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

But I only get:

output on stderr

If you comment the line "os = isolate_module(os)", you get the expected output back.

I was interested which attribute could trigger this problem and nailed down the issue to sys.__stdout__ and sys.stdout.

I guess that as there's still a ref to sys.__stdout__, python doesn't run its destructor at the same moment.

Please note that there are no problem on other version of python.

I suggest to do something in this taste::

def isolate_module(mod):
    """Copy a module so that we are isolated from aggressive mocking.

    If a test suite mocks os.path.exists (for example), and then we need to use                                                  
    it during the test, everything will get tangled up if we use their mock.                                                     
    Making a copy of the module when we import it will isolate coverage.py from                                                  
    those complications.                                                                                                         
    """
    if mod not in ISOLATED_MODULES:
        new_mod = types.ModuleType(mod.__name__)
        ISOLATED_MODULES[mod] = new_mod
        attributes = dir(mod)
        if mod.__name__ == "sys":
            attributes = set(attributes) - set(["__stdout__", "__stdin__",
                                               "stdout", "stdin"])
        for name in attributes:
            value = getattr(mod, name)
            if isinstance(value, types.ModuleType):
                value = isolate_module(value)
            setattr(new_mod, name, value)
    return ISOLATED_MODULES[mod]

It could be also limited to Python 2 probably. I couldn't figure how to quickly run all tox tests without having dozens of failing tests everywhere even with no modification.

Comments (2)

  1. Ned Batchelder repo owner

    Thanks for a chewy and detailed bug report! :)

    To be honest, I'm not sure isolate_module was a good idea in the first place. I am definitely not going to add to its complexity. There's a limit to how much coverage.py can do its magic, and yet remain absolutely invisible.

  2. Log in to comment