1. Ned Batchelder
  2. coverage.py
  3. Issues
Issue #186 wontfix

test_for_leaks failures with PyPy 1.8 when running test.test_oddball by itself

Marc Abramowitz
created an issue

This is from an Ubuntu 11 vm:

{{{ marca@ubuntu:~/dev/git-repos/coveragepy$ while true; do PATH=$PATH:~/src/pypy-1.8/bin .tox/pypy_py/bin/nosetests test.test_oddball; sleep 5; done ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1073152


Ran 11 tests in 10.175s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1069056


Ran 11 tests in 9.156s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 8.884s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 532480


Ran 11 tests in 9.005s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 8.638s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 11.813s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 11.329s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 798720


Ran 11 tests in 8.788s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1069056


Ran 11 tests in 12.260s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1073152


Ran 11 tests in 14.271s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1339392


Ran 11 tests in 8.791s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 8.391s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 1073152


Ran 11 tests in 8.842s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 802816


Ran 11 tests in 13.602s

FAILED (failures=1) ....F...... ====================================================================== FAIL: test_for_leaks (test.test_oddball.MemoryLeakTest)


Traceback (most recent call last): File "/home/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 161, in test_for_leaks self.assertTrue(ram_growth < 100000, "RAM grew by %d" % (ram_growth)) AssertionError: False is not true : RAM grew by 798720


Ran 11 tests in 8.870s

FAILED (failures=1) }}}

Comments (4)

  1. Ned Batchelder repo owner

    Hmm, looks like running the file by itself is a bad idea. Testing for memory leaks is kind of a crazy thing for me to attempt anyway. PyPy has a very different memory footprint than other implementations.

    I've added a note of warning in d12daaa05f8c , and am marking this as wontfix.

  2. Marc Abramowitz reporter

    Should test_oddball or test_oddball.ThreadingTest be skipped entirely on PyPy?

    [last: 0] marca@SCML-MarcA:~/dev/git-repos/coveragepy$ tox -e pypy_py
    GLOB sdist-make: /Users/marca/dev/git-repos/coveragepy/setup.py
    pypy_py create: /Users/marca/dev/git-repos/coveragepy/.tox/pypy_py
    pypy_py installdeps: nose
    pypy_py sdist-inst: /Users/marca/dev/git-repos/coveragepy/.tox/dist/coverage-3.5.3b1.zip
    pypy_py runtests: commands[0]
    pypy_py runtests: commands[1]
    ........................................................................................................................................................................................................................................FF.................................................................
    ======================================================================
    FAIL: test_thread_run (test.test_oddball.ThreadingTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 50, in test_thread_run
        [1,3,4,5,6,7,9,10,12,13,14], "")
      File "/Users/marca/dev/git-repos/coveragepy/test/coveragetest.py", line 328, in check_coverage
        self.assertEqual(analysis.missing_formatted(), missing)
    AssertionError: '5-7, 10' != ''
    
    ======================================================================
    FAIL: test_threading (test.test_oddball.ThreadingTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/marca/dev/git-repos/coveragepy/test/test_oddball.py", line 31, in test_threading
        [1,3,4,6,7,9,10,12,13,14,15], "10")
      File "/Users/marca/dev/git-repos/coveragepy/test/coveragetest.py", line 328, in check_coverage
        self.assertEqual(analysis.missing_formatted(), missing)
    AssertionError: '7, 10' != '10'
    
    ----------------------------------------------------------------------
    Ran 299 tests in 41.348s
    
    FAILED (failures=2)
    ERROR: InvocationError: '/Users/marca/dev/git-repos/coveragepy/.tox/pypy_py/bin/nosetests -w /Users/marca/dev/git-repos/coveragepy -e test_farm'
    _________________________________________________________________________ summary __________________________________________________________________________
    ERROR:   pypy_py: commands failed
    

    Something along the lines of:

    diff --git a/test/test_oddball.py b/test/test_oddball.py
    index 859648f..2be813e 100644
    --- a/test/test_oddball.py
    +++ b/test/test_oddball.py
    @@ -7,47 +7,48 @@ sys.path.insert(0, os.path.split(__file__)[0]) # Force relative import for Py3k
     from coveragetest import CoverageTest
     import osinfo
     
    -class ThreadingTest(CoverageTest):
    -    """Tests of the threading support."""
    -
    -    def test_threading(self):
    -        self.check_coverage("""\
    -            import threading
    -
    -            def fromMainThread():
    -                return "called from main thread"
    -
    -            def fromOtherThread():
    -                return "called from other thread"
    -
    -            def neverCalled():
    -                return "no one calls me"
    -
    -            other = threading.Thread(target=fromOtherThread)
    -            other.start()
    -            fromMainThread()
    -            other.join()
    -            """,
    -            [1,3,4,6,7,9,10,12,13,14,15], "10")
    -
    -    def test_thread_run(self):
    -        self.check_coverage("""\
    -            import threading
    -
    -            class TestThread(threading.Thread):
    -                def run(self):
    -                    self.a = 5
    -                    self.do_work()
    -                    self.a = 7
    -
    -                def do_work(self):
    -                    self.a = 10
    -
    -            thd = TestThread()
    -            thd.start()
    -            thd.join()
    -            """,
    -            [1,3,4,5,6,7,9,10,12,13,14], "")
    +if not hasattr(sys, 'pypy_version_info'):
    +    class ThreadingTest(CoverageTest):
    +        """Tests of the threading support."""
    +    
    +        def test_threading(self):
    +            self.check_coverage("""\
    +                import threading
    +    
    +                def fromMainThread():
    +                    return "called from main thread"
    +    
    +                def fromOtherThread():
    +                    return "called from other thread"
    +    
    +                def neverCalled():
    +                    return "no one calls me"
    +    
    +                other = threading.Thread(target=fromOtherThread)
    +                other.start()
    +                fromMainThread()
    +                other.join()
    +                """,
    +                [1,3,4,6,7,9,10,12,13,14,15], "10")
    +    
    +        def test_thread_run(self):
    +            self.check_coverage("""\
    +                import threading
    +    
    +                class TestThread(threading.Thread):
    +                    def run(self):
    +                        self.a = 5
    +                        self.do_work()
    +                        self.a = 7
    +    
    +                    def do_work(self):
    +                        self.a = 10
    +    
    +                thd = TestThread()
    +                thd.start()
    +                thd.join()
    +                """,
    +                [1,3,4,5,6,7,9,10,12,13,14], "")
     
     
     class RecursionTest(CoverageTest):
    

    which yields:

    [last: 0] marca@SCML-MarcA:~/dev/git-repos/coveragepy$ .tox/pypy_py/bin/nosetests test.test_oddball -v
    test_doctest (test.test_oddball.DoctestTest) ... ok
    test_exception (test.test_oddball.ExceptionTest) ... ok
    test_multi_layers (test.test_oddball.GettraceTest) ... ok
    test_round_trip (test.test_oddball.GettraceTest) ... ok
    test_for_leaks (test.test_oddball.MemoryLeakTest) ... ok
    test_pyexpat (test.test_oddball.PyexpatTest) ... ok
    test_long_recursion (test.test_oddball.RecursionTest) ... ok
    test_long_recursion_recovery (test.test_oddball.RecursionTest) ... ok
    test_short_recursion (test.test_oddball.RecursionTest) ... ok
    
    ----------------------------------------------------------------------
    Ran 9 tests in 1.118s
    
    OK
    [last: 2] marca@SCML-MarcA:~/dev/git-repos/coveragepy$ .tox/py27_py/bin/nosetests test.test_oddball -v
    test_doctest (test.test_oddball.DoctestTest) ... ok
    test_exception (test.test_oddball.ExceptionTest) ... ok
    test_multi_layers (test.test_oddball.GettraceTest) ... ok
    test_round_trip (test.test_oddball.GettraceTest) ... ok
    test_for_leaks (test.test_oddball.MemoryLeakTest) ... ok
    test_pyexpat (test.test_oddball.PyexpatTest) ... ok
    test_long_recursion (test.test_oddball.RecursionTest) ... ok
    test_long_recursion_recovery (test.test_oddball.RecursionTest) ... ok
    test_short_recursion (test.test_oddball.RecursionTest) ... ok
    test_thread_run (test.test_oddball.ThreadingTest) ... ok
    test_threading (test.test_oddball.ThreadingTest) ... ok
    
    ----------------------------------------------------------------------
    Ran 11 tests in 0.116s
    
    OK
    

    ?

  3. Ned Batchelder repo owner

    I'm talking to PyPy people about why the threading test fails. We can skip it in PyPy as a last resort, but there's no reason this test should fail on PyPy, so I'd like to understand more before taking that drastic course of action.

  4. Log in to comment