Lines incorrectly reporting as not covered when running with --branch

Issue #605 new
David MacIver
created an issue

I ran into this problem when testing Hypothesis: Lines that are definitely covered by a test are showing up as uncovered in coverage.

The following is the test file:

# coding=utf-8

from __future__ import division, print_function, absolute_import

import hypothesis.strategies as st


def test_very_deep_deferral():
    def strat(i):
        if i == 0:
            return st.deferred(lambda: st.one_of(strategies + [st.none()]))
        else:
            return st.deferred(
                lambda: st.tuples(strategies[(i + 1) % len(strategies)]))

    strategies = list(map(strat, range(100)))

    assert strategies[0].has_reusable_values
    assert not strategies[0].is_empty


if __name__ == '__main__':
    test_very_deep_deferral()

When testing Hypothesis commit 8844ca758737a6439a82507e7910972c6fa22358, this shows lines 156-158 in src/hypothesis/searchstrategy/strategies.py as uncovered. However the lines are definitely covered by this test (see below for evidence).

To reproduce this problem:

git clone git@github.com:HypothesisWorks/hypothesis-python.git
cd hypothesis-python
git checkout 8844ca758737a6439a82507e7910972c6fa22358

(now copy the above example as test_local.py)

PYTHONPATH=src python -m coverage run --branch --rcfile=/dev/null --include=src/hypothesis/searchstrategy/strategies.py test_local.py
coverage report --rcfile=/dev/null --show-missing

Salient features:

  • The lines 156-158 show up in the report.
  • If you rerun it without "--branch" in the "coverage run" line they do not.
  • If you add "assert False" at line 156, the test fails. Thus the coverage run without "--branch" is correct.

Comments (1)

  1. Ned Batchelder repo owner

    Thanks for the detailed instructions. I had to install a bunch of requirements (including enum34 manually, which isn't mentioned in any requirements file?). But then I did not see the results you see:

    $ PYTHONPATH=src python -m coverage run --branch --rcfile=/dev/null --include=src/hypothesis/searchstrategy/strategies.py test_local.py
    $ coverage report --rcfile=/dev/null --show-missing
    Name                                          Stmts   Miss Branch BrPart  Cover   Missing
    -----------------------------------------------------------------------------------------
    src/hypothesis/searchstrategy/strategies.py     244    116     68      3    46%   37-40, 143, 170-173, 212, 225-267, 278, 290-291, 304, 311, 320-322, 330-339, 342, 345, 370-371, 384-402, 405-414, 417, 420-421, 425-432, 445-448, 451, 454-459, 462, 467, 471-478, 482, 491-493, 496, 499-504, 507, 510-526, 530-534, 133->143, 369->370, 377->exit
    $ PYTHONPATH=src python -m coverage run --rcfile=/dev/null --include=src/hypothesis/searchstrategy/strategies.py test_local.py
    $ coverage report --rcfile=/dev/null --show-missing
    Name                                          Stmts   Miss  Cover   Missing
    ---------------------------------------------------------------------------
    src/hypothesis/searchstrategy/strategies.py     244    116    52%   37-40, 143, 170-173, 212, 225-267, 278, 290-291, 304, 311, 320-322, 330-339, 342, 345, 370-371, 384-402, 405-414, 417, 420-421, 425-432, 445-448, 451, 454-459, 462, 467, 471-478, 482, 491-493, 496, 499-504, 507, 510-526, 530-534
    
  2. Log in to comment