Incorrect coverage report with cover-branches under python 2.7

Issue #502 resolved
Артем Дайнеко created an issue

coverage report shows non-zero coverage on completely untested files (tested under 2.7.11 and 3.5.1 on Mac OS X El Capitan - only Python 2 is affected)

Minimal example of code you can see in repository: https://bitbucket.org/jamert/coverage-branch-report-issue

As preview: code like this

class Klass(object):
    def method(self):
        while True:
            return 1

while completely untested, shows non-zero coverage with the following configuration:

$ cat .coveragerc
[run]
source = src
branch = True

(I would expect zero percent coverage on it).

Steps for reproducing:

$ coverage run src/tested.py  # empty file
$ coverage report
Name            Stmts   Miss Branch BrPart  Cover
-------------------------------------------------
src/code.py         4      4      2      0    33%
src/tested.py       0      0      0      0   100%
-------------------------------------------------
TOTAL               4      4      2      0    33%

Under 3.5.1 is shows different numbers:

$ coverage report
Name            Stmts   Miss Branch BrPart  Cover
-------------------------------------------------
src/code.py         4      4      0      0     0%
src/tested.py       0      0      0      0   100%
-------------------------------------------------
TOTAL               4      4      0      0     0%

Comments (11)

  1. Ned Batchelder repo owner
    • changed status to open

    Hmm, odd. Thanks for the report. BTW, if you change "while True:" to "while x:", it correctly shows 0%, so there's something going on with the constant-branch detection.

  2. Артем Дайнеко reporter

    Yes, I forgot to mention this. Also, if you make it regular function, and not method then it shows 0%, too.

  3. Ned Batchelder repo owner

    When I make it a function like this:

    def function():
        while True:
            return 1
    

    I get 40% as the coverage number.

  4. Артем Дайнеко reporter

    So do I. It seems I tested something wrong when tried to reduce example)

  5. Loic Dachary

    The while True is statically marked as covered because it only has one branch (the loop will never end). When computing statistics, it is excluded from the list of missing arcs regardless of the fact that the code leading to this arc was run or not.

    The problem does not exist in python3 because coverage.py find while True using AST instead of regexps on the sources and avoid creating arcs that cannot be reached.

    $ COVERAGE_ASTDUMP=1 COVERAGE_TRACK_ARCS=1 coverage report -m
    Statements: set([1, 2, 3, 4])
    Multiline map: {}
    <Module
        body: [
            <ClassDef @ 1
                name: 'Klass'
                bases: [
                    <Name @ 1 id: 'object'>
                ]
                body: [
                    <FunctionDef @ 2
                        name: 'method'
                        args:
                            <arguments
                                args: [
                                    <Name @ 2 id: 'self'>
                                ]
                                vararg: None
                                kwarg: None
                                defaults: []
                            >
                        body: [
                            <While @ 3
                                test:
                                    <Name @ 3 id: 'True'>
                                body: [
                                    <Return @ 4
                                        value:
                                            <Num @ 4 n: 1>
                                    >
                                ]
                                orelse: []
                            >
                        ]
                        decorator_list: []
                    >
                ]
                decorator_list: []
            >
        ]
    >
    
    Adding arc: (-1, 1): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
              _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @931
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (1, -1): None, "didn't exit the module"
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
              _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @933
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (-1, 1): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
            _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @950
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (1, 2): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
            _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @951
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (2, -1): None, "didn't exit the body of class 'Klass'"
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
            _code_object__ClassDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @955
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (-2, 3): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
         _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @942
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (3, 4): 'the condition on line {lineno} was never true', None
         _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @942
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @617
                          add_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @588
                    _handle__While : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @904
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @616
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (4, -2): "the return on line {lineno} wasn't executed", "didn't return from function 'method'"
                    _handle__While : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @904
                     add_body_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @617
                          add_arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @588
                   _handle__Return : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @782
              process_return_exits : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @691
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (3, -2): 'the condition on line {lineno} was never false', "didn't return from function 'method'"
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
         _code_object__FunctionDef : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @943
              process_return_exits : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @691
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    Statements: set([])
    Multiline map: {}
    <Module body: []>
    
    Adding arc: (-1, 1): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
              _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @936
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    
    Adding arc: (1, -1): None, None
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/python.py @171
                              arcs : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @264
                      _analyze_ast : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @274
                           analyze : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @515
              _code_object__Module : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @937
                           add_arc : /home/loic/software/coveragepy/issue-502/coverage.py/coverage/parser.py @521
    Name            Stmts   Miss Branch BrPart  Cover   Missing
    -----------------------------------------------------------
    src/code.py         4      4      2      0    33%   1-4
    src/tested.py       0      0      0      0   100%
    -----------------------------------------------------------
    TOTAL               4      4      2      0    33%
    
  6. Ned Batchelder repo owner

    also use AST for while constants in python-2.7 #502

    The node.id is set to False, True or None is python-2.7: there is no reason to only check for it with python-3. It is more reliable than using the DEFAULT_PARTIAL_ALWAYS regexps on source lines.

    close #502

    → <<cset d9cc167007ef>>

  7. Log in to comment