1. Ned Batchelder
  2. coverage.py
  3. Issues
Issue #62 resolved

Something wrong about branch coverage of class statements

Jean-Paul Calderone
created an issue

Consider the attached module. Using "coverage run --branch" on it and then "coverage html" produces a coverage report which indicates two partially covered lines.

I would expect that all three methods in the example would be treated the same (ie, either that all three would be marked partially covered, or all three would be marked completely covered).

Comments (9)

  1. Ned Batchelder repo owner

    Branch coverage gets a bit tricky. I can explain why the coverage report looks the way it does. When analyzing code paths, coverage.py builds a list of "arcs": pairs of line numbers that are possible transitions during execution.

    In your code:

    class X:
    	def foo(self):
    		"hello"
    
    	def bar(self):
    		"world"
    
    	def baz(self):
    		"..."
    

    these are the arcs:

    [(-1, 1), (-1, 2), (-1, 5), (-1, 8), (1, -1), (1, 2), (2, -1), (2, 5), (5, -1), (5, 8), (8, -1)]

    In these arcs, a -1 is used specially: as the first element in the pair, it means execution can enter at the line number given as the second element. If a -1 is the second element in the pair, it means execution can leave a context at the line number given as the first element.

    So for example, (-1, 5) means that when executing the bar function, execution enters at line 5, and the (5, -1) arc means that execution also leaves at line 5.

    When actually executing the code, the lines executed are 1, then 2, then 5, then 8, so the arcs actually encountered during execution are (-1, 1), (1, -1), (1, 2), (2, 5), (5, 8), (8, -1). The possible arcs not encountered were (-1, 2), (-1, 5), (-1, 8), (2, -1), (5, -1). Lines missing branches are those named in the first elements of the missing arcs, so there are missing branches on 2 and 5.

    Look at the possible arcs for line 2: (2, -1) and (2, 5). The first is what would happen if you executed foo(). The second is what happens when you define foo. The problem is that for line 8, both execution and definition result in an arc of (8, -1). Execution runs from line 8 to the exit of the function, definition runs from line 8 to the exit of the class. There are really two possibilities, but they are both represented by -1, so the set of arcs doesn't include all the information it could.

    When the program is run, we get an arc of (8, -1), so there are no arc possibilities for line 8 that are missing from the execution data, so 8 is not shown as missing a branch.

    Now you can help me: is this just a hypothetical example, or do you really have code like this? Function defined with only docstrings are unusual.

  2. Ned Batchelder repo owner

    Thanks for the real code example. I can see a way forward to make all the methods be marked as missing a branch, but I'm not sure that's what you want.

    I would guess that you don't want coverage to draw your attention to these classes at all, since you know they are not executed, and are not going to be adding tests that will execute them.

    You can keep coverage from mentioning them by excluding these classes from the coverage measurement. The best way would be to create a coverage.ini file, and use a regex to exclude any class derived from Interface:

    # coverage.ini to control how coverage.py works
    [report]
    exclude_lines =
        # pragma: no cover
        class \w+\(Interface\):
        raise AssertionError
    

    (I included "pragma: no cover" since I assume you still want that, and "raise AssertionError" since those statements, if they exist, are never meant to be executed, though you may have tests that do exercise those lines.)

    If you use this coverage.ini file, then your interface classes will never be highlighted as missing execution.

  3. Tom Prince

    Excluding class \w+\(Interface\): will catch many but not all of the instances of this, since we have quite a few case of one interface inheriting from another (instead of from Interface directly.

    Having looked at the coverage code, I don't see any obvious way to catch these instances, though.

    Is there perhaps some way we could find them by inspecting the modules, and then telling coverage to ignore them?

  4. Tres Seaver

    I just stumbled across this myself today, while trying to assure that 'zope.interface' itself has 100% branch coverage. E.g.:

    $ git clone https://github.com/zopefoundation/zope.interface.git
    $ tox -e coverage # doesn't do branches
    $ .tox/coverage/bin/nosetests --with-coverage --cover-branches
    

    All the "branches" that get marked are empty "methods" of interfaces (defined using the 'class' statement, but with a different metaclass than 'type'): their suites consist purely of docstrings.

    (Using 'coverage-4.0a1-py27').

  5. Tristan Seligmann

    I'm still seeing this issue; the interface methods are marked "partially covered" since the body is never executed (and essentially can never be executed, there's no way to call it).

  6. Log in to comment