Confusing HTML report for certain partial branch coverage results

Issue #460 resolved
Alex Gaynor created an issue
(tempenv-5787194743781) ~/.v/tempenv-5787194743781 $ cat t.py
d = {
    1: lambda: [],
    2: lambda: [],
    3: lambda: [],
    4: lambda: [],
}

for k, v in d.iteritems():
    if k & 1:
        v()
(tempenv-5787194743781) ~/.v/tempenv-5787194743781 $ coverage run --branch t.py
(tempenv-5787194743781) ~/.v/tempenv-5787194743781 $ coverage report -m
Name    Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------
t.py        4      0      9      2    85%   1->-5, 1->-3

The coverage report -m mostly makes sense, lines 3 and 5 have partial coverage (though it's not really a branch, nor is line 1 involved). However the HTML report doesn't include this level of detail:

Screen Shot 2016-01-02 at 12.59.05 PM.png

It'd be great if the HTML coverage made clear what was going on.

Comments (14)

  1. Ned Batchelder repo owner

    This is a manifestation of bug #90. I see what you mean about the text report having more information than the HTML report.

  2. Alex Gaynor reporter

    Indeed, I guess there's two bugs here:

    • The ambiguous vocabulary around lambda expressions (which #90 covers)
    • The lost context in the HTML representation for them.
  3. Ned Batchelder repo owner

    If we don't have the option to annotate any line but line 1, what annotations on line 1 would be most useful?

  4. Alex Gaynor reporter

    I'm not sure I precisely understand the question, but anything that gives me an indication of which specific lambdas weren't executed would be a huge step up.

  5. Ned Batchelder repo owner

    OK, let me try to explain: in the coverage data, we have arcs, which are (from, to) pairs indicating that execution jumped from one line to another line. If the to line is negative, it means, exited the code object starting at -to. So, (20, -17) means that execution from line 20 exited the code object started on line 17.

    In this case, the -5 in the text report means, "exited the code object starting on line 5." The code object in this case is a lambda, but coverage.py doesn't record what kind of code object it was.

    The difficulty with the measurement here is that lines 1 through 5 are all credited to line 1 (because it's only one statement, on line 1), but we know that the code objects actually are on the other lines. So I could say something like, "Line 1 never exited the code object started on line 3". With more work, it could say "the lambda on line 3".

    BTW, the negative numbers in the text report are the subject of #469, so that is also up for change.

  6. Ned Batchelder repo owner

    There's a title attribute on the yellow annotation, "Line 1 was executed but never jumped to the function exit." I can definitely improve that text, and make it more apparent, maybe by making the "[?]" clickable?

  7. Ned Batchelder repo owner

    As of 866842ba65b9, the long message will be, "line 3 didn't execute the lambda on line 3". It's still a highlight on line 1, and the short message still says, "exit", but I'll see what I can do about that next.

  8. Ned Batchelder repo owner

    Now the long message is "line 3 didn't finish the lambda on line 3." I'll mark this resolved now, you let me know if it needs more work.

  9. Log in to comment