Issue #35 resolved

Try/except reports incomplete coverage when both paths are taken

Gary Bernhardt
created an issue

The following file reports that only one of the two branches were followed. Since I'm calling it both ways, I'd expect it to report both! :)

{{{

!python

def foo(a_list): try: a_list[0] except IndexError: pass

foo([]) foo([0]) }}}

Comments (14)

  1. Gary Bernhardt reporter

    I started trying to figure this out, but became quite confused when the following test passed without making changes to coverage:

        def test_try_except_when_both_are_executed(self):
            self.check_coverage("""\
                a = []
                def foo(a_list):
                    try:
                        a_list[0]
                        a.append('first')
                    except IndexError:
                        a.append('second')
                foo([0])
                foo([])
                assert a == ['first', 'second']
                """,
                arcz_missing="")
    

    It seems like the transitions are being recorded at a lower level, but the high-level reporter doesn't report that.

  2. Gary Bernhardt reporter

    That explanation makes sense, although I still don't understand why my test passed. It seems like the third invisible branch should've been in arcz_missing. :)

  3. Ned Batchelder repo owner

    Gary, I admire your fortitude attempting to add that test! The check_coverage method won't even attempt branch measurement if the arcz argument is omitted.

  4. Gary Bernhardt reporter

    I spent some more time trying to fix it, but it looks quite hard because it mixes levels in a way that coverage doesn't have to deal with currently. The generic except check is done by comparing the name of the thrown exception to each of the named except: blocks (LOAD_NAME/COMPARE_OP/JUMP_IF_FALSE). I tried to put in a horrible hack where I cross-referenced the line number of the current byte code against the original text of the code, and avoided adding it as an exit point if I was at the except: line. But then I started getting a headache and gave up. :(

    This probably isn't useful to you, given the amount of time you've spent with this stuff, but I thought I'd share. :) Thanks for all of the work you've done so far; I use coverage hundreds of times in an average day.

  5. Ned Batchelder repo owner

    Gary, there's another possibility: the bytecode for except clauses has a distinctive signature:

    COMPARE_OP 10 (exception match)

    But I'm also wondering whether it's right to hide the comparison being done here. And underlying all the uncertainty is the fact that no coverage theory out there seems to consider exception handling in the first place...

  6. Gary Bernhardt reporter

    Here's my reasoning about it: In Python, any line could potentially throw an exception. Code within a try without a catch-all except isn't actually any different in this regard (unless I'm missing something). So if we count potentially uncaught exceptions within a try as branches, we really should count every non-trivial line of code in that way, which we clearly don't want. :)

  7. Log in to comment