Last of certain multi-line statements is handled, instead of the first

Issue #180 invalid
Michael Armida created an issue

== Symptoms ==

Certain multi-line statements are mishandled, causing bad line numbers to be reported. Given this unit test:

{{{ import multiline

def test_something(): pass }}}

And this code in {{{}}}:

{{{ def something(): a=\ 1 }}}

Then 'coverage report' says line 3 is missing, instead of the expected 2, and 'coverage annotate' produces:


def something(): a=\ ! 1 }}}

I have observed this behavior under both 3.5.2, and the latest from BitBucket.

I've found a few other statements that cause line number weirdness:

{{{ print (1, \ 2) print [i for i in range(1)] }}}

The first item above gets treated like the original example; the second line gets counted. The listcomp case is weird: both lines get counted.

== Investigation ==

I've done some digging with pdb, and it seems like the culprit is {{{CodeParser._raw_parse}}}. Two things go wrong:

the returned {{{statements}}} list contains the last line of the multi-line statement

the {{{multiline}}} attribute doesn't contain an entry for this line

In the case of both 'coverage annotate' and 'coverage report', setting a breakpoint in {{{}}} at line 30, and doing some fiddling, saves the day. This is line 30:

{{{ self.statements, self.excluded = self.parser.parse_source() }}}

At that point, using the first example above, I ran the following in pdb:

{{{ self.statements = [1, 2] self.parser.multiline = {2: (2, 3), 3: (2, 3)} }}}

The next few lines of {{{}}} calculate the missed source lines by washing executed lines through {{{CodeParser.first_lines}}}, which depends on the contents of the {{{multiline}}} attribute. After the above fixes, everything's hunky dory.

This issue might be related to #66, but I'm running Linux, and these files have no CRs.

I'm going to continue investigating the token crawling code in {{{CodeParser._raw_parse}}}, and if I can, will submit a patch and a unit test.

Comments (3)

  1. Ned Batchelder repo owner

    I haven't looked into this yet, but my guess is that the true culprit is CPython. It gives us the set of executable lines to begin with.

  2. Michael Armida reporter

    You're right. CodeParser doesn't actually find the line numbers. ByteParser._find_statements is called, and inspects a compiled code object:

    >>> coverage.parser.ByteParser(text='a=\\\n1', filename='')._find_statements()

    Specifically, ByteParser._find_statements calls the _bytes_lines method, which uses the code object's co_firstlineno. For the above statement, that attribute is 2, instead of the expected 1. cPython must be getting that line number wrong when it compiles the code.

  3. Michael Armida reporter

    Side note: inspecting the source via tokenize yields the correct line numbers. But since coverage goes to the trouble of doing bytecode inspection, I assume just looking at source is insufficient.

  4. Log in to comment