Issue #31 resolved

IndentationErrors caused by mixed tab/space indents not caught

Danek Duvall
created an issue

When you have two lines of identical indentation, except that one uses spaces and the other uses tabs (eight spaces to a tabstop), then the tokenize module raises an IndentationError which coverage doesn't catch.

This can be considered a bug in the indentation, but it's only a style bug, as the code works just fine.

This could be considered a bug in tokenize, but I assume it's unlikely to get fixed.

It's a bit of a pain to fix in coverage, I think, because you're seeing it in the looping expression of a for loop. The loop would have to be restructured to pull the tokens from the generator inside the loop, inside a try/except, and either error out usefully (with filename and line number!) or try to go on. Still, it might not be too bad.

The attached file demonstrates the problem.

Comments (3)

  1. Ned Batchelder repo owner

    Actually, I explicitly expandtabs(4) just before tokenizing, so the bug is mine. But I wonder how best to know what size tabs you use? I guess defaulting to 8 is a better choice....

  2. Danek Duvall reporter

    We use 8, which seems like the sanest default to me. Most code I see which uses 4-space indent either goes all spaces or mixes spaces and tabs. If there are folks relying on the expandtabs(4) behavior, I suppose you could always add a flag to specify tab width; I dunno.

  3. Log in to comment