4.3 fails if there are HTML files in the source code

Issue #556 resolved
amercader
created an issue

It seems like 4.3 is trying to parse HTML files that are part of the source code, while 4.2 didn't. We noticed this in our Travis builds:

https://travis-ci.org/frictionlessdata/goodtables.io/builds/192669485

This is running our tests with pytest with pytest --cov goodtablesio --cov-report term-missing but running coverage report -m locally also raises the exception:

coverage report -m
Traceback (most recent call last):
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/parser.py", line 364, in __init__
    self.code = compile_unicode(text, filename, "exec")
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/phystokens.py", line 286, in compile_unicode
    code = compile(source, filename, mode)
  File "/home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/error404.html", line 1
    {% extends "layout.html" %}
     ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/summary.py", line 46, in report
    analysis = self.coverage._analyze(fr)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/control.py", line 899, in _analyze
    return Analysis(self.data, it)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/results.py", line 19, in __init__
    self.statements = self.file_reporter.lines()
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/python.py", line 176, in lines
    return self.parser.statements
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/python.py", line 171, in parser
    self._parser.parse_source()
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/parser.py", line 238, in parse_source
    self._raw_parse()
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/parser.py", line 207, in _raw_parse
    self.raw_statements.update(self.byte_parser._find_statements())
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/parser.py", line 96, in byte_parser
    self._byte_parser = ByteParser(self.text, filename=self.filename)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/parser.py", line 368, in __init__
    filename, synerr.msg, synerr.lineno
coverage.misc.NotPython: Couldn't parse '/home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/error404.html' as Python source: 'invalid syntax' at line 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adria/dev/pyenvs/gt/bin/coverage", line 11, in <module>
    sys.exit(main())
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/cmdline.py", line 756, in main
    status = CoverageScript().command_line(argv)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/cmdline.py", line 507, in command_line
    skip_covered=options.skip_covered, **report_args)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/control.py", line 977, in report
    return reporter.report(morfs, outfile=file)
  File "/home/adria/dev/pyenvs/gt/lib/python3.5/site-packages/coverage/summary.py", line 58, in report
    except StopEverything:
TypeError: catching classes that do not inherit from BaseException is not allowed

4.2 works as expected

Comments (14)

  1. amercader reporter

    @Ned Batchelder here you go:

    coverage debug data
    -- data ------------------------------------------------------
    path: /home/adria/dev/pyenvs/gt/src/goodtables.io/.coverage
    has_arcs: False
    
    72 files:
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/__init__.py: 5 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/app.py: 24 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/auth.py: 19 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/blueprints/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/blueprints/api.py: 20 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/blueprints/site.py: 18 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/blueprints/user.py: 12 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/celery_app.py: 5 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/exceptions.py: 6 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/models/__init__.py: 2 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/models/base.py: 15 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/models/job.py: 28 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/models/user.py: 25 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/__init__.py: 4 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/blueprint.py: 25 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/models/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/models/repo.py: 23 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/signals.py: 5 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/tasks/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/tasks/jobconf.py: 12 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/tasks/repos.py: 7 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/utils/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/utils/hook.py: 5 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/utils/repos.py: 2 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/utils/signature.py: 4 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/plugins/github/utils/status.py: 5 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/services.py: 2 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/settings.py: 24 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/signals.py: 9 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tasks/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tasks/validate.py: 6 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/error404.html: 14 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/header.html: 14 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/home.html: 19 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/job.html: 36 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/jobs.html: 22 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/templates/layout.html: 44 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/blueprints/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/blueprints/test_api.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/blueprints/test_site.py: 24 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/blueprints/test_user.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/conftest.py: 17 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/factories.py: 41 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/models/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/models/test_base.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/models/test_job.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/models/test_user.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/conftest.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/models/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/models/test_repo.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/tasks/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/tasks/test_jobconf.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/tasks/test_repos.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/test_blueprint.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/utils/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/utils/test_hook.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/utils/test_repos.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/utils/test_signature.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/plugins/github/utils/test_status.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/tasks/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/tasks/test_validate.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/test_factories.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/utils/__init__.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/utils/test_jobconf.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/tests/utils/test_jobtask.py: 0 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/utils/__init__.py: 1 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/utils/database.py: 7 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/utils/jobconf.py: 16 lines
    /home/adria/dev/pyenvs/gt/src/goodtables.io/goodtablesio/utils/jobtask.py: 6 lines
    
  2. Ned Batchelder repo owner

    @SGSabbage hmm, thanks for the data point, but that seems really unlikely. There's only one line of product code different between 4.3.2 and 4.3.3, and that should only affect either 2.6 or Jython. Can you verify? Perhaps there's data carrying over?

  3. SGSabbage

    Pretty sure it's the change between 4.3.3 and 4.3.2 - if I just change the version of the library I alternate between passing and failure.

    Full traceback (with some data redacted)

    (hub) ubuntu@ubuntu-xenial:/vagrant$ pip install coverage==4.3.3 --upgrade
    Collecting coverage==4.3.3
    Installing collected packages: coverage
      Found existing installation: coverage 4.3.2
        Uninstalling coverage-4.3.2:
          Successfully uninstalled coverage-4.3.2
    Successfully installed coverage-4.3.3
    (hub) ubuntu@ubuntu-xenial:/vagrant$ py.test --cov -q
    .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................INTERNALERROR> Traceback (most recent call last):
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/parser.py", line 238, in parse_source
    INTERNALERROR>     self._raw_parse()
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/parser.py", line 139, in _raw_parse
    INTERNALERROR>     tokgen = generate_tokens(self.text)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/phystokens.py", line 148, in generate_tokens
    INTERNALERROR>     self.last_tokens = list(tokenize.generate_tokens(readline))
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/tokenize.py", line 597, in _tokenize
    INTERNALERROR>     raise TokenError("EOF in multi-line statement", (lnum, 0))
    INTERNALERROR> tokenize.TokenError: ('EOF in multi-line statement', (59, 0))
    INTERNALERROR>
    INTERNALERROR> During handling of the above exception, another exception occurred:
    INTERNALERROR>
    INTERNALERROR> Traceback (most recent call last):
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/summary.py", line 46, in report
    INTERNALERROR>     analysis = self.coverage._analyze(fr)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/control.py", line 899, in _analyze
    INTERNALERROR>     return Analysis(self.data, it)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/results.py", line 19, in __init__
    INTERNALERROR>     self.statements = self.file_reporter.lines()
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/python.py", line 176, in lines
    INTERNALERROR>     return self.parser.statements
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/python.py", line 171, in parser
    INTERNALERROR>     self._parser.parse_source()
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/parser.py", line 246, in parse_source
    INTERNALERROR>     self.filename, err.args[0], lineno
    INTERNALERROR> coverage.misc.NotPython: Couldn't parse '/vagrant/hub/templates/form.jinja2' as Python source: 'EOF in multi-line statement' at line 59
    INTERNALERROR>
    INTERNALERROR> During handling of the above exception, another exception occurred:
    INTERNALERROR>
    INTERNALERROR> Traceback (most recent call last):
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/main.py", line 98, in wrap_session
    INTERNALERROR>     session.exitstatus = doit(config, session) or 0
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/main.py", line 133, in _main
    INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
    INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 250, in _wrapped_call
    INTERNALERROR>     wrap_controller.send(call_outcome)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/pytest_cov/plugin.py", line 236, in pytest_runtestloop
    INTERNALERROR>     self.cov_total = self.cov_controller.summary(self.cov_report)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/pytest_cov/engine.py", line 95, in summary
    INTERNALERROR>     total = self.cov.report(**options)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/control.py", line 977, in report
    INTERNALERROR>     return reporter.report(morfs, outfile=file)
    INTERNALERROR>   File "/home/ubuntu/.virtualenvs/hub/lib/python3.5/site-packages/coverage/summary.py", line 58, in report
    INTERNALERROR>     except StopEverything:
    INTERNALERROR> TypeError: catching classes that do not inherit from BaseException is not allowed
    
    535 passed in 18.67 seconds
    (hub) ubuntu@ubuntu-xenial:/vagrant$ pip install coverage==4.3.2 --upgrade
    Collecting coverage==4.3.2
    Installing collected packages: coverage
      Found existing installation: coverage 4.3.3
        Uninstalling coverage-4.3.3:
          Successfully uninstalled coverage-4.3.3
    Successfully installed coverage-4.3.2
    (hub) ubuntu@ubuntu-xenial:/vagrant$ py.test --cov -q
    .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    
    ----------- coverage: platform linux, python 3.5.2-final-0 -----------
    Name            Stmts   Miss Branch BrPart  Cover   Missing
    -----------------------------------------------------------
    hub/store.py        4      3      2      0    17%   2-5
    hub/tweens.py       8      1      2      1    80%   8, 7->8
    -----------------------------------------------------------
    TOTAL            2495      4    696      1    99%
    
    29 files skipped due to complete coverage.
    
    535 passed in 19.85 seconds
    (hub) ubuntu@ubuntu-xenial:/vagrant$
    

    Environment:

    platform linux -- Python 3.5.2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
    rootdir: /vagrant, inifile: tox.ini
    plugins: celery-4.0.2, hypothesis-3.6.0, mock-1.5.0, cov-2.4.0
    

    Happy to return any additional data required

  4. Log in to comment