Soft failure for loading older data files

Issue #428 resolved
Ionel Cristian Mărieș created an issue

Currently coverage fails hard when loading datafiles that were generated with older 3.7.

This is inconvenient for me in a "peculiar" test.

============================= test session starts ==============================

platform linux2 -- Python 3.2.5, pytest-2.8.2, py-1.4.30, pluggy-0.3.1 -- /home/travis/build/ionelmc/pytest-benchmark/.tox/3.2-pytest28-nodist-cover/bin/python3.2

cachedir: .cache

rootdir: /home/travis/build/ionelmc/pytest-benchmark, inifile: setup.cfg

plugins: benchmark-3.0.0a4, cov-2.2.0

collected 9 items 

tests/test_normal.py::test_normal PASSED

tests/test_normal.py::test_fast SKIPPED

tests/test_normal.py::test_slow SKIPPED

tests/test_normal.py::test_slower SKIPPED

tests/test_normal.py::test_parametrized[0] SKIPPED

tests/test_normal.py::test_parametrized[1] SKIPPED

tests/test_normal.py::test_parametrized[2] SKIPPED

tests/test_normal.py::test_parametrized[3] SKIPPED

tests/test_normal.py::test_parametrized[4] SKIPPED

--------------- coverage: platform linux2, python 3.2.5-final-0 ----------------

Name                             Stmts   Miss Branch BrMiss     Cover   Missing

-------------------------------------------------------------------------------

src/pytest_benchmark/__init__        1      1      0      0     0.00%   1

src/pytest_benchmark/compat          4      4      0      0     0.00%   1-6

src/pytest_benchmark/histogram      47     47     24     24     0.00%   1-102

src/pytest_benchmark/plugin        579    527    226    214     7.95%   1-435, 440, 447-470, 489-493, 495, 505, 519, 523, 526-551, 557-843, 847-850, 853, 858-935, 942-962, 965-974

src/pytest_benchmark/stats          88     88     30     30     0.00%   1-160

src/pytest_benchmark/timers         31     31     10     10     0.00%   1-38

src/pytest_benchmark/utils         176    154     56     51    11.64%   1-40, 45-46, 49, 54-96, 98, 105, 107, 110-115, 122-162, 166, 168, 172-183, 186-220, 222, 227-273

-------------------------------------------------------------------------------

TOTAL                              926    852    346    329     7.15%   

=========================== short test summary info ============================

SKIP [8] /home/travis/build/ionelmc/pytest-benchmark/src/pytest_benchmark/plugin.py:940: Benchmarks are disabled.

============ 1 passed, 8 skipped, 1 pytest-warnings in 0.18 seconds ============

coveralls create: /home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls

  /home/travis/build/ionelmc/pytest-benchmark/.tox$ /home/travis/virtualenv/python3.5.0/bin/python -m virtualenv --python /opt/python/3.4.2/bin/python3.4 coveralls >/home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls/log/coveralls-0.log

coveralls installdeps: coveralls

  /home/travis/build/ionelmc/pytest-benchmark$ /home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls/bin/pip install coveralls >/home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls/log/coveralls-1.log

coveralls installed: coverage==4.0,coveralls==1.0,docopt==0.6.2,requests==2.8.0,wheel==0.24.0

coveralls runtests: PYTHONHASHSEED='392714931'

coveralls runtests: commands[0] | coverage combine

  /home/travis/build/ionelmc/pytest-benchmark$ /home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls/bin/coverage combine 

Couldn't read data from '/home/travis/build/ionelmc/pytest-benchmark/.coverage.testing-worker-linux-docker-f2f3ad1b-3206-linux-4.1953.616168': UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

ERROR: InvocationError: '/home/travis/build/ionelmc/pytest-benchmark/.tox/coveralls/bin/coverage combine'

codecov create: /home/travis/build/ionelmc/pytest-benchmark/.tox/codecov

  /home/travis/build/ionelmc/pytest-benchmark/.tox$ /home/travis/virtualenv/python3.5.0/bin/python -m virtualenv --python /opt/python/3.4.2/bin/python3.4 codecov >/home/travis/build/ionelmc/pytest-benchmark/.tox/codecov/log/codecov-0.log

codecov installdeps: codecov

  /home/travis/build/ionelmc/pytest-benchmark$ /home/travis/build/ionelmc/pytest-benchmark/.tox/codecov/bin/pip install codecov >/home/travis/build/ionelmc/pytest-benchmark/.tox/codecov/log/codecov-1.log

codecov installed: codecov==1.5.1,coverage==4.0,requests==2.8.0,wheel==0.24.0

codecov runtests: PYTHONHASHSEED='392714931'

codecov runtests: commands[0] | coverage combine

  /home/travis/build/ionelmc/pytest-benchmark$ /home/travis/build/ionelmc/pytest-benchmark/.tox/codecov/bin/coverage combine 

Couldn't read data from '/home/travis/build/ionelmc/pytest-benchmark/.coverage.testing-worker-linux-docker-f2f3ad1b-3206-linux-4.1953.616168': UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

ERROR: InvocationError: '/home/travis/build/ionelmc/pytest-benchmark/.tox/codecov/bin/coverage combine'

___________________________________ summary ____________________________________

  3.2-pytest28-nodist-cover: commands succeeded

ERROR:   coveralls: commands failed

ERROR:   codecov: commands failed

The command "tox -v" exited with 1.

https://travis-ci.org/ionelmc/pytest-benchmark/jobs/84836346

I would upgrade to 4.0 but it don't work on 3.2 and I specifically want to test something there (crazy, don't ask).

Comments (4)

  1. Ionel Cristian Mărieș reporter

    I think I'm complaining here about the fact that load failures are hard failures (instead of some warnings). But that was fixed or was it not?

  2. Log in to comment