Allow tox to ignore missing python interpretors - pass when some are missing

Issue #59 on hold
Sorin Ionuț Sbârnea
created an issue

Tox should be able to run tests on all available (defined) interpretors but it should not fail if one or more of them are missing.

This is a feature request that would allow users to execute tox on various systems using the same tox.ini command.

It would be essential for tox to return a success return code when this option is used.

One of the main reasons for asking this, is usage of tox in automation, where you want to run tox on different platforms with the same configuration for all of them.

Comments (11)

  1. Holger Krekel repo owner

    There would be the caveat that if you acccidentally have an interpreter missing on some host, then the CI automation run would still report success. I guess you are aware that you can enumerate envs with "-e py25,py26,py27". This is bound to get some more wildcarding flexibility soon.

  2. Łukasz Balcerzak

    @Holger Krekel any decision on this? Personally, I would probably reject this. Tools should promote best practices and proposed switch doesn't look like one of them. When configuring CI one should know about available Python versions and for user it is probably better to see what interpreters are missing on their system but are supported by package being tested. It's just either passing that extra switch or Python versions explicitly, as it was already said.

    However, if you'd accept this enhancement, I could work on it (looks like a simple changes).

  3. Holger Krekel repo owner

    I think introducing a "skipped" state for missing interpreters or failures to install dependencies is useful. I have setups where i run tox on multiple hosts and if one host cannot run a test environment, it's not a problem. So i think that "failures" should be ones where the test commands failed and nothing else. @Łukasz Balcerzak , if you want to implement this change, i'd be happy to review the PR. Please note that tox since the last release writes a result json file which should also reflect this new "skipped" state.

  4. Barry Warsaw

    I do think it makes sense to skip missing environments. Currently, when building tox-tested packages for Debian, I have to fiddle with the command line to get a proper -e argument (which isn't completely trivial given the mapping from pythonX.Y -> pyXY). On Debian, we'll only have at most a couple of relevant Python environments we want to run the tests on, e.g. python2.7 and python3.3 for now (but probably including python3.4 at some point).

  5. Alexandre Conrad

    I'd like to see this happening, too. Our Jenkins instance supports 2.6, 2.7, pypy, 3.3, and 3.4. But my tests fail locally because I don't have all of these environments installed. A SKIPPED state would be nice.

  6. Log in to comment