We have always used tox to run our coverage commands. However recently we started running coverage on py2 and py3 and combining it into a single coverage report that should be 100%.
Anyway, this is done using 3 environments. One for py2, one for py3 and one to aggregate the results.
This is all fine but the interface has changed from
tox -e cover to
tox -e py2-cover,py3-cover,cover. Ideally tox would possibly support some sort of grouping instead of requiring us to wrap this invocation in something else.
But wait, we found a scary solution!
[testenv:py2-cover] commands = coverage run ... setenv = COVERAGE_FILE=.coverage.py2 [testenv:py3-cover] commands = coverage run ... setenv = COVERAGE_FILE=.coverage.py3 [textenv:cover] commands = coverage erase tox -e py2-cover tox -e py3-cover coverage combine coverage xml setenv = COVERAGE_FILE=.coverage deps = tox
A recursive tox file! Anyway this has some downsides like
tox -r is not propagated downward to the sub-toxes. However this solves our issues with the CLI, and even gives us the opportunity to do things prior to the sub-toxes like erase without introducing yet another tox env.
I don't have an actual proposal but I wanted to open an issue and get some thoughts on possible solutions inside or outside of tox.