--include=dir is ignored with --concurrency=multiprocessing

Issue #447 closed
hugovk created an issue

The problem I mentioned here still appears to be happening.

I'd like to start using nose's parallel testing to speed up Pillow's tests on Travis CI.

It looks like the --include=diris ignored with --concurrency=multiprocessing.

Currently it does something like this (using Coverage 4) (before some later processing to include C coverage, but let's ignore that for now).

coverage erase
coverage run --append --include=PIL/* selftest.py
coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py
coverage report

coverage report only lists files under PIL/ as expected, coverage is 81% (1863 missed of 9572). For example: https://travis-ci.org/hugovk/Pillow/jobs/91564047#L2566

To use nose's parallel testing, I added --processes=-1 --process-timeout=600 to nose, and then add --concurrency=multiprocessing to coverage, and run coverage combine at the end:

coverage erase
coverage run --include=PIL/* selftest.py
coverage run --include=PIL/* --concurrency=multiprocessing -m nose -vx --processes=-1 --process-timeout=600 Tests/test_*.py
coverage combine
coverage report

Nose runs the 670 tests in parallel, they take under 16 seconds instead of over 60 seconds.

It seems --include=PIL/* is being ignored as coverage report gives 43% coverage, 14431 of 28344 lines missed, and also lists files under Tests/ and /home/travis/virtualenv/python2.7.9/ which isn't desired. For example: https://travis-ci.org/hugovk/Pillow/jobs/91562327#L2737

Comments (5)

  1. Ned Batchelder repo owner

    Can I reproduce the problem with these steps? What repo should I clone, and what exact series of command should I use to get the behavior you are describing?

  2. hugovk reporter

    This is the repo/branch you need to reproduce: https://github.com/hugovk/Pillow/tree/processes_nose3

    The steps to reproduce are the contents of .travis.yml's install: and script: sections up to the first coverage report (the rest can be ignored/removed):

    https://github.com/hugovk/Pillow/blob/processes_nose3/.travis.yml#L47

    There's quite a bit of setup/installation there, but after you clone, you can enable your repo on Travis it's possible to test on there: https://travis-ci.org/profile

  3. Ned Batchelder repo owner

    @hugovk I don't understand everything here, but I believe the crux of the matter is that the command-line arguments to coverage are not being passed into the sub-processes. I don't understand how the subprocesses are running coverage in the first place! Actually, looking at the results, maybe they aren't?

    In any case, if you move the --include=PIL/* argument into the .coveragerc file instead, it works as you want it to.

  4. Log in to comment