Combine from same python process only considers first coverage run when using API
My use case:
I have a package with multiple modules. For each modules, I have a corresponding test module. I want to measure the coverage for each module explicitly targeted by its corresponding test module without side effects from other test modules and output the results in a single report.
Here's how I achieve this with command line:
coverage erase coverage run -p --branch --include=package/a_module.py nosetests test_package/test_a_module coverage run -p --branch --include=package/another_module.py nosetests test_package/test_another_module etc.. coverage combine coverage report -m
I wanted to run all this from within a python script:
import nose from coverage import coverage cov = coverage() cov.erase() for module in my_modules: module_cov = coverage(branch=True, include=path_to_module, data_suffix=True) module_cov.start() nose.run(argv=['', path_to_test_module]) module_cov.stop() module_cov.save() cov.combine() cov.save() cov.report(show_missing=True)
With this script, when looking at the results, the coverage is fine for the first module, but all other modules have no more coverage than the coverage coming from importing the module.
I realized that it comes from parallel coverage files.
With command line, I get:
.coverage.machine_name.first_coverage_run_PID.first_nose_run_PID .coverage.machine_name.second_coverage_run_PID.second_nose_run_PID etc...
But, when using the python script, I obviously get:
.coverage.machine_name.python_script_run_PID.first_nose_run_PID .coverage.machine_name.python_script_run_PID.second_nose_run_PID etc...
And it seems the bug comes from the parallel coverage files having the same python_script_run_PID.
My workaround currently is to use multiprocessing.Process to get different python_script_run_PID for each module:
def run_module_coverage(module_path, test_module_path): cov = coverage(branch=True, include=module_path, data_suffix=True) cov.start() nose.run(argv=['', test_module_path]) cov.stop() cov.save() cov = coverage() cov.erase() for module in my_modules: process = Process(target=run_module_coverage, args=(module_path, test_module_path)) process.start() process.join() cov.combine() cov.save() cov.report(show_missing=True)
And it works as expected.