Using a script to run coverage on baseline and then changeset as part of OpenStack tests for commits. The script cleans .pyc files and then runs coverage, checks out the newer code and does the same process, so can compare results. However, see different coverage results on each run.
Even when there are no code changes, the coverage results are different, on different runs.
Here is the OpenStack code review code that runs the coverage checks (hopefully you have access to this opensource code):
Even in the test run for this commit (which is only the coverage scripting), it shows the project coverage with minor differences between baseline and changeset (no code changes). Some files report more or less coverage.
Here is a link that a co-worker did of running this script multiple times with different changes in the project repo:
Let me know if you need the script or other information. It's preventing us from using the coverage tool upstream, so please help!