generated xml invalid paths for Cobertura report

Issue #526 resolved
Dirk Thomas
created an issue

When generating the coverage report as a .xml file the source + filename does not add up to a correct path. As a result the Jenkins Cobertura plugin can't show the source of each module: http://ci.ros2.org/job/test_coveragepy/2/cobertura/_/ament_package_dependency_py/

I have created a Jenkins job to reproduce the issue: http://ci.ros2.org/job/test_coveragepy/2/

The steps are fairly simple:

cd  /tmp
python3 -m venv venv
. venv/bin/activate
pip install coverage mock nose
git clone https://github.com/ament/ament_package.git
cd ament_package
python3 -m nose --nocapture --with-coverage --cover-package ament_package --cover-xml --cover-xml-file coverage.xml

The resulting coverage.xml file contains a source entry pointing to the Python module .../ament_package/ament_package. The class entries have a filename like ament_package/dependency.py and a name (dependency.py). The problem is (as far as I can see) that the source + filename doesn't add up to a correct path. Basically the package name folder is repeated in both. The correct full path should be .../ament_package/ament_package/dependency.py.

The full console output can be found here and the full coverage.xml file here.

Comments (27)

  1. Loic Dachary

    I've been able to run the reproducer and obtain the XML file that matches your description. I'm not able to reproduce the problem because I don't have access to a jenkins running the cobertura plugin. Could you confirm that running without the --cover-package ament_package generates an XML file with links to sources that are correct ?

    python3 -m nose --nocapture --with-coverage --cover-xml --cover-xml-file coverage.xml
    

    The sources element does not contain the extra package in this case. I've not been able to find the documentation for the expected content of the XML file. The DTD is validated and correct but only set the requirements for the elements and the attributes, not their content.

  2. Dirk Thomas reporter

    I just ran a third build without the argument (http://ci.ros2.org/job/test_coveragepy/3/). The result is "interesting". The breakdown list contains multiple rows - the relevant ones are "." and "ament_package". For the second one the sources are shown correctly. For the first one the problem is still the same. I think the build no. 2 (with the argument) only showed the "." entry which didn't work in.

    I hope that helps...

  3. Dirk Thomas reporter

    I was surprised since it installs the package from PIP. Therefore I added a pip freeze call to the latest build (http://ci.ros2.org/job/test_coveragepy/5/console). It shows that it installed coverage==4.3.1. So it looks like that the two lines during the installation are kind of misleading:

    Installing coverage-3.4 script to /var/lib/jenkins/jobs/test_coveragepy/workspace/venv/bin
    Installing coverage3 script to /var/lib/jenkins/jobs/test_coveragepy/workspace/venv/bin
    
  4. Loic Dachary

    @Dirk Thomas thanks for checking. And the XML file consistently displays the coverage version as well. It's not a version mismatch then.

    <coverage branch-rate="0" line-rate="0.3515" timestamp="1483719279629" version="4.3.1">
    
  5. Loic Dachary

    @Dirk Thomas could you please try to run coverage run -m nose ; coverage xml instead of python3 -m nose --nocapture --with-coverage --cover-xml --cover-xml-file coverage.xml ? The result is different when I run it locally. If it displays correctly we will know that the nose coverage plugin does something that is not right. If the output does not display well... we will be even more confused ;-)

  6. Loic Dachary

    @Dirk Thomas interesting. Do you know of a project successfully using the Cobertura Coverage Report jenkins plugin with coverage.py ? It would be helpful to see how it looks. I'll analysing this new run and try to guess what's going on.

  7. Loic Dachary

    @Dirk Thomas could you please try a run with coverage run --source=ament_package -m nose ; coverage xml ? That will change paths to be relative instead of absolute. It should not change anything in theory. Except if jenkins moves the directory around and the absolute paths is no longer valid. If, as I suspect, this change does not improve things, at least it will tell us the problem is unrelated to absolute vs relative path names.

  8. Ned Batchelder repo owner

    OK, the nose plugin is doing something weird. The coverage xml command produces the correct results. I'll take a look at what the plugin is doing and see if there is a way to have both work.

  9. bmerry

    I can confirm that using coverage xml after running the nose plugin works for me (with the Jenkins Cobertura plugin), while using nosetests with --cover-xml generates broken paths.

    The nose plugin actually generates nicer filenames e.g. mypackage/myfile.py instead of venv/lib/python2.7/site-packages/mypackage/myfile.py; but unfortunately the plugin sets to the source to venv/lib/python2.7/site-packages/mypackage instead of venv/lib/python2.7/site-packages, so the combined paths are wrong.

  10. Dirk Thomas reporter

    I don't think that the referenced commit fixes this. The problem described in this ticket is not related to __init__.py files. The problem is that the two pieces of information (source and filename) do not add up to a correct path. Therefore I think this should be reopened.

  11. Log in to comment