Incomplete file path in XML report

Issue #578 new
Anonymous created an issue

Hi there, we are relying on the XML coverage report generated in coverage.xml for our analysis.

In previous versions of coverage, when we run the command such as:

$ py.test --cov directory_one --cov directory_two --cov-report xml tests

We would get a coverage.xml file with sources listed as:

<sources>
    <source>/path/to/directory_one</source>
    <source>/path/to/directory_two</source>
</sources>

And each file would be reported in a line such as this:

<class branch-rate="0" complexity="0" filename="directory_one/__init__.py" line-rate="1" name="__init__.py">

Now in coverage 4.4 the first level of the directory path is missing and we get only:

<class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">

Is there any way to get the previous behaviour?

Comments (30)

  1. Ned Batchelder repo owner

    This was a bug that was fixed. The filename attribute should be combinable with a source entry to find the file. Can you tell me more about why you need the directory name in both places?

  2. Michal Karzynski

    If we have 2 files like this:

    • /path/to/directory_one/__init__.py
    • /path/to/directory_two/__init__.py

    Then they would both get a like like this:

    <class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
    

    How can I tell which file this refers to?

  3. David Multer

    I'm seeing the same thing. Not only is the source file reference ambiguous, it has also broken my use of coverage2clover in my Bamboo builds as it can't find the source file.

  4. David Multer

    No idea on how source paths is used, but the coverage2clover tool seems to just do an open() on the filename= value, so it must go back to the way it was previously for it to work again. Note that my coverage run produces multiple source paths as I use a shared repo for common code.

        <sources>
            <source>/Users/xxx/Documents/xxx/app-common/apis</source>
            <source>/Users/xxx/Documents/xxx/app-foo</source>
            <source>/Users/xxx/Documents/xxx/app-foo/apis</source>
        </sources>
    

    and

        <class branch-rate="0" complexity="0" filename="apis/__init__.py" line-rate="1" name="__init__.py">
    
  5. Ned Batchelder repo owner

    @David Multer @Michal Karzynski Can you provide me with a reproducible example, even if it's a link to an online repo with instructions of how to run the tests? I want to get this to work again for you. It's clear to me that combining the source path and the filename should be a real file path, so I think the fix was the correct one, but clearly I am not done since your use cases are broken.

  6. Ned Batchelder repo owner

    I want to make tools work, but I want all of them to work. I'm not going to simply revert the change. I'll make an improvement that will help everyone. It would be a big help to me if someone could provide a reproducible case of a multi-source result. Even better would be to advocate with them to make use of all of the information in the coverage.xml file.

  7. Ned Batchelder repo owner

    @David Multer thanks for that. I'm trying to understand the old behavior you want back. When I run your code under coverage==4.3.4, only two/__init__.py is in the XML output. one is missing completely. So the old code had some problems with this multiple-source scenario also. Is that true of your real code also?

  8. David Multer

    No, I think that's a side effect of not having any actual code to test in my example. You could easily just add a do-nothing function in each to get file output for both. You'll see the same problem. I think I'll go add it to the repo now...

  9. Ned Batchelder repo owner

    Yes, thanks. When I run this code with coverage 4.3.4, two/__init__.py is in the coverage1.xml file, but one/__init__.py is not. Even if I add an executable line to each of those files, only one of them appears in the results. There's more to be fixed here.

  10. David Multer

    Yes, coverage1.xml shows the single init.py file problem, and the coverage2.xml files shows just the filespec problem tools will have knowing where foo.py or bar.py are. I have the coverage2clover maintainer looking at these too.

  11. Ned Batchelder repo owner

    Just so we're clear: my point is about the old behavior in coverage.py 4.3.4: it didn't include all of the files in the XML report if the files had same names in different source trees. So just putting back the old behavior won't be enough. Maybe we agree on this, I'm not sure?

  12. David Multer

    Yes, I see that though I never noticed the problem before. I'll have to think a bit to understand why. So fixing that problem would probably be important, but if it still existed and the code went back I would think all was well ;-)

    It's the second problem that I felt was the real issue. I'm waiting to see what the coverage2clover person says...

  13. David Multer

    I've confirmed that files in different modules with the same name (i.e. init.py) never did show in my coverage output and I just failed to notice that I had no data one those files. Getting that fixed is obviously important, but less so than making sure my XML processing tools are able to parse the output properly.

  14. Dirk Thomas

    In order to avoid the ambiguity (while still keep the information of the sources basepath and the filename within that basepath separate) could the class tag get a new attribute source to clearly identify the base path the referenced file was being found? Obviously that would require downstream tools to be updated too but I don't see a good solution which works in all cases based on the information present in the current xml.

  15. Alex Gaynor

    This is hitting us on pyca/cryptography. Can be reproduced with:

    $ git clone https://github.com/pyca/cryptography
    $ cd cryptography
    $ tox -e py34 -- tests/test_x509.py
    $ coverage xml -i
    

    As you can see in the tox output's coverage report, there paths in our test directory have a test/ prefix, but after running coverage xml, in coverage.xml, files from the test/ directory do not. An easy file to see this with is test_x509.py.

  16. Alex Gaynor

    I'm not particularly convinced this change is correct, but looking at: https://bitbucket.org/ned/coveragepy/src/63dfe482a849e9cb0b82214242315a806a6a3d53/coverage/xmlreport.py?at=default&fileviewer=file-view-default#xmlreport.py-143:151

    Our challenge is that we have two paths in self.source_paths: /path/to/cryptography and /path/to/cryptographt/tests. Since it's a set, depending on your version of Python, ordering may not be stable, if /path/to/cryptography/tests comes first, rel_name ends up as test_x509.py, if /path/to/cryptography comes first you get rel_name = "tests/test_x509.py".

    The fix, which I'm not particularly confident in, is to change line 146 to be: for source_path in sorted(self.source_paths, key=len):.

    This will ensure we always get /path/to/cryptography/ first here. Tested locally and it works, but I have no sense of whether this is conceptually correct :-)

  17. Alexandre Conrad

    Also, pycobertura is affected by this change too.

    There should be a way to have a relative path to the source files when starting from the root of the package and we lost this information. Currently the <source> tag only provides absolute paths to the source on the machine it was generated on which isn't very useful if the coverage report gets interpreted on a different machine where the absolute path is unlikely to exist.

    For example, if a machine receives a coverage report that was generated for package "foo" at commit "abc123" with a filename __init__.py (which was originally foo/__init__.py) then we should be able to git-clone the source of package foo@abc123 and reconcile the paths somehow. But with the change there is no way to do this anymore.

    • The <source> tag only provides an absolute path that is not portable from machine to machine
    • The <class> tag has filename="__init__.py"
    • The <package> tag has name="."

    We're missing the information about the directory foo.

  18. Suriya Subramanian

    I am using coverage.py version 4.3.4. I see that only one file with a particular base name at a particular depth is reported. For example, with three files a/__init__.py, b/__init__.py, and a/c/__init__.py, we get only two files in the output a/__init__.py and a/c/__init__.py. This is especially problematic with a Django project and we get a report for only one models.py file.

  19. Log in to comment