1. Ned Batchelder
  2. coverage.py
  3. Issues
Issue #98 closed

No source for code

Alex Gaynor
created an issue

When running coverage html I'm getting a no source for code error, pointing at a file that doesn't exist, and never did exist. No idea what the cause is:

(shelfworthy)alex@alex-laptop:~/projects/eldarion/shelfworthy/shelfworthy$ coverage run ./manage.py test events
[snip]
................
----------------------------------------------------------------------
Ran 16 tests in 0.264s

OK
Destroying test database 'default'...
(shelfworthy)alex@alex-laptop:~/projects/eldarion/shelfworthy/shelfworthy$ coverage html --omit=/home/alex/.virtualenvs/
No source for code: '/home/alex/projects/eldarion/shelfworthy/shelfworthy/models.py'
(shelfworthy)alex@alex-laptop:~/projects/eldarion/shelfworthy/shelfworthy$ ls models.py
ls: cannot access models.py: No such file or directory
(shelfworthy)alex@alex-laptop:~/projects/eldarion/shelfworthy/shelfworthy$ coverage --version
Coverage.py, version 3.4.  http://nedbatchelder.com/code/coverage

Comments (21)

  1. Ned Batchelder repo owner

    Alex, this is indeed mysterious! Can you post (or email me) a tarball with a reproducible case? Barring that, can you show the output of "coverage debug sys" and "coverage debug data" ?

  2. Ned Batchelder repo owner

    Simplest things first: are you sure there's no models.pyc there?

    Do you have any models.py files in your tree that aren't represented in the debug data output? Maybe it is being mis-attributed to the shelfworthy directory.

    As another clue, we can list the actual 24 lines measured in that phantom file:

    $ python
    >>> import coverage
    >>> cov = coverage.coverage()
    >>> cov.data.read()
    >>> cov.data.line_data()['/home/alex/projects/eldarion/shelfworthy/shelfworthy/models.py']
    

    This will print a list of line numbers.

  3. Anonymous

    The identified line numbers are:

    [1, 2, 4, 6, 9, 10, 23, 26, 29, 30, 32, 33, 34, 36, 38, 40, 41, 43, 49, 56, 59, 62, 68, 75]
    

    Looking at the result of the data debug and comparing to the find -name "models.py" the only one unaccounted for (in my local dir, ignoring the venv) is totally empty. No pycs lingering in this dir.

  4. Alex Gaynor reporter

    Sigh, above anonymous was me.

    As another datapoint (possibly unrelated but I have no idea), when running the django-taggit (https://github.com/alex/django-taggit) tests, in the exact same fashion and generating html in the same way (same omit of the virtualenv) the resulting HTML still includes all my stuff in the virtualenv. Possibly a wholly unrelated thing, but I don't know.

  5. Ram Rachum

    I'm experiencing this bug also. For me it's in my open-source project and not a client project, so I can try to make a reproducible case. Will that help?

  6. Ram Rachum

    Here's the tarball for my project:

    https://github.com/cool-RR/GarlicSim/tarball/af5978a28bf7f86516e7ae5aed6adcd95e5340fc

    (This happens for me under Ubuntu 10.10, Python 2.6, nose 0.11.1, coverage 3.4.)

    How to reproduce:

    Add the equivalents of the following 3 directories to your PYTHONPATH:

    export PYTHONPATH="/home/coolrr/Desktop/GarlicSim/garlicsim:/home/coolrr/Desktop/GarlicSim/garlicsim_lib:/home/coolrr/Desktop/GarlicSim/garlicsim_wx:"

    Go to GarlicSim/tests/.

    Run nose:

    nosetests --exclude=unittest2 --with-coverage --cover-package=garlicsim,garlicsim_lib,garlicsim_wx

    Run coverage:

    coverage html

    You get:

    No source for code: '/home/coolrr/Desktop/GarlicSim/garlicsim/shared.py'

  7. Ned Batchelder repo owner

    Hmmm, I'm not seeing the same results. I get 4 test failures, because I have to install numpy and wx. In the meantime, I get this from "coverage debug data | grep shared". What do you get?

    ned.karmic ~/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/tests> coverage debug data | grep shared
    /home/ned/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/garlicsim/general_misc/address_tools/shared.py: 12 lines
    /home/ned/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/tests/test_asynchronous_crunching/test_shared.py: 3 lines
    /home/ned/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/tests/test_general_misc/test_cute_profile/test_shared.py: 5 lines
    /home/ned/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/tests/test_general_misc/test_sleek_refs/shared.py: 15 lines
    
  8. Ned Batchelder repo owner

    After installing numpy (wxpython doesn't play well with virtualenv), the errors changed, and now I get:

    ======================================================================
    ERROR: Failure: ImportError (No module named shared)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/ned/coverage/bug98/vpy/lib/python2.6/site-packages/nose/loader.py", line 382, in loadTestsFromName
        addr.filename, addr.module)
      File "/home/ned/coverage/bug98/vpy/lib/python2.6/site-packages/nose/importer.py", line 39, in importFromPath
        return self.importFromDir(dir_path, fqname)
      File "/home/ned/coverage/bug98/vpy/lib/python2.6/site-packages/nose/importer.py", line 86, in importFromDir
        mod = load_module(part_fqname, fh, filename, desc)
      File "/home/ned/coverage/bug98/cool-RR-GarlicSim-af5978a/garlicsim/tests/test_asynchronous_crunching/test_project.py", line 29, in <module>
        from .shared import MustachedThreadCruncher
    ImportError: No module named shared
    

    I don't understand how this worked for you: there is no shared.py in the tests/ directory, which I think is where it would have to be for this relative import to work. Perhaps there's a clue here, since my tests fail trying to find shared.py, and your "coverage html" fails trying to find shared.py...

  9. Ram Rachum

    I tried now and I got the same error as you, missing `shared.py` file. I don't know how this worked before.

    I'm currently working on a big release for the project, and I've been changing my code really rapidly. Mainly I'm writing a lot of tests, reorganizing them (for example I renamed the main `tests` folder to `test_garlicsim`), and checking coverage. Once in a few coverage runs I get this annoying bug, and now I got it again, for a different "file":

    No source for code: 'c:\\documents and settings\\user\\my documents\\python projects\\garlicsim\\garlicsim\\test_garlicsim\\settings.py'
    

    (Tarball is here: https://github.com/cool-RR/GarlicSim/tarball/b13f91b3c29a02b1a59bf5899a6a2ccee8c0f1a9 )

    I checked and I don't have `import settings` or ` .settings` in my test files. I don't know why this happens.

    One suspect I have is the current `test_garlicsim/shared.py` module. It has a function that does some dynamic importing. The function takes a Python package and imports all its sub-modules, using `garlicsim.general_misc.import_tools.import_all`. One of the submodules that some of these packages may have is called `settings`. You think this is what's causing this bug?

    (Also: I tried deleting all `.pyc` files in `test_garlicsim` and all its subfolders, but it didn't solve the problem.)

  10. Ram Rachum

    Also, in parallel to solving this problem, maybe it's possible to make `coverage` fail gracefully when it can't find a source file instead of messing up the whole html report?

  11. Ned Batchelder repo owner

    I'd like to get to the bottom of why coverage thinks these files exist when they don't, or vice-versa. But in the meantime, the -i switch to coverage html will tell it to ignore errors like these.

  12. Ned Batchelder repo owner

    Running your new tarball, I can reproduce your missing settings.py file. The output of "coverage debug data | grep settings.py" shows what's going on. It mentions 10 settings.py files that actually exist, then ends with:

    /tmp/temp_garlicsim_gWXgpH/_coin_flip/settings.py: 2 lines
    

    This is due to test_garlicsim/test_doc/test_tutorial_2/test.py, which creates a tempfile with prefix "temp_garlicsim_". The temp dir is removed at the end of the test, of course, so the file no longer exists for coverage ro use in reporting. The -i switch is the right way to deal with this situation.

  13. Ram Rachum

    Haha, that is quite hairy :)

    Though I don't think this is the end of this bug, since it has happened a few times, from before I wrote stuff using `tempdir`. If it'll come up again in something which doesn't involve `tempdir` I'll let you know.

    Thanks for diving in Ned.

  14. Bert JW Regeer

    I'm getting this error when using Chameleon and or Mako, both of which generate Python files as bytecode, which never exist or persist on disk anywhere. I'm about to push a branch to pylons/deform on Github that show cases the issue. For now I have silenced the error using -i on the coverage xml statements, but you can still see that it is raising an issue because it can't find the source.

    Here's a link to our travis: https://travis-ci.org/Pylons/deform/jobs/109753148

  15. Bert JW Regeer

    master: https://github.com/Pylons/deform

    clone deform
    cd deform
    pyenv .env
    .env/bin/pip install -U pip tox
    .env/bin/tox -e py2-cover,py3-cover,coverage
    

    See tox.ini for running the tests manually (basically coverage with nosetests).

    Basically the output for coverage will show:

    Name                                                       Stmts   Miss  Cover   Missing
    ----------------------------------------------------------------------------------------
    deform/__init__.py                                             8      0   100%   
    deform/checkbox_a057b4eebc1bfb8813482f509e95a97a.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/checkbox_a057b4eebc1bfb8813482f509e95a97a.py'.
    deform/checkbox_f2a1f6c7cb47a17c80c8e7e917364781.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/checkbox_f2a1f6c7cb47a17c80c8e7e917364781.py'.
    deform/compat.py                                              22      0   100%   
    deform/dateinput_8df481e144e7a9a9de22a1c83128e19a.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/dateinput_8df481e144e7a9a9de22a1c83128e19a.py'.
    deform/dateinput_e7062cccc6c67342b670b229f2be577a.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/dateinput_e7062cccc6c67342b670b229f2be577a.py'.
    deform/decorator.py                                            9      0   100%   
    deform/exception.py                                           10      0   100%   
    deform/field.py                                              238      0   100%   
    deform/form.py                                                40      0   100%   
    deform/form_310a8848ef71e85b84761891d6060a56.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/form_310a8848ef71e85b84761891d6060a56.py'.
    deform/form_6f0eb973509007b31ab85f2009dd47f5.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/form_6f0eb973509007b31ab85f2009dd47f5.py'.
    deform/i18n.py                                                 2      0   100%   
    deform/interfaces.py                                           6      0   100%   
    deform/mapping_2b28fa02277e71d2d47404207e3d8a7b.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/mapping_2b28fa02277e71d2d47404207e3d8a7b.py'.
    deform/mapping_fb00f3c79a4366f5570d60136035ce70.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/mapping_fb00f3c79a4366f5570d60136035ce70.py'.
    deform/mapping_item_91ab68fe6f60ba908abab2daf41784ee.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/mapping_item_91ab68fe6f60ba908abab2daf41784ee.py'.
    deform/mapping_item_e333eec4a297e648dbc0997e803ef00c.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/mapping_item_e333eec4a297e648dbc0997e803ef00c.py'.
    deform/password_0856a24544ef14d7f11d118f5bf96010.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/password_0856a24544ef14d7f11d118f5bf96010.py'.
    deform/password_f63f3d723a8db8544752894f76656485.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/password_f63f3d723a8db8544752894f76656485.py'.
    deform/radio_choice_559e7e9a63b9bd7355843d880b969722.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/radio_choice_559e7e9a63b9bd7355843d880b969722.py'.
    deform/radio_choice_cd87d98b5eec0cf128225db63526fbd8.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/radio_choice_cd87d98b5eec0cf128225db63526fbd8.py'.
    deform/schema.py                                              23      0   100%   
    deform/sequence_6c7d63acbfe1c2112ee8084a475e1a05.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/sequence_6c7d63acbfe1c2112ee8084a475e1a05.py'.
    deform/sequence_ff7ed164938ea4fd6a8112780e6bd73f.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/sequence_ff7ed164938ea4fd6a8112780e6bd73f.py'.
    deform/sequence_item_46a061cdf2d04cc8842b7f48657f6723.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/sequence_item_46a061cdf2d04cc8842b7f48657f6723.py'.
    deform/sequence_item_72ce703fb3c8ad17f4e26c3d5f2fda8d.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/sequence_item_72ce703fb3c8ad17f4e26c3d5f2fda8d.py'.
    deform/template.py                                            31      0   100%   
    deform/test_2cf38d7892fd68f05df3d44b0281d9ba.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/test_2cf38d7892fd68f05df3d44b0281d9ba.py'.
    deform/test_caeed92ee490d5804fcd5c7e99cae85d.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/test_caeed92ee490d5804fcd5c7e99cae85d.py'.
    deform/tests/__init__.py                                       0      0   100%   
    deform/tests/test_api.py                                      12      0   100%   
    deform/tests/test_decorator.py                                19      0   100%   
    deform/tests/test_exception.py                                21      0   100%   
    deform/tests/test_field.py                                   594      0   100%   
    deform/tests/test_form.py                                    133      0   100%   
    deform/tests/test_functional.py                              171      0   100%   
    deform/tests/test_interfaces.py                               12      0   100%   
    deform/tests/test_schema.py                                   78      0   100%   
    deform/tests/test_template.py                                103      0   100%   
    deform/tests/test_widget.py                                 1867      0   100%   
    deform/textinput_19547a94fd98e871f4db67ed1c83685b.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/textinput_19547a94fd98e871f4db67ed1c83685b.py'.
    deform/textinput_27c770a62b06ddc01056faf6cde683ac.py   NoSource: No source for code: '/Users/xistence/Projects/deform/deform/textinput_27c770a62b06ddc01056faf6cde683ac.py'.
    deform/widget.py                                             766      0   100%   
    ----------------------------------------------------------------------------------------
    TOTAL                                                       4165      0   100%
    
  16. Ned Batchelder repo owner

    Bert JW Regeer I'm not sure what coverage can do automatically here. Chameleon is compiling a code object, and claiming that its filename is "deform/textinput_19547a94fd98e871f4db67ed1c83685b.py" (for example). That name is misleading and useless. Better would be for Chameleon to give it a name that makes clear what it is, like "<Template compiled from 'deform/textinput.pt'>" or something.

    You can ignore these files with a simple -i switch, or you can try to omit them with a more elaborate pattern in the configuration file.

    Another option would be to write a very simple Coverage.py plugin that would claim these files, and indicate not to trace them.

  17. Log in to comment