importlib.import_module() does not trigger coverage (replacing imp)

Issue #599 invalid
mscuthbert
created an issue

Hello -- I saw a dramatic drop in coverage (using the coverage.Coverage object) on a system when I recently moved from imp to importlib and it turns out the the lines which are executed when the module loads were not being covered. I think that this is similar to https://groups.google.com/forum/#!topic/nose-users/6Y7P4hfhJSc but that thread did not lead to a solution as those with the problem found a way around it.

Here I will import a module (one file on disk) in four different ways and show the results. Ideally, of course, we'd do "import music21", but because of how my test system is working with multiple modules, that won't work.

1 imp -- works

import imp, coverage
cov = coverage.Coverage()
cov.start()
imp.load_source('music21', '/Users/cuthbert/git/music21base/music21/__init__.py')
*<module 'music21' from '/Users/cuthbert/git/music21base/music21/__init__.py'>*
cov.stop()
cov.report()
* Name                                                  Stmts   Miss  Cover*
*-------------------------------------------------------------------------*
*/Users/cuthbert/git/music21base/music21/__init__.py      23      1    96%*
*95.65217391304348*

works great, but imp is deprecated.

2 importlib -- preferred new Python way

import importlib, coverage
cov = coverage.Coverage()
cov.start()
importlib.import_module('music21')
*<module 'music21' from '/Users/cuthbert/git/music21base/music21/__init__.py'>*
cov.stop()
cov.report()
*Coverage.py warning: No data was collected. (no-data-collected)*
*CoverageException: No data to report*

3 _ import _

cov = coverage.Coverage()
cov.start()
__import__('music21')
*<module 'music21' from '/Users/cuthbert/git/music21base/music21/__init__.py'>*
cov.stop()
cov.report()
*Coverage.py warning: No data was collected. (no-data-collected)*
*CoverageException: No data to report*

4 exec (just for fun; didn't expect it to work)

cov = coverage.Coverage()
cov.start()
exec('import music21')
cov.stop()
cov.report()
*Coverage.py warning: No data was collected. (no-data-collected)*
*CoverageException: No data to report*

Also tried

importlib.util.module_from_spec(importlib.util.find_spec('music21'))

but same result as above.

Tried on three different modules so I don't think that the module has something to do with it.

Thanks!

Comments (7)

  1. Ned Batchelder repo owner

    Thanks for the detailed report. I'll look into this, but it may take some time before I can get to it. Will these examples work with simpler code that your full repo? Having reproducible test cases is really great.

  2. mscuthbert reporter

    Sorry about the slow response -- my notifications were sent to the wrong email. I think I've reduced it to the minimum (or close thereto) necessary to reproduce the problem. There are two files i'll attach next: testCoverageImportlib.py and init.py which goes in a directory at the same level as testCoverageImportlib.py called "testCoverage" (so in a temp dir, there should be testCoverageImportlib.py and a testCoverage/init.py).

    When running "python3 testCoverageImportlib.py importlib" (or "imp" or "exec" or "underscore_import") it works fine -- all except "spec", so it looks like no problem. And also opening up a Python prompt and typing:

    import testCoverageImportlib testCoverageImportlib.main('importlib')

    works great too! But here's what I've found:

    import testCoverageImportlib testCoverageImportlib.main('imp') testCoverageImportlib.main('importlib')

    causes no data to be reported for the second, while reversing the order:

    import testCoverageImportlib testCoverageImportlib.main('importlib') testCoverageImportlib.main('imp')

    works fine. I'm not mixing importlib and imp in my calls, but I'm guessing that by working through a collection of modules using 'importlib' on a module1 that then imports module2, by the time module2 is imported using importlib it's a problem. I'll try to investigate further soon. Thanks Ned -- your work is amazing!

  3. Ned Batchelder repo owner

    The difference here is in the behavior of imp.load_source and importlib.import_module. The imp.load_source docs (https://docs.python.org/2.7/library/imp.html#imp.load_source) say: "If the module was already initialized, it will be initialized again." So doing the second import with imp will run the code in the .py again. If you do the second import with importlib, the code will not be run again. So with "imp then importlib", the code isn't run the second time, so no data is collected. With "importlib then imp", the code is run the second time, and coverage can measure it.

  4. Log in to comment