HTML reporting fails if `cover` directory doesn't exist

Issue #376 resolved
Caleb Evans
created an issue

Hi, Ned. First, I just want to say that I absolutely love your package, and I use it frequently for my personal projects. However, I have a strange issue which I'd like to report.

I'm working on some unit tests for a Python 3 project (run via nose), and I've been using the unittest.mock module to patch built-in functions (particularly those apart of the os module, as well as open). I am using setup and teardown functions to start and stop the patchers accordingly, and running nosetests yields passing tests with no errors.

However, when I try to view coverage by running nosetests --with-coverage with the --cover-html flag (which I prefer for inspecting partial branches), I receive a FileNotFoundError if the cover directory did not exist before running the command. Here's the full traceback:

  File "/Users/Caleb/.virtualenvs/my-project/bin/nosetests", line 11, in <module>
    sys.exit(run_exit())
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/core.py", line 121, in __init__
    **extra_args)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/core.py", line 207, in runTests
    result = self.testRunner.run(self.test)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/core.py", line 66, in run
    result.printErrors()
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/result.py", line 110, in printErrors
    self.config.plugins.report(self.stream)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/plugins/manager.py", line 99, in __call__
    return self.call(*arg, **kw)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/plugins/manager.py", line 167, in simple
    result = meth(*arg, **kw)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/nose/plugins/cover.py", line 196, in report
    self.coverInstance.html_report(modules, self.coverHtmlDir)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/coverage/control.py", line 662, in html_report
    return reporter.report(morfs)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/coverage/html.py", line 113, in report
    self.report_files(self.html_file, morfs, self.config.html_dir)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/coverage/report.py", line 84, in report_files
    report_fn(cu, self.coverage._analyze(cu))
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/coverage/html.py", line 257, in html_file
    self.write_html(html_path, html)
  File "/Users/Caleb/.virtualenvs/my-project/lib/python3.4/site-packages/coverage/html.py", line 143, in write_html
    fout = open(fname, "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'cover/src_local.html'

Do you understand what's going on here and why? Please let me know if I need to clarify or try something.

Thanks so much,
Caleb

Comments (9)

  1. Ned Batchelder repo owner

    @Caleb Evans This is odd. The html_report function will make the directory if need be. You are using the nose coverage plugin, which I don't support, I wonder if something else is going on in there? Is src_local.py a file in your project? Also, what version of coverage.py are you using?

  2. Caleb Evans reporter

    I'm using the latest version of coverage (3.7.1).

    Normally, when I run nosetests --with-coverage with the --cover-html flag, a cover directory is created at the root of my project directory. src_local.py is a generated file which maps to a module I am testing with coverage (src.local). I've attached a screenshot of my project hierarchy for your convenience.

    For what it's worth, I've also been able to work around the issue by creating the directory if it doesn't exist before running nosetests --with-coverage (I run tests and coverage using invoke, so I just added that logic to my tasks.py file).

    hierarchy.png

  3. Ned Batchelder repo owner

    @Caleb Evans I don't understand. First you said that it fails if you don't have a cover directory, and now you say that it usually doesn't fail. I don't understand what the difference is. Can you provide a reproducible test case? A pointer to your repo, and precise instructions for making it fail?

  4. Caleb Evans reporter

    My apologies—I know it's a strange issue, though I contend it's ultimately my fault (see below). However, I believe I know why and what the fix would be.

    Anyway, here are the exact steps to reproducing the issue on my one repo where the issue occurs:

    git clone https://github.com/caleb531/ssh-wp-backup.git
    cd ssh-wp-backup
    git checkout 01104ffdc2b281881d803a2e8083631d17d06047
    pip3 install -r requirements.txt
    nosetests --with-coverage --cover-html
    

    However, after searching through the code for coverage, I think I see what the issue is. As I mentioned previously, I'm using unittest.mock to patch several functions apart of the os module, one of which is makedirs. Furthermore, I see that on line 80 of report.py, you are using the os.makedirs function.

    Therefore, my theory is this: coverage runs report_files (and ultimately os.makedirs) while the patcher thereof is still active, thus causing the mock os.makedirs function (which is a noop) to be run instead. The fix (should you choose to fix it; I honestly wouldn't blame you if you didn't bother) would be to save a reference to the original os.makedirs function and call that when necessary.

  5. Ned Batchelder repo owner

    Ah, I see. I don't know much about your testing code, but I see that you create the mock in before_all, and then reset it in after_each. Perhaps if you created it in before_each, then the effects wouldn't carry over into the reporting phase. Another option is to not use the nose --cover plugin, and instead use two coverage commands:

        $ coverage run -m nose ....
        $ coverage html
    
  6. Caleb Evans reporter

    Running those two coverage commands works for me, actually, so it does seem to be a nose issue. However, I very much like the inline coverage table that nose outputs, so I will stick with using nosetests for interacting with coverage. Needless to say, outstanding job on the package, and I will keep those two commands in mind if I encounter any more issues in the future.

    I'll close the issue now since it's not actually an issue with coverage. Thanks for your help and guidance, Ned.
    Caleb

  7. Log in to comment