test_farm fails in git mirror

Issue #610 invalid
Daniel Hahler
created an issue

On coverage-4.4.1 (but also master):

% tox -e py36 -- -n0 -x -k test_farm                                                                                                                                           (c) (m) 🐍?
py36 develop-inst-noop: /Vcs/coveragepy
py36 installed: apipkg==1.4,-e git://github.com/nedbat/coveragepy.git@1f174f76a6d285545dabd7523b31347f2ddd43aa#egg=coverage,covtestegg1==0.0.0,decorator==4.1.2,enum-compat==0.0.2,eventlet==0.21.0,execnet==1.4.1,flaky==3.3.0,gevent==1.2.1,greenlet==0.4.12,mock==2.0.0,pbr==3.1.1,py==1.4.34,PyContracts==1.7.15,pyparsing==2.2.0,pytest==3.0.7,pytest-warnings==0.2.0,pytest-xdist==1.16.0,six==1.11.0,unittest-mixins==1.3
py36 runtests: PYTHONHASHSEED='2699930373'
py36 runtests: commands[0] | python setup.py --quiet clean develop
warning: no previously-included files found matching 'ci/appveyor.token'
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching 'tests/eggsrc/dist'
no previously-included directories found matching 'tests/eggsrc/*.egg-info'
py36 runtests: commands[1] | python igor.py zip_mods install_egg remove_extension
py36 runtests: commands[2] | python igor.py test_with_tracer py -n0 -x -k test_farm
=== CPython 3.6.2 with Python tracer (.tox/py36/bin/python) ===
....
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
--Return--
…/Vcs/coveragepy/tests/test_farm.py:1: ResourceWarning: unclosed file <_io.BufferedReader name='/home/user/.pdbrc.py'>
  # Licensed under the Apache License: http://www.apache.org/licenses/LICENSE-2.0
> …/Vcs/coveragepy/tests/test_farm.py(155)copy()->None
-> pdb.set_trace()
(Pdb) exc
*** NameError: name 'exc' is not defined
(Pdb) l
150         pwd = os.getcwd()
151         try:
152             shutil.copytree(src, dst)
153         except Exception as exc:
154             import pdb
155  ->         pdb.set_trace()
156     
157     
158     def run(cmds, rundir="src", outfile=None):
159         """Run a list of commands.
160     
(Pdb) pwd
'…/Vcs/coveragepy/tests/farm/annotate'
(Pdb) dst
'out_multi'
(Pdb) src
'src'
(Pdb)

The exception is:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

filename = 'tests/farm/annotate/run_multi.py'

    @pytest.mark.parametrize("filename", TEST_FILES)
    def test_farm(filename):
        if env.JYTHON:
            # All of the farm tests use reporting, so skip them all.
            skip("Farm tests don't run on Jython")
>       FarmTestCase(filename).run_fully()

tests/test_farm.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_farm.py:136: in run_fully
    self()
tests/test_farm.py:127: in __call__
    execfile(self.runpy, glo)
tests/backtest.py:19: in execfile
    exec(compile(code, filename, 'exec'), globs)
run_multi.py:4: in <module>
    ???
tests/test_farm.py:150: in copy
    shutil.copytree(src, dst)
.tox/py36/lib/python3.6/shutil.py:315: in copytree
    os.makedirs(dst)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'out_multi', mode = 511, exist_ok = False

    def makedirs(name, mode=0o777, exist_ok=False):
        """makedirs(name [, mode=0o777][, exist_ok=False])

        Super-mkdir; create a leaf directory and all intermediate ones.  Works like
        mkdir, except that any intermediate path segment (not just the rightmost)
        will be created if it does not exist. If the target directory already
        exists, raise an OSError if exist_ok is False. Otherwise no exception is
        raised.  This is recursive.

        """
        head, tail = path.split(name)
        if not tail:
            head, tail = path.split(head)
        if head and tail and not path.exists(head):
            try:
                makedirs(head, mode, exist_ok)
            except FileExistsError:
                # Defeats race condition when another thread created the path
                pass
            cdir = curdir
            if isinstance(tail, bytes):
                cdir = bytes(curdir, 'ASCII')
            if tail == cdir:           # xxx/newdir/. exists if xxx/newdir exists
                return
        try:
>           mkdir(name, mode)
E           FileExistsError: [Errno 17] File exists: 'out_multi'

.tox/py36/lib/python3.6/os.py:220: FileExistsError

There are other related failures.

Does tox -e py36 work for you on the Git mirror?

Comments (5)

  1. Ned Batchelder repo owner

    I tried this after a make clean, and it worked fine:

    $ tox -e py36 -- -n0 -x -k test_farm
    py36 develop-inst-nodeps: /Users/ned/coverage/trunk
    py36 installed: apipkg==1.4,appdirs==1.4.3,-e hg+ssh://hg@bitbucket.org/ned/coveragepy@e9937fd58c198f6ab6efb08f299332655171d5a0#egg=coverage,covtestegg1==0.0.0,decorator==4.1.2,enum-compat==0.0.2,eventlet==0.21.0,execnet==1.4.1,flaky==3.4.0,gevent==1.2.2,greenlet==0.4.12,mock==2.0.0,packaging==16.8,pbr==3.1.1,py==1.4.34,PyContracts==1.7.15,pyparsing==2.2.0,pytest==3.2.2,pytest-forked==0.2,pytest-xdist==1.20.0,six==1.11.0,unittest-mixins==1.3
    py36 runtests: PYTHONHASHSEED='4162502473'
    py36 runtests: commands[0] | python setup.py --quiet clean develop
    no previously-included directories found matching 'doc/_build'
    no previously-included directories found matching 'tests/eggsrc/build'
    no previously-included directories found matching 'tests/eggsrc/dist'
    no previously-included directories found matching 'tests/eggsrc/*.egg-info'
    warning: no previously-included files matching '*.py[co]' found anywhere in distribution
    py36 runtests: commands[1] | python igor.py zip_mods install_egg remove_extension
    py36 runtests: commands[2] | python igor.py test_with_tracer py -n0 -x -k test_farm
    === CPython 3.6.3 with Python tracer (.tox/py36/bin/python) ===
    .......
    ===================================================================== 760 tests deselected =====================================================================
    7 passed, 760 deselected in 7.35 seconds
    py36 runtests: commands[3] | python setup.py --quiet build_ext --inplace
    py36 runtests: commands[4] | python igor.py test_with_tracer c -n0 -x -k test_farm
    === CPython 3.6.3 with C tracer (.tox/py36/bin/python) ===
    .......
    ===================================================================== 760 tests deselected =====================================================================
    7 passed, 760 deselected in 6.99 seconds
    ___________________________________________________________________________ summary ____________________________________________________________________________
      py36: commands succeeded
      congratulations :)
    
  2. Daniel Hahler reporter

    Thanks for looking into it.

    It works from a fresh clone now also for me.

    It seems like the out_multi dir was not cleaned for some reason (maybe even because of using pdb before).

    But it should somehow ensure that it does not fail setting up tests afterwards because of this directory being there already.

    To reproduce this you can create the dir yourself manually: mkdir ./tests/farm/annotate/out_multi, and then it will fail again. The same is true for at least ./tests/farm/run and ./tests/farm/annotate/run.

    Maybe there should be a clean at the beginning of run_multi.py already? And/or to prevent this from happening the clean at the end should get done always, e.g. via try … finally?

  3. Log in to comment