tox doesn't install the package being tested

Issue #176 on hold
Mikhail Korobov created an issue

Hi,

Tox doesn't install the package being tested for me, seemingly because it thinks it is already installed.

The workaround is to add python setup.py install to commands, but this is suboptimal (dependencies can be processed twice, wheels are not installed, and this is not how tox used to work).

I'm using tox 1.7.1, virtualenv 1.11.6, pip 1.5.6 and setuptools 4.0.1. The OS is Mac OS X 10.9.3; the issue is reproducable with both Python 2.7.5 and python 3.4.1 virtualenvs (Pythons are installed using homebrew).

I tried to downgrade to virtualenv 1.11.2 and run tox -r -e py27, but it didn't help. Downgrading to tox 1.6.1 also didn't help.

After running tox own tests, .tox/py27/log/py27-2.log looks suspicious:

actionid=py27
msg=installpkg
cmdargs=[local('/Users/kmike/svn/tox/.tox/py27/bin/pip'), 'install', '--pre', '/Users/kmike/svn/tox/.tox/dist/tox-1.7.2.dev1.zip']
env={'PYTHONIOENCODING': 'utf_8', 'PROJECT_HOME': '/Users/kmike/dev', 'VCPROMPT_FORMAT': '[%b%m%u]', 'TERM_PROGRAM_VERSION': '326', 'TMPDIR': '/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/', 'LOGNAME': 'kmike', 'USER': 'kmike', 'HOME': '/Users/kmike', 'PATH': '/Users/kmike/svn/tox/.tox/py27/bin:/Users/kmike/envs/scraping/bin:/Users/kmike/.pythonbrew/bin:/Users/kmike/.pythonbrew/bin:/usr/local/bin:/usr/local/share/python:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin', 'PS1': '(scraping)\\[$grey\\]\\u \\[$reset\\]\\w \\[$green\\]$(vcprompt)\\[$reset\\]> ', 'DISPLAY': '/tmp/launch-AUcqwK/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'Apple_Terminal', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm-256color', 'SHELL': '/bin/bash', 'SHLVL': '1', 'SECURITYSESSIONID': '186a4', 'TEAMLOCAL_DEBUG': '1', 'SHIFTGIG_DEBUG': '1', 'PYTHONHASHSEED': '4220916990', 'EDITOR': 'nano', 'PYTHONPATH': '', 'WORKON_HOME': '/Users/kmike/envs', 'TERM_SESSION_ID': '19E53C2B-013A-429B-8867-A9E0A35EE1E7', 'CC': 'clang', 'FFLAGS': '-ff2c', 'SSH_AUTH_SOCK': '/tmp/launch-Jsdoay/Listeners', 'FAB_COMPLETION_CACHE_TASKS': 'true', 'Apple_PubSub_Socket_Render': '/tmp/launch-r2J2kt/Render', '_': '/Users/kmike/envs/scraping/bin/tox', 'VIRTUALENVWRAPPER_PROJECT_FILENAME': '.project', 'VIRTUAL_ENV': '/Users/kmike/envs/scraping', 'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/kmike/envs', 'CXX': 'clang++', 'OLDPWD': '/Users/kmike/svn', 'FAB_COMPLETION_CACHED_TASKS_FILENAME': '.fab_tasks~', '__CF_USER_TEXT_ENCODING': '0x1F7:0:0', 'PWD': '/Users/kmike/svn/tox', 'PIP_DOWNLOAD_CACHE': '/Users/kmike/.pip/download', '__CHECKFIX1436934': '1'}
Unpacking ./.tox/dist/tox-1.7.2.dev1.zip
  Running setup.py (path:/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pip-RHLGrc-build/setup.py) egg_info for package from file:///Users/kmike/svn/tox/.tox/dist/tox-1.7.2.dev1.zip

  Requirement already satisfied (use --upgrade to upgrade): tox==1.7.2.dev1 from file:///Users/kmike/svn/tox/.tox/dist/tox-1.7.2.dev1.zip in /Users/kmike/svn/tox
Downloading/unpacking virtualenv>=1.11.2 (from tox==1.7.2.dev1)
  Using download cache from /Users/kmike/.pip/download/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fv%2Fvirtualenv%2Fvirtualenv-1.11.6-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.17 in ./.tox/py27/lib/python2.7/site-packages (from tox==1.7.2.dev1)
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

The suspicious line is

Requirement already satisfied (use --upgrade to upgrade): tox==1.7.2.dev1 from file:///Users/kmike/svn/tox/.tox/dist/tox-1.7.2.dev1.zip in /Users/kmike/svn/tox

-- sdist is not installed because pip finds the same package in a local folder; this happen even when a new environment is created, so the package doesn't get installed into test virtuelanv at all.

and the tests fail:

> tox -e py27
GLOB sdist-make: /Users/kmike/svn/tox/setup.py
py27 recreate: /Users/kmike/svn/tox/.tox/py27
py27 installdeps: pytest>=2.3.5
py27 inst: /Users/kmike/svn/tox/.tox/dist/tox-1.7.2.dev1.zip
py27 runtests: PYTHONHASHSEED='4220916990'
py27 runtests: commands[0] | py.test --junitxml=/Users/kmike/svn/tox/.tox/py27/log/junit-py27.xml
===================================================================================================================== test session starts =====================================================================================================================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2
tox comes from: '/Users/kmike/svn/tox/tox/__init__.pyc'
collected 208 items 

tests/test_config.py .........................................................................................F................s
tests/test_interpreters.py s........
tests/test_quickstart.py .............
tests/test_result.py ...
tests/test_venv.py .................s....................
tests/test_z_cmdline.py .........F..F..........F..............

========================================================================================================================== FAILURES ===========================================================================================================================
_______________________________________________________________________________________________________________ TestCmdInvocation.test_version ________________________________________________________________________________________________________________

self = <test_config.TestCmdInvocation instance at 0x10cd5a248>, cmd = <tox._pytestplugin.Cmd instance at 0x10cd5a440>

    def test_version(self, cmd):
        result = cmd.run("tox", "--version")
        assert not result.ret
        stdout = result.stdout.str()
>       assert tox.__version__ in stdout
E       assert '1.7.2.dev1' in '1.7.1 imported from /Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/__init__.pyc'
E        +  where '1.7.2.dev1' = tox.__version__

/Users/kmike/svn/tox/tests/test_config.py:1159: AssertionError
----------------------------------------------------------------------------------------------------------------------- Captured stdout -----------------------------------------------------------------------------------------------------------------------
/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_defaults_changed_dir0/abc$ tox --version
1.7.1 imported from /Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/__init__.pyc
____________________________________________________________________________________________________________ test_run_custom_install_command_error ____________________________________________________________________________________________________________

cmd = <tox._pytestplugin.Cmd instance at 0x10cec83f8>, initproj = <function initproj at 0x10ced37d0>

    def test_run_custom_install_command_error(cmd, initproj):
        initproj("interp123-0.5", filedefs={
            'tox.ini': '''
                [testenv]
                install_command=./tox.ini {opts} {packages}
            '''
        })
        result = cmd.run("tox")
        result.stdout.fnmatch_lines([
>           "ERROR: invocation failed, args: ['*/tox.ini*",
        ])
E       assert "ERROR: invoc... ['*/tox.ini*" == 'python inst: ...rp123-0.5.zip'
E         - ERROR: invocation failed, args: ['*/tox.ini*
E         + python inst: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/.tox/dist/interp123-0.5.zip

/Users/kmike/svn/tox/tests/test_z_cmdline.py:203: AssertionError
----------------------------------------------------------------------------------------------------------------------- Captured stdout -----------------------------------------------------------------------------------------------------------------------
created project in /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123
/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123$ tox
GLOB sdist-make: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/setup.py
python create: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/.tox/python
python inst: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/.tox/dist/interp123-0.5.zip
nomatch: "ERROR: invocation failed, args: ['*/tox.ini*"
    and: u'GLOB sdist-make: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/setup.py'
    and: u'python create: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/.tox/python'
    and: u'python inst: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_run_custom_install_comman1/interp123/.tox/dist/interp123-0.5.zip'
----------------------------------------------------------------------------------------------------------------------- Captured stderr -----------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kmike/envs/scraping/bin/tox", line 9, in <module>
    load_entry_point('tox==1.7.1', 'console_scripts', 'tox')()
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 26, in main
    retcode = Session(config).runcommand()
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 303, in runcommand
    return self.subcommand_test()
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 446, in subcommand_test
    self.installpkg(venv, sdist_path)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 394, in installpkg
    venv.installpkg(sdist_path, action)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_venv.py", line 245, in installpkg
    self._install([sdistpath], extraopts=extraopts, action=action)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_venv.py", line 318, in _install
    action=action, extraenv=extraenv)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_venv.py", line 285, in run_install_command
    extraenv=env, action=action)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_venv.py", line 367, in _pcall
    return action.popen(args, cwd=cwd, env=env, redirect=redirect)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 97, in popen
    stdout=f, stderr=STDOUT)
  File "/Users/kmike/envs/scraping/lib/python2.7/site-packages/tox/_cmdline.py", line 155, in _popen
    stdout=stdout, stderr=stderr, env=env)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied
________________________________________________________________________________________________________________ test_skip_unknown_interpreter ________________________________________________________________________________________________________________

cmd = <tox._pytestplugin.Cmd instance at 0x10cd5c050>, initproj = <function initproj at 0x10ced3aa0>

    def test_skip_unknown_interpreter(cmd, initproj):
        initproj("interp123-0.5", filedefs={
            'tests': {'test_hello.py': "def test_hello(): pass"},
            'tox.ini': '''
                [testenv:python]
                basepython=xyz_unknown_interpreter
                [testenv]
                changedir=tests
            '''
        })
        result = cmd.run("tox", "--skip-missing-interpreters")
>       assert not result.ret
E       assert not 2
E        +  where 2 = <tox._pytestplugin.RunResult instance at 0x10cec8f80>.ret

/Users/kmike/svn/tox/tests/test_z_cmdline.py:256: AssertionError
----------------------------------------------------------------------------------------------------------------------- Captured stdout -----------------------------------------------------------------------------------------------------------------------
created project in /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_skip_unknown_interpreter0/interp123
/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_skip_unknown_interpreter0/interp123$ tox --skip-missing-interpreters
----------------------------------------------------------------------------------------------------------------------- Captured stderr -----------------------------------------------------------------------------------------------------------------------
usage: tox [-h] [--version] [-v] [--showconfig] [-l] [-c CONFIGFILE]
           [-e envlist] [--notest] [--sdistonly] [--installpkg PATH]
           [--develop] [--set-home] [-i URL] [-r] [--result-json PATH]
           [--hashseed SEED] [--force-dep REQ] [--sitepackages]
           [args [args ...]]
tox: error: unrecognized arguments: --skip-missing-interpreters
____________________________________________________________________________________________________________________ TestToxRun.test_json _____________________________________________________________________________________________________________________

self = <test_z_cmdline.TestToxRun instance at 0x10cc923f8>, cmd = <tox._pytestplugin.Cmd instance at 0x10cd5a2d8>, example123 = None

    def test_json(self, cmd, example123):
        # see that tests can also fail and retcode is correct
        testfile = py.path.local("tests").join("test_hello.py")
        assert testfile.check()
        testfile.write("def test_fail(): assert 0")
        jsonpath = cmd.tmpdir.join("res.json")
        result = cmd.run("tox", "--result-json", jsonpath)
        assert result.ret == 1
        data = json.load(jsonpath.open("r"))
>       verify_json_report_format(data)

/Users/kmike/svn/tox/tests/test_z_cmdline.py:445: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

data = {'host': 'imac.local', 'installpkg': {'basename': 'example123-0.5.zip', 'md5': 'b9c178528f5fd913a0a71b8cac044fbc', 'sh...: 'a48f1c0019c59af0e15abc28ce267a071f1cc1b5b2b28ec420724fe1370c38e3'}, 'platform': 'darwin', 'reportversion': '1', ...}
testenvs = True

    def verify_json_report_format(data, testenvs=True):
        assert data["reportversion"] == "1"
>       assert data["toxversion"] == tox.__version__
E       assert '1.7.1' == '1.7.2.dev1'
E         - 1.7.1
E         + 1.7.2.dev1

/Users/kmike/svn/tox/tests/test_z_cmdline.py:672: AssertionError
----------------------------------------------------------------------------------------------------------------------- Captured stdout -----------------------------------------------------------------------------------------------------------------------
created project in /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123
/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123$ tox --result-json /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/res.json
GLOB sdist-make: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/setup.py
python create: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python
python installdeps: pytest
python inst: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/dist/example123-0.5.zip
python runtests: PYTHONHASHSEED='322203576'
python runtests: commands[0] | py.test --basetemp=/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/tmp --junitxml=junit-python.xml
ERROR: invocation failed, logfile: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/log/python-3.log
ERROR: actionid=python
msg=runtests
cmdargs=[local('/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/bin/py.test'), '--basetemp=/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/tmp', '--junitxml=junit-python.xml']
env={'DISPLAY': '/tmp/launch-AUcqwK/org.macosforge.xquartz:0', 'SHLVL': '1', 'TEAMLOCAL_DEBUG': '1', 'CC': 'clang', 'TERM_PROGRAM_VERSION': '326', 'PS1': '(scraping)\\[$grey\\]\\u \\[$reset\\]\\w \\[$green\\]$(vcprompt)\\[$reset\\]> ', 'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/kmike/envs', '__CF_USER_TEXT_ENCODING': '0x1F7:0:0', 'CXX': 'clang++', 'LOGNAME': 'kmike', 'PYTHONHASHSEED': '322203576', 'TERM': 'xterm-256color', 'EDITOR': 'nano', 'TMPDIR': '/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/', 'VIRTUALENVWRAPPER_PROJECT_FILENAME': '.project', 'VIRTUAL_ENV': '/Users/kmike/envs/scraping', 'HOME': '/Users/kmike', 'PYTHONPATH': '/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123', 'FFLAGS': '-ff2c', 'TERM_PROGRAM': 'Apple_Terminal', 'SECURITYSESSIONID': '186a4', 'LANG': 'en_US.UTF-8', '__CHECKFIX1436934': '1', 'OLDPWD': '/Users/kmike/svn', 'Apple_PubSub_Socket_Render': '/tmp/launch-r2J2kt/Render', 'PIP_DOWNLOAD_CACHE': '/Users/kmike/.pip/download', 'FAB_COMPLETION_CACHED_TASKS_FILENAME': '.fab_tasks~', 'PATH': '/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/bin:/Users/kmike/svn/tox/.tox/py27/bin:/Users/kmike/envs/scraping/bin:/Users/kmike/.pythonbrew/bin:/Users/kmike/.pythonbrew/bin:/usr/local/bin:/usr/local/share/python:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin', 'FAB_COMPLETION_CACHE_TASKS': 'true', 'PROJECT_HOME': '/Users/kmike/dev', 'PWD': '/Users/kmike/svn/tox', 'SHELL': '/bin/bash', 'USER': 'kmike', 'TERM_SESSION_ID': '19E53C2B-013A-429B-8867-A9E0A35EE1E7', 'WORKON_HOME': '/Users/kmike/envs', 'VCPROMPT_FORMAT': '[%b%m%u]', 'SHIFTGIG_DEBUG': '1', '_': '/Users/kmike/envs/scraping/bin/tox', 'SSH_AUTH_SOCK': '/tmp/launch-Jsdoay/Listeners'}
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2
collected 1 items

test_hello.py F

=================================== FAILURES ===================================
__________________________________ test_fail ___________________________________

>   def test_fail(): assert 0
E   assert 0

test_hello.py:1: AssertionError
 generated xml file: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/tests/junit-python.xml 
=========================== 1 failed in 0.02 seconds ===========================

ERROR: InvocationError: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/bin/py.test --basetemp=/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/tmp --junitxml=junit-python.xml (see /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/example123/.tox/python/log/python-3.log)
___________________________________ summary ____________________________________
ERROR:   python: commands failed
wrote json report at: /private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gq/T/pytest-129/test_json0/res.json
-------------------------------------------------------------------------------------------- generated xml file: /Users/kmike/svn/tox/.tox/py27/log/junit-py27.xml --------------------------------------------------------------------------------------------
=================================================================================================================== short test summary info ===================================================================================================================
SKIP [2] /Users/kmike/svn/tox/.tox/py27/lib/python2.7/site-packages/_pytest/skipping.py:132: condition: sys.platform != 'win32'
SKIP [1] /Users/kmike/svn/tox/.tox/py27/lib/python2.7/site-packages/_pytest/skipping.py:132: condition: not sys.platform.startswith('linux')
FAIL tests/test_config.py::TestCmdInvocation::()::test_version
FAIL tests/test_z_cmdline.py::test_run_custom_install_command_error
FAIL tests/test_z_cmdline.py::test_skip_unknown_interpreter
FAIL tests/test_z_cmdline.py::TestToxRun::()::test_json
====================================================================================================== 4 failed, 201 passed, 3 skipped in 78.50 seconds =======================================================================================================
ERROR: InvocationError: '/Users/kmike/svn/tox/.tox/py27/bin/py.test --junitxml=/Users/kmike/svn/tox/.tox/py27/log/junit-py27.xml'
___________________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________________
ERROR:   py27: commands failed

Comments (7)

  1. Ryan Campbell

    I'm seeing what looks to be the exact same problem here.

    @kmike did you ever figure out a workaround for this?

  2. Shaun Crampton

    I just hit this and worked around by putting this in my [testenv] section of tox.ini:

    install_command =
        pip install -U {opts} {packages}
    
  3. Log in to comment