With pip: project's sdist package is only installed in testenv during second test run

Issue #131 on hold
jenisys created an issue

VERSION INFO: tox 1.6.1, pip 1.4.1, virtualenv 0.10.1 (python 2.7)

When pip is used, the actual sdist package seems not to be installed when the test environment is created. When you run tox again on the same test environment it is installed and available (for example together with its console scripts).

NOTE: The required packages, that project depends on, are installed during the "sdist" installation step (during the first step).

EXAMPLE: Create a new test environment in TOX's "tox.ini" file:

# -- TOX tox.ini (extended with xxx environment)
changedir = {envdir}
    tox --version

Then run this test environment:

shell> tox -e xxx -r
GLOB sdist-make: .../tox/setup.py
xxx recreate: .../tox/.tox/xxx
xxx inst: .../tox/.tox/dist/tox-1.6.2.dev2.zip
xxx runtests: commands[0] | tox --version
WARNING:test command found but not installed in testenv
  cmd: /Library/Frameworks/Python.framework/Versions/2.7/bin/tox
  env: .../tox/.tox/xxx
Maybe forgot to specify a dependency?
1.6.1 imported from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tox-1.6.1-py2.7.egg/tox/__init__.pyc

==> OOPS: Initial tox is used, not the one under test.

shell> tox -e xxx
GLOB sdist-make: .../tox/setup.py
xxx inst-nodeps: .../tox/.tox/dist/tox-1.6.2.dev2.zip
xxx runtests: commands[0] | tox --version
1.6.2.dev2 imported from .../tox/.tox/xxx/lib/python2.7/site-packages/tox/__init__.pyc

IMPACT: This behavior is the killer if the console scripts of the python package should be used during the test run. Check also that after the first round (with recreate):

  • the tox sdist package is not under virtualenv site-package directory
  • "pip list" also states that package is not installed
  • the tox script is not in the .tox/xxx/bin directory

I am not sure if this is a pip problem or tox problem or a combination of both but it currently kills my use case.

WORKAROUND: Add a "python setup.py -q install" as first command to ensure that project package is installed. But then you install it twice :-(

Comments (13)

  1. jenisys reporter

    If found the cause and the real work-around. Explaining and writing it done seems to help the thought process sometimes ;-)

    When you add the following "install_command" everything works fine and as expected. The "-U" option is the important one.

    # -- TOX tox.ini
    install_command = pip install -U --pre {opts} {packages}
    changedir = {envdir}
        tox --version

    Now the test from above runs fine in both iterations.

  2. Holger Krekel repo owner

    Can not reproduce on trunk nor on 1.6.1. I simply get:

    (0)hpk@teta:~/p/tox$ tox -e xxx -r -v
    using tox.ini: /home/hpk/p/tox/tox.ini
    using tox-1.6.1 from /home/hpk/p/tox/tox/__init__.py
    GLOB sdist-make: /home/hpk/p/tox/setup.py
      /home/hpk/p/tox$ /home/hpk/venv/0/bin/python /home/hpk/p/tox/setup.py sdist --formats=zip --dist-dir /home/hpk/p/tox/.tox/dist >/home/hpk/p/tox/.tox/log/tox-0.log
    xxx recreate: /home/hpk/p/tox/.tox/xxx
      /home/hpk/p/tox/.tox$ /home/hpk/venv/0/bin/python /home/hpk/venv/0/local/lib/python2.7/site-packages/virtualenv.py --setuptools --python /home/hpk/venv/0/bin/python xxx >/home/hpk/p/tox/.tox/xxx/log/xxx-0.log
    xxx inst: /home/hpk/p/tox/.tox/dist/tox-1.6.1.zip
      /home/hpk/p/tox$ /home/hpk/p/tox/.tox/xxx/bin/pip install --pre /home/hpk/p/tox/.tox/dist/tox-1.6.1.zip >/home/hpk/p/tox/.tox/xxx/log/xxx-1.log
    xxx runtests: commands[0] | tox --version
      /home/hpk/p/tox/.tox/xxx$ /home/hpk/p/tox/.tox/xxx/bin/tox --version 
    1.6.1 imported from /home/hpk/p/tox/.tox/xxx/local/lib/python2.7/site-packages/tox/__init__.pyc
    __________________________________________ summary __________________________________________
      xxx: commands succeeded
      congratulations :)
  3. jenisys reporter

    Mmh, I uninstalled and reinstalled virtualenv and tox. But I can repeat it again and again. If I do not override the "install_command" and add the "-U" option, my package is not installed and not available under lib/pythonXY/site-packages". The same happens not only with my own package but also with "tox" as package (at least for me).

    Maybe my example does not work for you due to some PATH settings on your environment. Note that I did not perform "python setup.py develop" with the tox workspace.

    You can close this issue, if you think that is environment problem on my side. As long as nobody else runs into this issue ...

  4. Holger Krekel repo owner

    I do think that if this issue was commonly happening that the issue tracker would have heart of it or people would have asked on IRC. I don't suppose everybody is running "python setup.py develop" with a tox checkout.

    i don't want to close it just so. That being said, i don't currently have an idea of why this would happen, though. Can you do attach or link a full "tox -vv" output on the first run (where the package is not installed)?

  5. Holger Krekel repo owner

    And or maybe create a simple example project "abc" with a setup.py and see if it also fails there for you?

  6. jenisys reporter

    Mmh, I used the "parse" module as a simple test candidate and their it works, meaning the package is installed on the first try and pip lists it (tested w/ tox-1.6.1 and tox-1.6.2dev2).

    Differences: The "parse" module has no dependencies that need to be installed or … (something else).

    => I will check if I can find out what is causing this (in my environment). You either close this issue, reduce the priority/severity or ...

  7. Ralf Schmitt

    I've also ran into this.

    I'm testing the mwlib package. Under [testenv] I have deps=mwlib.rl among other things. mwlib.rl itself depends on mwlib and if tox runs without a .tox directory it install mwlib.rl, which pulls in mwlib from pypi.

    It then tries to install the sdist, but the initial pip command being used doesn't have -U and the result is that you're testing the old version.

    here's the complete tox.ini, it already uses the install_command workaround: https://github.com/pediapress/mwlib/blob/master/tox.ini

  8. Ralf Schmitt

    The version on pypi and the version in your local checkout have to be the same when trying to reproduce the bug. otherwise the sdist will get installed because it has a different version.

  9. Ralf Schmitt

    looking at the first message again, it looks like I maybe should have opened another bug report, but the bug's title describes my situation.

  10. Michael König

    Hi there!

    I essentially had the same problem as jenisys described. Here is what I found out after a few hours:

    Apparently, when tox creates a virtual environment for the first time, it tries to install your package with

    pip install package_name

    When tox is run for the second time, it finds an existing virtual environment. However, tox knows that your package probably will have changed, so it will try to upgrade it with the command

    pip install -U package_name

    When I ran

    pip freeze

    in the directory of my package, my package was actually listed as installed. When I ran the command in any other directory, my package was not installed - as I expected. This behavior was caused by the current directory being contained in the PYTHONPATH environment variable, for example like this:


    Whenever I ran tox, it used setup.py sdist to create a package of my code. In this moment, it placed an .egg-info directory in the folder which also contained the tox.ini file. When tox continued its execution in "recreate environment" mode, the just packaged module would be considered installed by pip, since the package is found in the faulty PYTHONPATH. Then, pip install package_name will not do anything, since the package is already installed.

    Now, if you change the directory by using the changedir command as in jenisys's example file, the package which was deemed installed can no longer be found.

    Consequent runs of tox would upgrade the "installed" package and actually install it properly in the virtual environment.

    To summarize: I had to fix my PYTHONPATH by removing the current directory. I did this just before running tox.

    Hope this helps!



  11. Vasily Kuznetsov

    I just ran into the same problem with tox 2.3.1. Indeed it turned out that I had PYTHONPATH set to . and after unsetting it the problem went away. Thanks Michael for the fix.

    I was wondering, perhaps it would make sense to make Tox always do pip install -U package_name to avoid this. It would do no harm to those that have sane PYTHONPATH and will protect the ones who don't.

  12. Log in to comment