1. Holger Krekel
  2. tox
  3. Issues

Issues

Issue #106 on hold

Add option to use wheel when installing packages

Rotem Yaari
created an issue

The new pip 1.4 (and virtualenv 1.10) support the wheel package format, and can automatically cache downloaded and built packages in a custom location. This would be extremely useful for tox, as rerunning it multiple times will require no extra build time...

Comments (15)

  1. Rotem Yaari reporter

    I believe the greatest benefit would come if running "pip wheel --wheel-dir=X --use-wheel ." on the current project dir.

    This would package the current module, as well as any dependency, to the specified "X" directory (~/.toxcache, or a similar name). Then the installation of the package in the virtualenv can be done with "--use-wheel --find-links=X", which can speed up installation significantly. A good example for this that I've run into is packages depending on lxml, pyzmq or sqlalchemy, requiring some heavy compilation which takes a lot of time and CPU... Wheel speeds up these cases to mere seconds.

  2. Holger Krekel repo owner

    Needs some experimentation but sounds interesting. Any chance that you could tweak tox yourself and see how well it works? There already is a mechanism for detecting if an venv needs to be re-created, so i guess it's not too hard to try adding the options, and then to try it out on some real projects.

  3. Bogdan Hodorog

    I do think it would be useful to extend the proposed behavior by having a option (globally and maybe also override-able for each environment) which would allow the user to specify it's own options to the "core" pip install command which installs the project's sdist.

    I've just been hit by the need to have the package installed with pip1.4 with --pre (pre-development releases). Since others have similar needs (--wheel, I suspect other options to pip install would be usefull) I guess having an option in the ini file like:

    pip_install_options="any valid option specified by pip install --help command"

  4. Holger Krekel repo owner
    • changed status to open

    Actually, supporting wheel house should remain its own issue as it's not clear that issue35 will fully address this. Rotem Yaari and others: could you check how much PR61's "installer_command" support would help with implementing wheel-house support?

  5. Daniel Hahler

    Just for information and reference: you can make tox use wheels by configuring pip globally (for the user):

    [global]
    download_cache = ~/.cache/pip/download
    wheel_dir = ~/.cache/pip/wheelhouse
    find_links =
        /home/user/.cache/pip/wheelhouse
        /home/user/.cache/pip/download
    

    And to create the wheels:

    .tox/py27/bin/pip install wheel
    .tox/py27/bin/pip wheel -r test_requirements.txt
    
  6. Holger Krekel repo owner

    What about introducing a wheelhouse=yes and then configure the pip commands accordingly? The pip configuration would then use a per-tox dir configuration, not a per-user one. I am slightly skeptical about using per-user directories as this violates tox-run isolation.

  7. Daniel Hahler

    wheelhouse=yes would be good, and could be even used by default.

    However, pip might do this automatically in the future (creating wheels in a cache and installing it from there). But that's against the idea of isolation then again.

  8. Marcel Martin

    You can solve the problem in two ways; both make sure that pip 7.0 or later is installed in the virtualenvs that tox creates before it runs setup.py within them.

    The first option is to install tox into a virtual environment and run it from there. The installation will pull in a recent version of virtualenv, which in turn comes with a recent version of pip.

    The second option is to add pip and wheel to the [testenv] section in tox.ini:

    [testenv]
    deps =
        pip>=7.0.0
        wheel
    

    All packages listed there will be installed with the old pip version, but then the setup.py is run with the new one.

    It seems to me that no changes to tox itself are really required (anymore). As soon as the distributions come with more recent virtualenv versions, this will have solved itself. And if you don’t use a distribution package, then it already works!

  9. Log in to comment