Add option to use wheel when installing packages

Issue #106 on hold
Rotem Yaari created an issue

The new pip 1.4 (and virtualenv 1.10) support the wheel package format, and can automatically cache downloaded and built packages in a custom location. This would be extremely useful for tox, as rerunning it multiple times will require no extra build time...

Comments (15)

  1. Rotem Yaari reporter

    I believe the greatest benefit would come if running "pip wheel --wheel-dir=X --use-wheel ." on the current project dir.

    This would package the current module, as well as any dependency, to the specified "X" directory (~/.toxcache, or a similar name). Then the installation of the package in the virtualenv can be done with "--use-wheel --find-links=X", which can speed up installation significantly. A good example for this that I've run into is packages depending on lxml, pyzmq or sqlalchemy, requiring some heavy compilation which takes a lot of time and CPU... Wheel speeds up these cases to mere seconds.

  2. Holger Krekel repo owner

    Needs some experimentation but sounds interesting. Any chance that you could tweak tox yourself and see how well it works? There already is a mechanism for detecting if an venv needs to be re-created, so i guess it's not too hard to try adding the options, and then to try it out on some real projects.

  3. Bogdan Hodorog

    I do think it would be useful to extend the proposed behavior by having a option (globally and maybe also override-able for each environment) which would allow the user to specify it's own options to the "core" pip install command which installs the project's sdist.

    I've just been hit by the need to have the package installed with pip1.4 with --pre (pre-development releases). Since others have similar needs (--wheel, I suspect other options to pip install would be usefull) I guess having an option in the ini file like:

    pip_install_options="any valid option specified by pip install --help command"

  4. Holger Krekel repo owner
    • changed status to open

    Actually, supporting wheel house should remain its own issue as it's not clear that issue35 will fully address this. @vmalloc and others: could you check how much PR61's "installer_command" support would help with implementing wheel-house support?

  5. Daniel Hahler

    Just for information and reference: you can make tox use wheels by configuring pip globally (for the user):

    download_cache = ~/.cache/pip/download
    wheel_dir = ~/.cache/pip/wheelhouse
    find_links =

    And to create the wheels:

    .tox/py27/bin/pip install wheel
    .tox/py27/bin/pip wheel -r test_requirements.txt
  6. Holger Krekel repo owner

    What about introducing a wheelhouse=yes and then configure the pip commands accordingly? The pip configuration would then use a per-tox dir configuration, not a per-user one. I am slightly skeptical about using per-user directories as this violates tox-run isolation.

  7. Daniel Hahler

    wheelhouse=yes would be good, and could be even used by default.

    However, pip might do this automatically in the future (creating wheels in a cache and installing it from there). But that's against the idea of isolation then again.

  8. Ronny Pfannschmidt

    as far as i understood it would be suficient if to would install a cached suffuciently recent pip version, pip 7.0 will go as far as building and caching wheels locally of everything

  9. Marcel Martin

    You can solve the problem in two ways; both make sure that pip 7.0 or later is installed in the virtualenvs that tox creates before it runs within them.

    The first option is to install tox into a virtual environment and run it from there. The installation will pull in a recent version of virtualenv, which in turn comes with a recent version of pip.

    The second option is to add pip and wheel to the [testenv] section in tox.ini:

    deps =

    All packages listed there will be installed with the old pip version, but then the is run with the new one.

    It seems to me that no changes to tox itself are really required (anymore). As soon as the distributions come with more recent virtualenv versions, this will have solved itself. And if you don’t use a distribution package, then it already works!

  10. Alain Péteut

    Something like a pre_deps would be helpful, so wheel would be in place when pip installs deps.

    pre_deps =
    deps = -rrequirements.txt
  11. Log in to comment