Issues

Issue #167 new

Add an option to run commands after virtualenv creation but before other steps

Mikhail Korobov
created an issue

Hi,

I first mentioned this idea at https://bitbucket.org/hpk42/tox/issue/42/dependencies-should-be-installed-one-at-a (it was shut down then). I think it deserves its own ticket :)

The problem is the following:

  1. Some packages require certain software to build. Most common case - a C/Cython extension that uses numpy or scipy C-API.
  2. One solution is to put this software to deps section. The issue is that it can be hard/slow to install numpy and scipy to virtualenv using pip. System packages is not a solution either because you may need different versions. Also, there could be chains like "my package needs A to build, and A needs B to build", so you can't put both A and B into deps.

As an alternative, I can use Anaconda to get numpy, scipy and similar packages to virtualenv. It is fast and works quite well. But you need to

  1. run 'pip install conda' and 'conda init' after virtualenv creation;
  2. then run 'conda install numpy scipy --yes';

Both these steps should happen before installation of the package being tested. I can't set install_commans to conda install {packages} because conda can't install local zip packages.

There are other possible uses of this: for example, packages may need C/C++ libraries to build, and one may fetch them and install to virtualenv.

So, I'd like to be able to run some commands after virtualenv creation but before installation of my package (and better before installation of the dependencies).

Comments (7)

  1. Mikhail Korobov reporter

    Another reason why using install_command is hard is that conda itself must be installed into virtualenv. I created a script that can be used as an install_command:

    #!/usr/bin/env python
    
    if __name__ == '__main__':
        import sys
        import os
        import subprocess
    
        for package in sys.argv[1:]:
            if os.path.isfile(package):
                cmd = "pip install " + package
            elif package == 'conda':
                cmd = "pip install conda; conda init"
            else:
                cmd = "conda install --yes " + package
            print(cmd)
            subprocess.call(cmd, shell=True)
    

    but it looks like a hack.

  2. Mikhail Korobov reporter

    At conda mailing list it was suggested to add direct conda support to tox - add an option to create conda environments instead of virtualenv environments. conda inside virtualenv actually doesn't always work well, there are issues with shared C libraries and other issues.

  3. Andy Hayden

    Using conda instead of virtual env would definitely be the preferred solution here. For one thing running tox --conda (or something) could just conda install the correct python version and "just work"... no more InterpreterNotFound errors.

  4. holger krekel repo owner

    For the record, i am open to support and/or maybe take advantage of conda with tox. I don't know too much about conda so would be glad to hear a proposal what would need to be added exactly, preferably in the form of a documentation PR that explains the new "tox conda" integration which could then become the eventual documentation for it.

  5. Log in to comment