Error - no module named virtualenv

Issue #360 on hold
Jason R. Coombs
created an issue

When using the advertised setuptools integration on a clean operating system that doesn't already have virtualenv installed, the tox run will fail when it attempts to invoke virtualenv:

py35 create: /vagrant/Dropbox/code/public/cherrypy/.tox/py35
ERROR: invocation failed (exit code 1), logfile: /vagrant/Dropbox/code/public/cherrypy/.tox/py35/log/py35-0.log
ERROR: actionid: py35
msg: getenv
cmdargs: ['/usr/bin/python3', '-m', 'virtualenv', '--python', '/usr/bin/python3.5', 'py35']
env: {'TERM': 'xterm-256color', 'HOME': '/home/vagrant', 'SHLVL': '1', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'MAIL': '/var/mail/vagrant', 'LOGNAME': 'vagrant', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'USER': 'vagrant', 'PWD': '/vagrant/p/cherrypy', 'SSH_CLIENT': '10.0.2.2 65337 22', 'LANG': 'en_US', 'SSH_CONNECTION': '10.0.2.2 65337 10.0.2.15 22', 'PYTHONHASHSEED': '2586869941', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'XDG_SESSION_ID': '9', '_': '/usr/bin/python3', 'SSH_TTY': '/dev/pts/0', 'PLAT': 'linux-x86_64', 'VIRTUAL_ENV': '/vagrant/Dropbox/code/public/cherrypy/.tox/py35', 'LANGUAGE': 'en_US:', 'PATH': '/vagrant/Dropbox/code/public/cherrypy/.tox/py35/bin:/home/vagrant/bin:/home/vagrant/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games', 'OLDPWD': '/home/vagrant', 'XDG_RUNTIME_DIR': '/run/user/900', 'SHELL': '/bin/bash'}

/usr/bin/python3: No module named virtualenv

ERROR: InvocationError: /usr/bin/python3 -m virtualenv --python /usr/bin/python3.5 py35 (see /vagrant/Dropbox/code/public/cherrypy/.tox/py35/log/py35-0.log)

I think the issue is obvious - tox is launching a subprocess to invoke virtualenv, but since virtualenv was loaded dynamically by the setuptools distutils command test, the subprocess doesn't have that library.

I've worked around this issue in the past (when using pytest-runner to test libraries whose test suite would launch subprocesses) by adding all of the items in sys.path to the environment variable PYTHONPATH when launching the subprocess.

I think there are a few options to address this issue.

  • Tox could rely on pyvenv on supported Python 3 versions, eliminating the dependency on virtualenv in those environments.
  • Tox could launch virtualenv in-process rather than as a subprocess.
  • Setuptools could implement the PYTHONPATH technique for all test commands, avoiding these issues in the general case.
  • Tox could implement the PYTHONPATH technique when launching virtualenv.
  • Tox could explicitly ensure that virtualenv will be on the path in the subprocess (knowing that it could have been resolved by the test command.
  • Tox could drop support for the distutils-bootstrapped technique, possibly preferring rwt as an alternate one-line bootstrapper for tox (though rwt itself currently requires bootstrapping).

There may be other options as well. I've listed these approaches in order by my preference.

What are the thoughts and preferences of the tox maintainers?

Comments (3)

  1. Holger Krekel repo owner

    What happens if we just make virtualenv a test requirement for the "setuptools test command integration"?

    Right now, tox kind of hard-depends on virtualenv and i think it's cleanest to express this. At the sprint in June we had talks about supporting venv and conda but it's some work obviously.

  2. Log in to comment