TOX DELETED ALL OF MY CODE

Issue #352 on hold
Andrew Tipton
created an issue

TL;DR: due to a (slightly) wrong tox invocation... tox punished me by deleting all of the files in my project.

Transcript of my shell session:

$ pwd
/home/andrew/tmp/dataplanet
$ ls -l
total 44
4 -rw-r--r-- 1 andrew andrew  311 Jul 27 10:47 bitbucket-pipelines.yml
4 drwxr-xr-x 2 andrew andrew 4096 Jul 27 10:47 docs
4 -rw-r--r-- 1 andrew andrew  167 Jul 27 10:47 MANIFEST.in
8 -rw-r--r-- 1 andrew andrew 4308 Jul 27 10:47 README
4 -rw-r--r-- 1 andrew andrew 1021 Jul 27 10:47 setup.cfg
4 -rw-r--r-- 1 andrew andrew 1564 Jul 27 10:47 setup.py
4 drwxr-xr-x 4 andrew andrew 4096 Jul 27 10:47 src
4 drwxr-xr-x 2 andrew andrew 4096 Jul 27 10:47 tests
4 -rw-r--r-- 1 andrew andrew  998 Jul 27 10:47 TODO
4 -rw-r--r-- 1 andrew andrew 1147 Jul 27 10:47 tox.ini
$ tox -e tests
GLOB sdist-make: /home/andrew/tmp/dataplanet/setup.py
tests create: /home/andrew/tmp/dataplanet
ERROR: invocation failed (exit code 1), logfile: /home/andrew/tmp/dataplanet/log/tests-0.log
ERROR: actionid: tests
msg: getenv
cmdargs: ['/usr/bin/python3', '-m', 'virtualenv', '--python', '/usr/bin/python3', 'dataplanet']
ERROR: InvocationError: /usr/bin/python3 -m virtualenv --python /usr/bin/python3 dataplanet (see /home/andrew/tmp/dataplanet/log/tests-0.log)
env: {... environment variables omitted ...}

Traceback (most recent call last):
  File "/home/andrew/.local/lib/python3.4/site-packages/virtualenv.py", line 840, in install_wheel
    from urlparse import urljoin
ImportError: No module named 'urlparse'

_________________________________________________________________ summary __________________________________________________________________
ERROR:   tests: InvocationError: /usr/bin/python3 -m virtualenv --python /usr/bin/python3 dataplanet (see /home/andrew/tmp/dataplanet/log/tests-0.log)                                                                                                                                  
andrew@ip-10-0-20-167:~/tmp/dataplanet>
total 0
$ pwd
/home/andrew/tmp/dataplanet
$ cd ../dataplanet
$ ls -l
total 16
4 drwxr-xr-x 2 andrew andrew 4096 Jul 27 10:39 bin
4 drwxr-xr-x 2 andrew andrew 4096 Jul 27 10:39 include
4 drwxr-xr-x 3 andrew andrew 4096 Jul 27 10:39 lib
4 drwxr-xr-x 2 andrew andrew 4096 Jul 27 10:39 log
$ pwd
/home/andrew/tmp/dataplanet

Here's a minimal tox.ini that will reproduce the problem:

[tox]
envlist = check, py34-tests


[testenv]
envdir=
    check:          {toxworkdir}/py
    py34:           {toxworkdir}/py34


skip_install=
    check:          true
    tests:          false


deps=
    check:          flake8
    tests:          pytest


commands=
    check:          flake8 --show-source {toxinidir}/src
    tests:          py.test --strict --doctest-modules {toxinidir}/tests

Did you see what happened there? I meant to run tox -e py34-tests, but instead ran tox -e tests. This somehow resulted in {toxworkdir} getting set to {toxinidir} (!!!) and as a result tox helpfully deleted my project's code and replaced it with a tox working directory....

THIS IS NOT ACCEPTABLE BEHAVIOUR.

In case you're wondering "why configure things in this particular manner?", my full configuration has additional environments e.g. py27-tests, py34-docs, py34-build, etc. and I want to speed things up by sharing the working directory between these different targets. (That is, I want exactly one toxworkdir per Python version.)

Comments (9)

  1. Pi Delport

    The problem here is that the tox.ini sets envdir to empty by default, which overrides the default value of {toxworkdir}/{envname}:

    envdir=
        check:          {toxworkdir}/py
        py34:           {toxworkdir}/py34
    

    I don't think there's anything that Tox can reasonably do about this misconfiguration except maybe raising an overridable warning if envdir is empty? This would mean breaking backward compatibility a bit, but it may be justifiable in this case?

  2. Andrew Tipton reporter

    My understanding was that if the "environment" didn't match either check or py34 then the envdir setting would remain the default.

    Note that the following does not work:

    envdir=
        {toxworkdir}/default
        check:  {toxworkdir}/py
        py34:   {toxworkdir}/py34
    

    Again, let me reiterate that UNDER NO CIRCUMSTANCES SHOULD TOX EVER DELETE MY PROJECT DIRECTORY. There is absolutely no reason for anyone to ever intentionally configure tox in such a manner, and it should therefore refuse to proceed if such a situation arises due to misconfiguration. Are we 100% clear on this particular aspect of the issue?

  3. Oliver Bestwalter

    I agree. This should never happen. This is basically the worst thing that can happen during development, especially because it makes it very probable that you lose hours of work because you are testing the changes locally and have not pushed them to the remote yet.

  4. Andrew Tipton reporter

    Oliver, thanks for the PR. I think it's a good idea to have a sanity check for this specific situation.

    I still believe that the tox.ini parsing needs to be fixed, however. Recall my original configuration:

    envdir=
        check:          {toxworkdir}/py
        py34:           {toxworkdir}/py34
    

    The "principle of least surprise" suggests that this snippet should be interpreted as "if the environment name contains check, set envdir={toxworkdir}/py; if the environment name contains py34, set envdir={toxworkdir}/py34; otherwise, leave envdir at its default value."

    It's stunningly non-obvious that the actual semantics are "set envdir to a blank value (i.e. remove the default value but do not replace it with anything else) if the environment name doesn't contain either check or py34."

  5. Oliver Bestwalter

    Hi Andrew,

    I have not much experience with the codebase and the backwards compatibility implications mentioned by @Pi Delport so I went for the easy way out, to make sure that it's simply not possible to delete the project dir, whatever happens, as I am sure nobody wants that.

  6. Oliver Bestwalter

    It's stunningly non-obvious that the actual semantics are "set envdir to a blank value (i.e. remove the default value but do not replace it with anything else) if the environment name doesn't contain either check or py34."

    I agree and I would venture a guess that this is a bug and not a feature.

  7. Log in to comment