tox must create the source distribution with the Python of the virtual environment

Issue #236 on hold
Victor Stinner created an issue

Hi,

I just got a bug in the Oslo Messaging project on requirements: a unit test ensures that requirements are installed, and this test fails. tox currently starts by building a source distribution using "python setup.py sdist". The problem is that in Oslo Messaging, requirements are different on Python 2 and Python 3: oslo.messaging.egg-info/requires.txt is different if you run "python2 setup.py sdist" or "python3 setup.py sdist".

I wrote a patch to always rebuild the source distribution with the Python of the virtual environment in subcommand_test(). I kept the first call to sdist() which may be redundant in some cases, but I don't know tox enough to decide when the first call can be skipped (I don't want to introduce a regression).

Victor

Comments (24)

  1. Holger Krekel repo owner

    One of the ideas behind tox is to create one sdist and then make sure it installs/works everywhere. And so far this model has worked for most people apparently (except for those where it hasn't and i never heart about it, of course).

    Rebuilding an sdist per-env is not a straight forward change i guess and certainly not what we want to do by default. One issue is that the "build" environment is often different from the "install environment" and tox really manages the install environments so far.

    That all being said, i think tox may grow more flexibility/configurability in building artefacts to be installed. I am more thinking of wheels though.

    Note that there is a --installpkg PATH option which lets tox skip building alltogether. With a little "tox-front" script you could probably do the building yourself and then call "tox --installpkg=... -e env1,env2" for the time being.

    I agree with Robert though that trying to have a single-sdist is probably best, though, as it avoids all hassles.

  2. Holger Krekel repo owner

    setup.py install depends on the dev-environment (e.g. files might be present which are not present in the built package) and it doesn't allow to get certainty that a particular artifact successfully works in all environments.

  3. Former user Account Deleted

    @rbtcollins and I have been working through https://review.openstack.org/181795. This adds support for environment markers to pbr. Once this lands, and Oslo Messaging is building with it and starts taking advantage of the env markers, we should have a happy situation where a single sdist with a single requires.txt will install a different set of dependencies on python 2 vs python 3.

  4. Barry Warsaw

    I don't think it would be unreasonable to perhaps support a 'build' environment which the tox.ini author could use to specify which Python and how their sdist/wheel is built.

  5. Holger Krekel repo owner

    supporting a build env makes some sense i guess. Maybe also even an option to build a wheel per python/platform. The former should not be too hard to implement if anyone wants to give it a go (i don't have this problem myself at the moment and am unlikely to work on it during my free time currently).

  6. Nick Coghlan

    I'm not sure if I'm hitting this same problem, or something else, but I'm having trouble getting tox to work while attempting to adapt release-monitoring.org to run under both Python 2 & 3: https://github.com/fedora-infra/anitya/pull/322

    Specifically, I get this in the setuptools output:

    $ cat anitya.egg-info/requires.txt
    alembic
    bunch
    dateutils
    docutils
    fedmsg
    flask
    flask-openid
    flask-wtf
    markupsafe
    python-openid; python_version < '3.0'
    python3-openid; python_version >= '3.0'
    sqlalchemy
    straight.plugin==1.4.0-post-1
    wtforms
    

    But when I run tox with the following config file:

    [tox]
    envlist = py26,py27,py35
    skip_missing_interpreters = True
    
    [testenv]
    commands = nosetests
    # The rpm bindings aren't available via PyPI
    sitepackages = True
    # Need to ensure all PyPI published dependencies are installed in the venv
    install_command = pip install --ignore-installed {opts} {packages}
    deps =
        nose
        vcrpy
    

    The "python-openid" and "python3-openid" libraries get installed on both versions with this arrangement, which leads to both of them failing: they collide with each other at the filesystem level so you end up with an unimportable mess.

    I can get either Python 2 or Python 3 to pass (but not both at the same time) by commenting out one of the OpenID dependency declarations.

    Perhaps I should refile this as a new RFE to support PEP 508 environment markers in tox, rather than as a bug report?

  7. Nick Coghlan

    Aha, I was able to get the tests passing under tox by way of the following workaround:

    [testenv]
    # tox doesn't appear to understand environment markers properly yet,
    # so we use conditional commands to work around that
    # Since recreating the environments is pretty fast, we also just force that
    # rather than trying to detect if we're in an already fixed venv
    # (without this, reused environments will fail on the 'uninstall' command)
    recreate = True
    commands =
        pip uninstall -y python-openid python3-openid
        py26,py27: pip install python-openid
        py35: pip install python3-openid
        nosetests
    

    This strongly suggests the problem is with tox's handling of environment markers that appear in install_requires, so I'll make that an explicit RFE.

  8. Nick Coghlan

    In regards to .tox/log, that just shows the output from the sdist build command (which is where the idea of doing cat anitya.egg-info/requires.txt to get the declared requirements list came from). I'm not sure if there's a way to get tox to also append the precise installation commands it's using to update the venv used for test execution.

  9. Nick Coghlan

    (Note: I haven't created the RFE yet, as it's still possible I'm missing something here and this is currently expected to work)

  10. RobertR

    I'm about 100% sure its not related to /tox/ handling environment markers. Tox doesn't install stuff itself - it passes it to pip and pip processes it.

    I want the tox log so that I can look for e.g. python-openid already being installed at the RPM layer, because you're using site packages. Actually seeing the log will tell us things like that that catting the requires.txt won't. Attaching all the tox log output should let me diagnose this pretty quickly.

    Possibilities that come to mind are: - the use of site packages with python-openid already installed - the use of RPM installed setuptools thats too old -

    When you see both installed, which version of python has its tests running ?

  11. Nick Coghlan

    Good point re: explicitly checking that I don't have these installed system wide.

    Unfortunately the implicit pip invocation output doesn't appear to be getting captured in the log. However, the explicit pip uninstall commands from the workaround are visible in the main output, and suggest that they're genuinely present in the venv:

    py27 runtests: commands[0] | pip uninstall -y python-openid python3-openid
    Uninstalling python-openid-2.2.5:
      Successfully uninstalled python-openid-2.2.5
    Uninstalling python3-openid-3.0.10:
      Successfully uninstalled python3-openid-3.0.10
    

    Even though I was only reminded of the need for system site access after working around the openid problem, I also checked with rpm -qa python-openid python3-openid, /usr/bin/python -c "import openid", and /usr/bin/python3 -c "import openid" to be 100% sure they weren't there.

    You did give me an idea, which was to try running pip directly from my Py3 venv on the generated requirements file, as well as using the system pip to check the Py2 behaviour:

    $ pip install -r anitya.egg-info/requires.txt
    Ignoring python-openid: markers "python_version < '3.0'" don't match your environment
    .... # Output noting everything else is already installed
    
    $ /usr/bin/pip install -r anitya.egg-info/requires.txt
    Ignoring python3-openid: markers "python_version >= '3.0'" don't match your environment
    ... # Ctrl-C'ed out to skip actually trying to install anything
    

    So as far as I can tell, what's coming out of the setuptools sdist build is right, but somehow the installation into the tox venv is going wrong. Perhaps there's something I could add to the custom install_command to try and debug that more precisely?

  12. RobertR

    .tox/log/tox-0.log would be the thing to attach. That is the key thing to allow diagnosing whether your issue is this issue or some other. Thats a little separate to fixing your issue :) - but eliminating the sdist step would be useful. Unless of course you have skip_sdist set (which I can't tell from the fragments of config you've supplied).

    I'm feeling a little frustrated here, debugging without the request information is way harder than it needs to be :/

    If I was debugging this locally, I'd start by looking closely at all the log files which definitely capture enough info:

    .tox/log/tox-.log .tox/py$N/log/py$N-.log

  13. Nick Coghlan

    Ah, I didn't know there were per-venv logs, which is why I was confused - I was only looking at .tox/log/tox-.log which doesn't contain anything except the sdist build step. I'll attach all the logs now :)

  14. Nick Coghlan

    Attached Python 3.5 tox logs (py35-2.log is the one that looks suspicious, as it collects and installs both python-openid and python3-openid, even though the environment markers are clearly listed during the collect stage)

    I'll try running pip directly against the anitya sdist mentioned in the log, and see how that behaves.

  15. Nick Coghlan

    Checking the pip versions in the venv's:

    $ /home/ncoghlan/fedoradevel/anitya/.tox/py27/bin/pip -V
    pip 8.1.2 from /home/ncoghlan/fedoradevel/anitya/.tox/py27/lib/python2.7/site-packages (python 2.7)
    
    $ /home/ncoghlan/fedoradevel/anitya/.tox/py35/bin/pip -V
    pip 8.1.2 from /home/ncoghlan/fedoradevel/anitya/.tox/py35/lib/python3.5/site-packages (python 3.5)
    
  16. Nick Coghlan

    OK, I can confirm I can reproduce tox's misbehaviour in the local venv with just pip: if I run pip install .tox/dist/anitya-0.9.1.zip pip tries to install python-openid into my Python 3.5 venv. However, if I first unzip the archive and then run pip install -r .tox/dist/anitya-0.9.1/anitya.egg-info/requires.txt it correctly prints the message about ignoring python3-openid because the environment marker doesn't match the current environment.

    So perhaps there's some path through the "local sdist install" logic in pip that's bypassing the check for environment markers?

    Also, should we close this and move the discussion over to pip's issue tracker? As near as I can tell based on the additional logs, tox is doing everything right when it comes to delegating environment marker handling to pip, so both my issue and the one Victor originally reported can be considered resolved at the tox level.

  17. Nick Coghlan

    @hpk42 As per the above discussion, I think this issue can be closed now. The build environment question should be largely addressed by PEP 518 and pyproject.toml, and it looks the environment marker processing is being correctly delegated to pip (it's just that pip may not be getting it right for local sdist installations)

  18. Log in to comment