Issue #345 on hold
John Vandenberg
created an issue

PyPy3 5.2 is in alpha releases now, targeting Python 3.3.5 compatibility http://doc.pypy.org/en/latest/release-pypy3.3-v5.2-alpha1.html

It installs a pypy3.3 binary with a pypy3 symlink. It seems that vendors may be happy to replace their 'pypy3' package that is PyPy 2.4 https://packages.gentoo.org/packages/virtual/pypy3 https://bugzilla.redhat.com/show_bug.cgi?id=1353595

In which case, there is a strong argument that the tox.config.default_factors entry for pypy3 should continue to point to executable pypy3, which could be either PyPy 2.4 or PyPy 5.2 depending on which is in the PATH first.

However that means that pypy3 factor maps to either Python lang 3.2 or 3.3, which are two very different beasts. There will be many packages that can support the new PyPy3 5.2 , but do not intend to support the old pypy3, and they would normally describe that in tox.ini envlist by listing a default factor for each supported version.

And then what happens when PyPy releases a Python lang 3.4 compatible version. Is it also pypy3?

One alternative is to add pypy3x default factors, pointing at executables pypy3.x, which PyPy3 installs by default (and pypy2.7 for that matter).

Another is to detect PyPy language compatibility and map them to both factor py3x and factor pypy3, so that e.g. env py32-pypy3 and py33-pypy3 can be used if someone needs to distinguish between the two, and they will also fall back to a pypy3 env if it exists, or py32 or py33 if those factors are used in an env name.

Comments (6)

  1. Ryan Hiebert

    I'm in favor of adding pypyXX envs, for whatever variations are appropriate.

    Cross-matching factors seems like a bad idea, adding complexity to the configuration and to the implementation, as well as likely raising backward compatibility concerns. The semantic separation of implementation from python version is nice, but perhaps too costly. For instance, if we need to have py33-pypy3, it seems that should more properly be spelled py33-pypy (confusing for backward compatibility), and would also indicate that a CPython factor would be appropriate to specify as well py33-cpython. There's some upsides to the idea, but the trade-off doesn't strike me as providing the right balance.

  2. Danilo J. S. Bellini

    If the problem is the version confusion, probably that's not enough. Suppose I want to test on PyPy v2.6.1, v4.0.1 and v5.3.1, besides PyPy3 v2.4.0 and v5.2.0-alpha1 (the last versions as of today), using full names like pypy27_26, pypy27_40, pypy27_53, pypy32_24 and pypy33_52 would be nice and explicit. Or even pypy_2, pypy_4, pypy_5, pypy3_2 and pypy3_5 (the _ is just to avoid confusion, it can be any valid symbol). But, actually, AFAIK, there's not even a default distinct path for those, and distribution packages for different versions are likely to be incompatible. Unless PyPy includes executables named like pypy2-5.3 in their binary releases, or installs in full-named directories like /opt/pypy3-5.2.0-alpha1/bin/pypy3, it will be hard to fix that only relying on path and virtualenv. And I'm still not talking about the Software Transactional Memory release. Does anyone know if multiple pypy or pypy3 versions is possible in Windows, and where each version is installed there?

    There's something tox can do to detect the pypy version and see if it matches the entry version, it can use something like:

    from subprocess import Popen, PIPE
    import re
    pypyvinfo = Popen(["pypy3", "-V"], stderr=PIPE).stderr.read().decode("utf-8")
    pypyentry = "pypy{0[0]}{0[1]}_{1[0]}{1[1]}".format(*re.findall(r"Py\S*\s*(\d)\.(\d)", pypyvinfo))
    

    And the same with "pypy" or any other executable for PyPy. The pypyentry would be "pypy32_24" for PyPy3-2.4.0 and "pypy27_53" for PyPy2-5.3.1 (tested here). That would allow testing against multiple versions of PyPy and erroring/skipping for the wrong version. It would also be useful to add the specific Numpy version entry in the deps for each PyPy version. The problem is, where are those multiple PyPy versions installed? In Arch Linux, PyPy3-2.4.0 conflicts with the PyPy3 development version as installed by pypy3-hg AUR package.

  3. John Vandenberg reporter

    The pypy binary & source installer and virtualenv creates pypy3.2 for pypy3 2.4 (sys.version 3.2) and pypy3.3 for pypy3 5.2 . I suspect that pyenv does not do this (hence https://github.com/travis-ci/travis-ci/issues/6304). So it is already easy for tox to invoke the correct binary for new factors like pypy3_2 and pypy3_3.

    Like Arch, debian only provides pypy, and not pypy-2.7. (and ubuntu is the same )

    Fedora includes a /usr/lib/pypy-5.0.1/bin/pypy , and a /usr/lib64/pypy3-2.4.0/pypy3. I downloaded pypy3-5.2 from https://copr-be.cloud.fedoraproject.org/results/churchyard/pypy33/fedora-24-x86_64/00375596-pypy3/ , and it conflicts with the pypy3 package

    $ rpm -i --conflicts pypy3-5.2.0-0.1.alpha1.fc24.x86_64.rpm pypy3-libs-5.2.0-0.1.alpha1.fc24.x86_64.rpm 
    warning: pypy3-5.2.0-0.1.alpha1.fc24.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID 70a7f86f: NOKEY
        file /usr/share/doc/pypy3/README.rst from install of pypy3-libs-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-libs-2.4.0-6.fc24.x86_64
        file /usr/share/licenses/pypy3/LICENSE from install of pypy3-libs-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-libs-2.4.0-6.fc24.x86_64
        file /usr/share/doc/pypy3/README.rst from install of pypy3-libs-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-2.4.0-6.fc24.x86_64
        file /usr/share/licenses/pypy3/LICENSE from install of pypy3-libs-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-2.4.0-6.fc24.x86_64
        file /usr/share/doc/pypy3/README.rst from install of pypy3-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-libs-2.4.0-6.fc24.x86_64
        file /usr/share/licenses/pypy3/LICENSE from install of pypy3-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-libs-2.4.0-6.fc24.x86_64
        file /usr/bin/pypy3 from install of pypy3-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-2.4.0-6.fc24.x86_64
        file /usr/share/doc/pypy3/README.rst from install of pypy3-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-2.4.0-6.fc24.x86_64
        file /usr/share/licenses/pypy3/LICENSE from install of pypy3-5.2.0-0.1.alpha1.fc24.x86_64 conflicts with file from package pypy3-2.4.0-6.fc24.x86_64
    

    So I think it is fair to say that most people will only ever have one pypy2 and one pypy3, unless they install multiple versions outside of their package management system. It would be worth checking how the pypy Windows installers operate regarding multiple versions, but the pypy project has not released a Windows installer of pypy3 yet.

    Also, while pypy2-2.5/6.x may survive a little longer, I am extremely confident that pypy3-2.4 is walking dead and nobody will use/test on it as soon as a Windows installer for pypy3-5.2 exists and the first round of major Windows specific bugs are fixed.

    Note the STM pypy provides a binary pypy-stm, so tox could provide a stm factor and that problem is resolved, as there is only one version of it so far, but the stm binary release is completely not ready for prime time, so that new factor would only benefit the very few people who want to build STM from source, and they are unlikely to be releasing software that needs to be tested under pypy and pypy-stm via tox.

  4. Danilo J. S. Bellini

    All 3 version of PyPy are based on CPython 2.7, i.e. sys.version_info[:2] == (2, 7) there. The _ idea was exactly to be explicit about the PyPy version, not the CPython version it's supposed to be equivalent to, where pypyXY_ZW would reference this PyPy/PyPy3 version (calling pypy -V or pypy3 -V):

    Python X.Y.··· (···)
    [PyPy Z.W.··· with ···]
    

    Y and W can be optional, and the default XY would be "27", but Z is far more important than Y when we're talking about multiple PyPy versions.

    About PyPy3, as there's no stable release after its v2.4.0, tox shouldn't deprecate it. Actually, it should never deprecate PyPy3-2.4.0, as we're not talking about the interpreters in which tox itself should run, but the interpreters that tox should accept as valid ones for the testing environments, and I think tox should accept as many interpreters as possible, even the older ones.

  5. Log in to comment