Wiki

Clone wiki

compatibility / Contributing

Contributing

Please contribute to this wiki

To better serve the community, we need more entries and better information on the present ones. Help us to cover more software compatibility with PyPy in better detail.

If you're an author or user of any Python software, please try it with PyPy and contribute to this wiki if you can. Any information on success or failure, even for very specific use cases, is valuable.

The usual way to test for compatibility with PyPy is to run the software's test suite and compare with a run on top of CPython. However, not all test suites give good coverage, nor do all packages have a test suite. Often, something is slightly broken in the tests and easy to spot, but other times someone familiar with the code is needed to check for compatibility.

In many cases we have no way to test compatibility without learning, massaging and building something with the code. Even so, test coverage will be small unless we spend a lot of time building the test application. So it'd be great to collaborate with people that actually use each Python software listed here to make a fair assessment of compatibility. PyPy people together with users or authors will be able to more precisely diagnose any incompatibility and, in some cases, figure out what it would take to make any given piece of software to work.

How to contribute

Spreading the word about this wiki helps. Testing your software on top of PyPy and either reporting the results or asking for help making it compatible helps more.

Testing

The usual way to test for compatibility is to run the software's test suite. Another case is that you already use some application and want to try it with PyPy. Here's how to do either of them.

Installing PyPy

Download and install PyPy in the most convenient way for your system. Creating a virtualenv to run tests inside is recommended, but you can choose to normally install packages on your PyPy path or even run tests without installing any packages. To better control environment differences, it's better to have a CPython 2.7 installed or set up the same way your PyPy is going to be, so that you avoid picking up system libraries or stray modules with different versions from what you're supposedly testing.

Installing a software to test

Download and either install or prepare the software to run tests from its source/build folder. For both cases, you'll have to make any dependencies the software has available to it, which means installing it in either the virtualenv or PyPy path, or copying into a folder you'll temporarily add to PYTHONPATH.

Once you have the pre-requisites available, it's time to find the test suite its own testing dependencies. Some packages do not distribute their test suites with their downloads. You'll need to grab the source from its VCS to be able to run tests in this case.

Common testing dependencies include nose, pytest, mock and webtest. Figuring out which one is required might take reading setup.{py,cfg}, readmes and test code. It's also common to find a test runner script like tests/run_all.py or equivalent. You can make the testing dependencies available in the same way you did for general ones.

Some pure-Python packages and most of those containing C extensions need to be built to run tests, which can be done with:

pypy setup.py build

Depending on your install method, you might need this instead:

PYTHONPATH=/path/to/dep1:/path/to/dep2/ /full/path/to/bin/pypy setup.py build

Once you have all dependencies and the target package ready, it's time to run the tests. If you fail to build the package or its dependencies, see Failures.

Testing your existing software

If you already use virtualenv for your development, you're almost ready: just create one for PyPy with all your dependencies. Otherwise, try to create a matching environment with PyPy, making your dependencies available to it.

Install/initialize/copy your application into the PyPy environment (or point the existing application to PyPy).

Of course, running your test suite (if any) is interesting. However, the great contribution you can make is to test real cases and flex your application to see if it breaks. That allows us to have a better picture about compatibility than many test suites could give. If you feel like it, also check performance changes when running under PyPy, reporting speedups and regressions is a nice way to help.

If you fail to build your application or its dependencies, your tests fail or you find errors in operations that work correctly under CPython, see Failures.

Running tests

It's OK to test in PyPy without a CPython control. If all tests pass, there's nothing else to check, and you can add results to the wiki. If you get failures, errors or skips, before considering them to be real you should test with a CPython installed in the same way, to spot flaky/broken tests. However, it's fine to report without checking against CPython, as long as you make it explicit.

You should call the test runner, which can be a script in the tests directory or an external one line py.test or nosetests. Sometimes the test runner (or support scripts) will fail under PyPy due to spurious errors, e.g. PyCrypto's ___builtins__['dict'] or feedparser's lack of closing opened files. Fixes like this are a good addition to this wiki, please report them.

If you get hangs or crashes, commenting out the problem test makes getting a better picture of the compatibility issues possible (but remember to report the hang or crash!).

Failures

Build errors

Both C extensions and pure-Python packages might fail to build.

C extensions might use parts of the C-API not supported by PyPy. It might be a missing part of the public API, which should eventually be implemented by PyPy, or an implementation detail.

Python packages might fail to build due to PyPy bugs or reliance on implementation details (that is, a bug of the package instead). If that reliance is unrelated to the package's main focus, we can host a patch for it in this wiki. Otherwise, it might be unsupported until it's ported to PyPy's own implementation details.

Test errors

Tests that error out, fail or are skipped might signal true PyPy bugs or reliance on implementation details. After a test run you can inspect the failed tests and assess its validity. Here are some hints for spotting spurious errors:

  • Checks against exact error messages (when exception types match).
  • Relying on reference counting for e.g. closing files immediately
  • Reference counting tests (usually involve the gc module).
  • Tests depending on dict-order.
  • ...

Reporting the errors, even if spurious, is appreciated. We might forward spurious errors to the software developers so they are more cross- implementation compatible and we can fix real errors.

Some test might crash PyPy instead. This is always worth reporting, but if the crash involves ctypes or CPyExt it's less serious, as both are known to be relatively unsafe compared to most Python code.

Adding to the wiki

If you are interested in directly adding some software to the wiki, please do so. You can copy and paste another entry and modify it to get the format right, (or take a look at the template) for the Home page. You can also add entry pages to describe the tests you ran and the results you got.

If you have tested something and want it to appear here but don't want to edit the wiki, report your findings on #pypy on freenode and someone will put them in the wiki for you.

Also see the Discussion page, which has more detail on how things are done (so far) and planned additions (so far) to this wiki.

Updated