Pypy sprint and conference report
The Pypy project held a 4 day sprint at the Marvin Center, George
Washington University in Washington DC, USA on 19-22 March 2005.
The sprint was part of the pre-Pycon sprinting sessions and we shared
rooms with sprints for Distutils, Chandler, CPython AST, Twisted,
ZODB, Mailman, Zope and others. The environment was quite inspiring
and there was quite a bit of exchange between the different sprints.
Participants at the Pypy sprint were:
Michael Chermside <firstname.lastname@example.org>
Brian Dorsey <email@example.com>
Richard Emslie <firstname.lastname@example.org>
Alex Martelli <email@example.com>
Alan Mcintyre <firstname.lastname@example.org>
Lutz Pälike <email@example.com>
Jonathan Riehl <firstname.lastname@example.org>
Alan Mcintyre was only present the first day. Michael Chermside joined
the sprint Monday. All the others stayed for the whole sprint.
The sprint was started with an introduction to Pypy by Armin and
an orientation in the directory structure by Holger.
We then had a discussion about what to focus on and how to divide the work.
We decided to put our energies into making Pypy as compliant to the CPython
implementation as possible by trying to fulfil the CPython regression tests.
Christian and Alex worked on implementing Pickle, fixing a large
number of bugs and corner cases in the process. Samuele helped
Christian tracking a bunch of those.
Related to pickling failing tests, we discovered that our
type.__base__ implementation was too naive, Armin reworked it. Also
Samuele found out that some of our string operations because they were
dispatching on ANY ended up being executed by unicode code (unicode
being a better match than ANY) after an automatic conversion which
would fail for non-ascii characters. This has been fixed.
Anders and Richard worked on completing the builtin types, especially
focusing on the FrameType. In particular they enabled sys.settrace
and sys.setprofile functionality, also paying attention that
app-level helper frames are appropriately hidden. The design of how
to achieve the latter was discussed with Armin and Samuele.
Jacob found an already implemented version of the datetime module by talking
to Tim Peters. This required some adjustments before it was put into the
Pypy library. When we later got access to the CPython tests, it turned out
that the module needed some further fixes, which Christian applied.
Jacob then turned to the binascii module and was later joined by Brian
in this work. About half the module was implemented by the end of the sprint.
Holger and Brian initially worked on making the CPython tests work
under py.test. Samuele also paired a bit on this with Holger. Holger
then went on to assist Michael and Jonathan while Brian went to work
Holger alone and then pairing with Samuele reworked our tool to track
our builtins implementations such that there's a summary page and it
is run on codespeak after each check-in, the latest output can always
be seen at http://codespeak.net/pypy/rev/current/ .
Jonathan focussed on implementing a Python parser in Python.
Michael wanted to interface the sre module with Pypy, and got started
on the project.
Armin wrote an example object space that can wrap a given other space
and implements lazy objects and a become operation similar to the one
found in Smalltalk implementations.
[More stuff about what people did. I have no clue about what Alan,
Armin, Lutz or Samuele did. Also, what I have written above is
probably incomplete and in some cases downright wrong. This phappens
if you focus tightly on your own coding. Please add and correct.]
On the whole the sprint was very successful. We made great progress
on the compliancy issue. While there are still many modules that need
implementing, the builtin types are getting very close to being complete.
The missing types generally require interfacing to system calls, which
we are not yet able to handle.
Talk at the conference
Armin Rigo held a talk on the subject of Pypy and Type Inference It
was lively and animated (both abstractly and literally
speaking). Armin explained the concept of Object Spaces, first running
an interpreter with an Object Space that added integers, and then with
one that added fruit. He then went on to explain that you could run
the interpreter with any sort of Object Space; for instance one that
looks at the instructions and tries to deduce what types the variables
involved have. Another object space can do translation to lower level
languages (possibly using the annotations provided by a previous run
with the annotator).
Holger spoke about py.test and the py lib, which are intimately connected
to the Pypy project.
Other Pypy relevant talks at the conference were
Localized Type Inference in Python by Brett Cannon
Python on the .NET platform (IronPython) by Jim Hugunin
Decimal Module for Beginners by Michael Chermside
Decimal data type by Facundo Batista
Pulling Java Lucene into Python: PyLucene by Andi Vajda
OpenSpace discussion about how to handle non-CPython implementations of
Guido van Rossum (CPython)
Jim Hugunin (IronPython)
Brian Zimmer (Jython)
1. We need to modify the CPython test suite so that it makes a
difference between language compliance tests and CPython
implementation tests. Guido if fine with this, but wants a proposal
written and discussed on python-dev. Jim is busy the next 2 months and
will not start testing. Pypy will modify the tests for its own
purposes and will then propose things based on the experience of
making the modifications. Jython presently has a number of "if not
jython" defines in the test suite. Some of these are CPython
implementation tests, while some are compliance tests that Jython
2. There was a long discussion of how to handle non-portable platform
features and how to handle features that you don't know if you want or
not. It was agreed that "from __experimental__ import ..." should be
used for features that may go away. It was also agreed that it is a
good idea to use "from __jython__ import ..." when you want to
override standard builtin Python features with something that makes
better sense in a specific platform environment.
3. There was along discussion about where the sweetspot for a platform
specific implementation should be. It didn't rellay conclude anything
Pypy post-conference meeting
After the conference, we held a meeting to discuss the division of work for
the next few weeks.
We should go ahead with the Oxford sprint. Holger wants to do a "private"
sprint in Göteborg a week before Europython.
Armin - will evaluate different alternatives for the translator and write
Samuele - will work on finishing the annotator
Holger - will work on completing the test framework
Arre - will work on making the std object space fully CPython compiant
Christian - will work the C translator
Lutz - will work on a GUI for generating flow graphs
Jonathan - will continue on his parser
Jacob - will finish binascii