Commits

Maciej Fijalkowski committed 0faca70 Draft

a draft

Comments (0)

Files changed (1)

blog/ecosystem.rst

+We're closing into 10 years of existance of the PyPy project and while we
+went a long way, I would like to write down few words where we're going and
+why we're not at the end of the road.
+
+PyPy came up as a relatively silly learning toy and used to be called
+"minimal python", which was "just" a Python implementation in Python.
+Later on, we added stuff. There was first an idea that you can have a
+restricted subset of Python, called RPython, then the idea that you can use
+whole-program transformation to do cool stuff like introduce a
+`just in time compiler`_ or a `garbage collector`_, without actually writing
+them by hand. There was also `EU research funding`_ (I didn't know
+Bang & Olufsen was a potential PyPy partner, did you?) along the way, which
+helped to explore a lot of research ideas, in the process converting the
+PyPy project into essentially a very cool scientific vehicle.
+
+Then later, after quite a bit of cleaning up and wrangling, we shifted our
+focus and released PyPy 1.4, which was advertised as having the JIT
+`stable enough to be production ready`_. This was almost exactly two years
+ago.
+
+Now, what it means "production ready" anyway? It mostly means "random
+crashes should be rare and will be treated seriously". Back then we
+run Python code pretty well and it has only improved since. Now what it
+does not mean is, your web service will run under PyPy as a drop in
+replacement without changes and it does not mean "here is a tutorial
+how to deploy fast web servers on PyPy". This seems to be point of many
+disappointments and I'm very sorry that we did not advertise it correctly.
+After all a lot of strength of Python is precisely in it's ecosystem,
+documentation and community.
+
+So what's you gonna do about it?
+================================
+
+It's simple. We need an ecosystem. Now for the most part, we don't need an
+ecosystem where you have different copies of packages that are destined
+to run on PyPy or on CPython. However, we identified the problem a while
+ago and released `cffi`_ which is probably the first sane way to
+call C from PyPy and I think the best API to call C from Python (CPython
+or PyPy) in general. It's much more minimalistic than ctypes,
+much easier to get going fast on PyPy than Cython and much better than
+writing anything in C, say using CPython C API. This brings us closer
+to being able to deploy a nice web app without too much trouble on top
+of PyPy and make it really fast. Things that are worth keeping track of:
+
+* Postgres adapter using cffi: `psycopg2cffi`_
+
+* `Sqlite3 using cffi`_ (to be integrated in PyPy)
+
+* `JSON encoder speedups`_ (with JSON decoder speedups in the pipeline)
+
+* `python web server comparison`_ (use tornado or cyclone on top of PyPy)
+
+* recent XML parsing speedups on top of PyPy (ElementTree is 5x faster what it
+  used to be, 2x behind cElementTree on CPython, I'll write a blog post
+  soon)
+
+After all, who knows, maybe someone will write a blog post how to deploy
+a fast PyPy web server?
+
+So, to summarize the current status of affairs - we're trying to do our
+best to provide a rock solid VM implementation that interprets Python.
+We did a relatively good job at that, with impressive speed results and
+we're consistently working to make it better. However, now we want **you**
+to come along and provide an ecosystem, or more precisely, make the current
+ecosystem support PyPy better. We're willing to offer insight, help and
+profiling. CFFI is just one of the tools that make it much much easier
+to support both CPython and PyPy in a fast fashion. Stay tuned for more.
+
+Cheers,
+fijal
+
+.. _`just in time compiler`: http://en.wikipedia.org/wiki/Just-in-time_compilation
+.. _`garbage collector`: http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)
+.. _`EU research funding`: http://mail.python.org/pipermail/pypy-dev/2003-June/000809.html
+.. _`stable enough to be production ready`: http://morepypy.blogspot.com/2010/11/pypy-14-ouroboros-in-practice.html
+.. _`cffi`: http://cffi.readthedocs.org/en/latest/
+.. _`psycopg2cffi`: http://pypi.python.org/pypi/psycopg2cffi/0.1.2
+.. _`Sqlite3 using cffi`:
+.. _`JSON encoder speedups`: http://morepypy.blogspot.com/2011/10/speeding-up-json-encoding-in-pypy.html
+.. _`python web server comparison`: http://casbon.me/what-will-pypy-do-for-your-website-benchmarki