Alexis Metaireau avatar Alexis Metaireau committed 79681bd

Added a first version of the PEP.

Comments (0)

Files changed (1)

+Python Enhancement Proposal
+###########################
+
+This document propose a new RESTful API for PyPI, the Python Package Index,
+using CouchDB.
+
+Current APIs to access PyPI
+===========================
+
+Currently, there are two ways to retrieve data from the PyPI: using the simple
+API, and using the XML/Rpc interface, and both have drawbacks.
+
+The simple API
+--------------
+
+This is the API available at http://pypi.python.org/simple/. It provides HTML
+pages with informations on the distributions and the projects. No metadata is
+provided here, so when it comes to use the simple index for packaging purposes,
+we need to download all the distributions we want and extract them in order to
+get the metadata.
+
+Still, this simple API have been made mirrored by the recent works introduced
+by the PEP 381. This API contains archives of the files that can be downloaded.
+
+The XML/RPC API
+---------------
+
+The other way to get informations on the distributions, is using the XML/RPC
+interface. This one is working fine, but have some drawbacks as well:
+
+* It is not replicated, so if the main server falls, then it's not possible to
+  get informations that way.
+* The queries are run agains the server. If a large number of queries are made,
+  the server can fall, or be annoyingly slow.
+
+CouchDB
+=======
+
+CouchDB is a distributed database, which provides out of the box an easy
+replication. CouchDB is also working in a completely RESTful way.
+
+Having a version of PyPI metadata available in a couchdb instance (or many) can
+help to provide a high reliable packaging system, without the need to rely on
+techniques such as scrapping, which, even if they are working by now, are not
+really reliable, and have shown some limits (it's not possible to add metadata
+informations for instance).
+
+What I propose with this PEP is to run a CouchDB node directly aside the PyPI
+system, which can update directly the informations in the CouchDB server. As
+a proof of concept, I have made a little import script which get all the
+informations from the XML/RPC API and put them in a CouchDB server (More
+informations on http://blog.notmyidea.org/pypi-on-couchdb.html).
+
+Distutils2 client script
+========================
+
+Distutils2 already provides clients to work with the XML/RPC and the Simple
+APIs. A client can be done using the HTTP RESTful interface as well, and then
+be used by distutils2 installer directly.
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.