Title: Inclusion of pip bootstrap in Python installation
Author: Richard Jones <email@example.com>
Type: Standards Track
This PEP proposes the inclusion of a pip boostrap executable in the Python
installation to simplify the use of 3rd-party modules by Python users.
This PEP does not propose to include the pip implementation in the Python
standard library. Nor does it propose to include any package management or
installation mechanisms beyond those specified in PEPs 470 and TODO distlib PEP.
Currently the user story for installing 3rd-party Python modules is
not as simple as it could be. It requires that all 3rd-party modules inform
the user of how to install the installer, typically via a link to the
installer. That link may be out of date or the steps required to perform the
install of the installer may be enough of a roadblock to prevent the user from
Large Python projects which emphasise a low barrier to entry have shied away
from depending on third party packages because of the introduction of this
potential stumbling block for new users.
With the inclusion of the package installer command in the standard Python
installation the barrier to installing additional software is considerably
reduced. It is hoped that this will therefore increase the likelihood that
Python projects will reuse third party software.
It is also hoped that this is reduces the number of proposals to include
more and more software in the Python standard library, and therefore that
more popular Python software is more easily upgradeable beyond requiring
Python installation upgrades.
An additional new Python package will be proposed, "pypublish", which will be a
tool for publishing packages to PyPI. It would replace the current "python
setup.py register" and "python setup.py upload" distutils commands. Again
because of the measured Python release cycle and extensive existing Python
installations these commands are difficult to bugfix and extend. Additionally
it is desired that the "register" and "upload" commands be able to be performed
over HTTPS with certificate validation. Since shipping CA certificate keychains
with Python is not really feasible (updating the keychain is quite difficult to
manage) it is desirable that those commands, and the accompanying keychain, be
made installable and upgradeable outside of Python itself.
TODO pypublish... hmm... but pypub is taken
Python install includes an executable called "pip" that attempts to import pip
machinery. If it can then the pip command proceeds as normal. If it cannot it
will bootstrap pip by downloading the pip install. Once installed, the pip
command proceeds as normal.
A boostrap is used in the place of a the full pip code so that we don't have
to bundle pip and also the install tool is upgradeable outside of the regular
Python upgrade timeframe and processes.
To avoid issues with sudo we will have the bootstrap install the pip
implementation to the per-user site-packages directory defined in PEP 370 and
implemented in Python 2.6/3.0. Since we avoid installing to the system Python we
also avoid conflicting with any other packaging system (on Linux systems, for
The bootstrapping process will proceed as follows:
1. The user system has Python (3.4+) installed. In the "scripts" directory of
the Python installation there is the bootstrap script called "pip".
2. The user will invoke a pip command, typically "pip install <package>", for
example "pip install Django".
3. The boostrap script will attempt to import the pip implementation. If this
succeeds, the pip command is processed normally.
4. On failing to import the pip implementation the bootstrap notifies the user
that it is "upgrading pip" and contacts PyPI to obtain the latest download
wheel file (see PEP 427.)
5. Upon downloading the file it is installed using the distlib installation
machinery for wheel packages. Upon completing the installation the user
is notified that "pip has been upgraded." TODO how is it verified?
6. The pip tool may now import the pip implementation and continues to process
the requested user command normally.
The Fedora variant of Linux has had a separate program called "pip" (a Perl
package installer) available for install for some time. The current Python "pip"
program is installed as "pip-python". It is hoped that the Fedora project will
resolve this issue by renaming the Perl installer.
The download of the pip install wheel file cannot be fully secured. We can make
an SSL (HTTPS) connection, but we cannot verify the PyPI certificate without a
CA certificates keychain. As mentioned before, this will not be available in a
Python installation due to the problems of managing the keychain. Any data
transferred from PyPI, even over HTTPS, must therefore be considered vulnerable
to Man-In-The-Middle (MITM) attack.
One solution to the MITM attack could be to ship a public key matching the PyPI
key with Python and validate just that against the PyPI server key. Any approach
here must consider the response to the PyPI key being expired or replaced.
Currently pip depends upon setuptools functionality. It is intended that before
Python 3.4 is shipped that the required functionlity will be present in Python's
standard library as the distlib module, and that pip would be modified to use
that functionality when present. TODO PEP reference for distlib?
None, so far, beyond the PEPs.
Nick Coghlan for his thoughts on the proposal and dealing with the Red Hat
Jannis Leidel and Carl Meyer for their thoughts.
This document has been placed in the public domain.