misc / notes / pep_ideas / index.rst

Various Ideas for Python and CPython

My ability to generate ideas for ways in which Python could potentially be improved in various areas vastly outstrips the time I'm willing to spend following up on them, kicking them around with other community members and turning them into suggestions that may actually prove to be a genuine enhancement to the language rather than just useless cruft new Python programmers have to learn in order to understand code other programmers might write.

As a result, there are plenty of ideas I've had that may actually have some merit, but that I don't have the time to explore. Some of these are going to be bad ideas that justifiably won't survive the gauntlet of python-ideas and python-dev review, while others may make sense as esoteric PyPI modules rather than as standard library or core language additions.

Beyond that, there are many fascinating things other people are working on, where I may be able to contribute some thoughts, but don't have the time to get more directly involved. Those noted below are all core development related - there are many fascinating things going on in the community and education space, as well as at higher levels of the application stack (especially in web development and scientific computing) but in those spaces I truly am just an observer (albeit one that is enthusiastically cheering on those doing the work).

Current Activities

These are things I am currently working on directly.

Working on the CPython startup sequence is currently fairly painful, and it can also be hard to tweak appropriately when embedding CPython in a larger application. I'm currently working on a plan to do something about that for Python 3.4:

  • PEP 432 (simplifying the CPython startup sequence)

Potentially following on from that are various ideas people have had (most notably Graham Dumpleton) to make subinterpreters more robust. I'm considering following up PEP 432 with additional work on that, including finally pushing module cleanup completely over to dependence on the cyclic GC (which should help with misbehaviour at interpreter shutdown, but needs to be done with great care to avoid creating ridiculous amounts of uncollectable garbage).

The test helpers in our regression test suite currently suffer from discoverability problems, so I've been working with Chris Jerdonek to make a package, and move the helpers out from their current location as peer modules of the regression tests themselves. Chris has a series of patches on the tracker for this, and we're just waiting for the final 3.2 maintenance release to be published before we apply them.

Finally, I've been taking an active interest in Daniel Holth's work on the binary wheel format and related PEPs:

  • PEP 425 (binary compatibility tags)
  • PEP 426 (updated package metadata format)
  • PEP 427 (the wheel format itself)

Help Needed

These are things I'd really like to see happen for 3.4, but I'm not actively working on them and, as far as I'm aware, neither is anyone else:

I think the first two items above will close a couple of important gaps in the Python 3 unicode support, while the last will resolve a regression in functionality that complicates the porting of some code from Python 2, as well as making certain forms of metaprogramming significantly simpler to read, write and use.

However, they don't impact me directly the way the way the startup and test suite issues do, so they drop a bit further down my personal priority list.

Other Interests

Other people are working on things where I'm an interested observer, contributing my thoughts but not specifically driving the process in any way:

  • Vinay Sajip and others are working on distlib, a new approach to providing better stdlib infrastructure for packaging and distribution for 3.4 (after the failed attempt based on distutils2 for 3.3)
  • Antoine Pitrou is working on PEP 428, a possible higher level API for access to various aspects of file and filesystem manipulation
  • Antoine is also working on PEP 3154, a new version of the pickle protocol (which should have some flow-on benefits for multi-processing)
  • Lennart Regebro is working on PEP 431, bringing political timezone support directly into the standard library
  • Guido van Rossum is working on PEP 3156, a standardised API for explicitly asychronous code

There are also rather a lot of tracker issues I am following, although my level of interest in those varies greatly :)

Tinkering with Ideas

These are other ideas that I'm actually working on (albeit intermittently).

I have two deferred drafts in the main PEP index, which are competing ideas to allow Python to represent thoughts that follow the mathematical pattern of "let <name> = <expression> where <define subexpressions>" in that order in addition to the current algorithmic order, just as we allow both recursive and iterative implementations of algorithms, and the use of both classes and closures for scoping, etc:

  • PEP 403 (the @in decorator clause)
  • PEP 3150 (the given statement local namespace clause)

(These are deferred because I think there are larger problems in the grand scheme of things, but "we want something like Ruby blocks" is also a topic that comes up fairly regularly, so it's useful to have these available as a focus for any discussions. I currently have a slight preference for the @in clause as I think it suitably targets the niche I am interested in, and better integrates with existing language concepts like decorators)

The following ideas are ones which I don't even consider baked enough to put together as PEPs. I may also decide not to submit them (primarily if someone else decides to run with them first or if I decide they're a bad idea after all)

Ideas for PyPI Prototype Packages

This section is for ideas that could conceivably be implemented as a package on the Python Package Index, and hence should be field tested in that environment before incorporation into the standard library can be seriously considered.

My own walkdir and shell-command packages fit here. They're mostly languishing because I'm not sure the ideas underlying them are actually all that solid.

Dubious Ideas

This section is for ideas that would require changes to the Python language definition. All such proposals are deemed dubious by default - ideas that would make the core Python language definition better are vastly outnumbered by those that would make it worse :)

Past Ideas

These are archived ideas that were either elaborated into full PEPs, or which I now actively consider to be a bad idea: