Source

python-peps / pep-0413.txt

Full commit
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
PEP: 413
Title: Faster evolution of the Python Standard Library
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 2012-02-24
Post-History: 2012-02-24, 2012-02-25
Resolution: TBD


Abstract
========

This PEP proposes the adoption of a separate versioning scheme for the
standard library (distinct from, but coupled to, the existing language
versioning scheme) that allows accelerated releases of the Python standard
library, while maintaining (or even slowing down) the current rate of
change in the core language definition.

Like PEP 407, it aims to adjust the current balance between measured
change that allows the broader community time to adapt and being able to
keep pace with external influences that evolve more rapidly than the current
release cycle can handle (this problem is particularly notable for
standard library elements that relate to web technologies).

However, it's more conservative in its aims than PEP 407, seeking to
restrict the increased pace of development to builtin and standard library
interfaces, without affecting the rate of change for other elements such
as the language syntax and version numbering as well as the CPython
binary API and bytecode format.


Rationale
=========

To quote the PEP 407 abstract:

    Finding a release cycle for an open-source project is a delicate exercise
    in managing mutually contradicting constraints: developer manpower,
    availability of release management volunteers, ease of maintenance for
    users and third-party packagers, quick availability of new features (and
    behavioural changes), availability of bug fixes without pulling in new
    features or behavioural changes.

    The current release cycle errs on the conservative side. It is adequate
    for people who value stability over reactivity. This PEP is an attempt to
    keep the stability that has become a Python trademark, while offering a
    more fluid release of features, by introducing the notion of long-term
    support versions.

I agree with the PEP 407 authors that the current release cycle of the 
*standard library* is too slow to effectively cope with the pace of change
in some key programming areas (specifically, web protocols and related
technologies, including databases, templating and serialisation formats).

However, I have written this competing PEP because I believe that the
approach proposed in PEP 407 of offering full, potentially binary
incompatible releases of CPython every 6 months places too great a burden
on the wider Python ecosystem.

Under the current CPython release cycle, distributors of key binary
extensions will often support Python releases even after the CPython branches
enter "security fix only" mode (for example, Twisted currently ships binaries
for 2.5, 2.6 and 2.7, NumPy and SciPy suport those 3 along with 3.1 and 3.2,
PyGame adds a 2.4 binary release, wxPython provides both 32-bit and 64-bit
binaries for 2.6 and 2.7, etc).

If CPython were to triple (or more) its rate of releases, the developers of
those libraries (many of which are even more resource starved than CPython)
would face an unpalatable choice: either adopt the faster release cycle
themselves (up to 18 simultaneous binary releases for PyGame!), drop
older Python versions more quickly, or else tell their users to stick to the
CPython LTS releases (thus defeating the entire point of speeding up the
CPython release cycle in the first place).

Similarly, many support tools for Python (e.g. syntax highlighters) can take
quite some time to catch up with language level changes.

At a cultural level, the Python community is also accustomed to a certain
meaning for Python version numbers - they're linked to deprecation periods,
support periods, all sorts of things. PEP 407 proposes that collective
knowledge all be swept aside, without offering a compelling rationale for why
such a course of action is actually *necessary* (aside from, perhaps, making
the lives of the CPython core developers a little easier at the expense of
everyone else).

However, if we go back to the primary rationale for increasing the pace of
change (i.e. more timely support for web protocols and related technologies),
we can note that those only require *standard library* changes. That means
many (perhaps even most) of the negative effects on the wider community can
be avoided by explicitly limiting which parts of CPython are affected by the
new release cycle, and allowing other parts to evolve at their current, more
sedate, pace.


Proposal
========

This PEP proposes the introduction of a new kind of CPython release:
"standard library releases". As with PEP 407, this will give CPython 3 kinds
of release:

* Language release: "x.y.0"
* Maintenance release: "x.y.z" (where z > 0)
* Standard library release: "x.y (xy.z)" (where z > 0)

Under this scheme, an unqualified version reference (such as "3.3") would
always refer to the most recent corresponding language or maintenance
release. It will never be used without qualification to refer to a standard
library release (at least, not by python-dev - obviously, we can only set an
example, not force the rest of the Python ecosystem to go along with it).

Language releases will continue as they are now, as new versions of the
Python language definition, along with a new version of the CPython
interpreter and the Python standard library. Accordingly, a language
release may contain any and all of the following changes:

* new language syntax
* new standard library changes (see below)
* new deprecation warnings
* removal of previously deprecated features
* changes to the emitted bytecode
* changes to the AST
* any other significant changes to the compilation toolchain
* changes to the core interpreter eval loop
* binary incompatible changes to the C ABI (although the PEP 384 stable ABI
  must still be preserved)
* bug fixes

Maintenance releases will also continue as they do today, being strictly
limited to bug fixes for the corresponding language release. No new features
or radical internal changes are permitted.

The new standard library releases will occur in parallel with each
maintenance release and will be qualified with a new version identifier
documenting the standard library version. Standard library releases may
include the following changes:

* new features in pure Python modules
* new features in C extension modules (subject to PEP 399 compatibility
  requirements)
* new features in language builtins (provided the C ABI remains unaffected)
* bug fixes from the corresponding maintenance release

Standard library version identifiers are constructed by combining the major
and minor version numbers for the Python language release into a single two
digit number and then appending a sequential standard library version
identifier.


Release Cycle
-------------

When maintenance releases are created, *two* new versions of Python would
actually be published on python.org (using the first 3.3 maintenance release,
planned for February 2013 as an example)::

    3.3.1       # Maintenance release
    3.3 (33.1)  # Standard library release

A further 6 months later, the next 3.3 maintenance release would again be
accompanied by a new standard library release::

    3.3.2       # Maintenance release
    3.3 (33.2)  # Standard library release

Again, the standard library release would be binary compatible with the
previous language release, merely offering additional features at the
Python level.

Finally, 18 months after the release of 3.3, a new language release would
be made around the same time as the final 3.3 maintenance and standard
library releases::

    3.3.3       # Maintenance release
    3.3 (33.3)  # Standard library release
    3.4.0       # Language release

The 3.4 release cycle would then follow a similar pattern to that for 3.3::

    3.4.1       # Maintenance release
    3.4 (34.1)  # Standard library release

    3.4.2       # Maintenance release
    3.4 (34.2)  # Standard library release

    3.4.3       # Maintenance release
    3.4 (34.3)  # Standard library release
    3.5.0       # Language release


Programmatic Version Identification
-----------------------------------

To expose the new version details programmatically, this PEP proposes the
addition of a new ``sys.stdlib_info`` attribute that records the new
standard library version above and beyond the underlying interpreter
version. Using the initial Python 3.3 release as an example::

    sys.stdlib_info(python=33, version=0, releaselevel='final', serial=0)

This information would also be included in the ``sys.version`` string::

    Python 3.3.0 (33.0, default, Feb 17 2012, 23:03:41) 
    [GCC 4.6.1]


Security Fixes and Other "Out of Cycle" Releases
------------------------------------------------

For maintenance releases the process of handling out-of-cycle releases (for
example, to fix a security issue or resolve a critical bug in a new release),
remains the same as it is now: the minor version number is incremented and a
new release is made incorporating the required bug fixes, as well as any
other bug fixes that have been committed since the previous release.

For standard library releases, the process is essentially the same, but the
corresponding "What's New?" document may require some tidying up for the
release (as the standard library release may incorporate new features,
not just bug fixes).


User Scenarios
==============

The versioning scheme proposed above is based on a number of user scenarios
that are likely to be encountered if this scheme is adopted. In each case,
the scenario is described for both the status quo (i.e. slow release cycle)
the versioning scheme in this PEP and the free wheeling minor version number
scheme proposed in PEP 407.

To give away the ending, the point of using a separate version number is that
for almost all scenarios, the important number is the *language* version, not
the standard library version. Most users won't even need to care that the
standard library version number exists. In the two identified cases where
it matters, providing it as a separate number is actually clearer and more
explicit than embedding the two different kinds of number into a single
sequence and then tagging some of the numbers in the unified sequence as
special.


Novice user, downloading Python from python.org in March 2013
-------------------------------------------------------------

**Status quo:** must choose between 3.3 and 2.7

**This PEP:** must choose between 3.3 (33.1), 3.3 and 2.7.

**PEP 407:** must choose between 3.4, 3.3 (LTS) and 2.7.

**Verdict:** explaining the meaning of a Long Term Support release is about as
complicated as explaining the meaning of the proposed standard library release
version numbers. I call this a tie.


Novice user, attempting to judge currency of third party documentation
----------------------------------------------------------------------

**Status quo:** minor version differences indicate 18-24 months of
language evolution

**This PEP:** same as status quo for language core, standard library version
numbers indicate 6 months of standard library evolution.

**PEP 407:** minor version differences indicate 18-24 months of language
evolution up to 3.3, then 6 months of language evolution thereafter.

**Verdict:** Since language changes and deprecations can have a much bigger
effect on the accuracy of third party documentation than the addition of new
features to the standard library, I'm calling this a win for the scheme
in this PEP.


Novice user, looking for an extension module binary release
-----------------------------------------------------------

**Status quo:** look for the binary corresponding to the Python version you are
running.

**This PEP:** same as status quo.

**PEP 407 (full releases):** same as status quo, but corresponding binary version
is more likely to be missing (or, if it does exist, has to be found amongst
a much larger list of alternatives).

**PEP 407 (ABI updates limited to LTS releases):** all binary release pages will
need to tell users that Python 3.3, 3.4 and 3.5 all need the 3.3 binary.

**Verdict:** I call this a clear win for the scheme in this PEP. Absolutely
nothing changes from the current situation, since the standard library
version is actually irrelevant in this case (only binary extension
compatibility is important).


Extension module author, deciding whether or not to make a binary release
-------------------------------------------------------------------------

**Status quo:** unless using the PEP 384 stable ABI, a new binary release is
needed every time the minor version number changes.

**This PEP:** same as status quo.

**PEP 407 (full releases):** same as status quo, but becomes a far more
frequent occurrence.

**PEP 407 (ABI updates limited to LTS releases):** before deciding, must first
look up whether the new release is an LTS release or an interim release. If
it is an LTS release, then a new build is necessary.

**Verdict:** I call this another clear win for the scheme in this PEP. As with
the end user facing side of this problem, the standard library version is
actually irrelevant in this case. Moving that information out to a
separate number avoids creating unnecessary confusion.


Python developer, deciding priority of eliminating a Deprecation Warning
------------------------------------------------------------------------

**Status quo:** code that triggers deprecation warnings is not guaranteed to
run on a version of Python with a higher minor version number.

**This PEP:** same as status quo

**PEP 407:** unclear, as the PEP doesn't currently spell this out. Assuming the
deprecation cycle is linked to LTS releases, then upgrading to a non-LTS
release is safe but upgrading to the next LTS release may require avoiding
the deprecated construct.

**Verdict:** another clear win for the scheme in this PEP since, once again, the
standard library version is irrelevant in this scenario.


Alternative interpreter implementor, updating with new features
---------------------------------------------------------------

**Status quo:** new Python versions arrive infrequently, but are a mish-mash of
standard library updates and core language definition and interpreter
changes.

**This PEP:** standard library updates, which are easier to integrate, are
made available more frequently in a form that is clearly and explicitly
compatible with the previous version of the language definition. This means
that, once an alternative implementation catches up to Python 3.3, they
should have a much easier time incorporating standard library features as
they happen (especially pure Python changes), leaving minor version number
updates as the only task that requires updates to their core compilation and
execution components.

**PEP 407 (full releases):** same as status quo, but becomes a far more
frequent occurrence.

**PEP 407 (language updates limited to LTS releases):** unclear, as the PEP
doesn't currently spell out a specific development strategy. Assuming a
3.3 compatibility branch is adopted (as proposed in this PEP), then the
outcome would be much the same, but the version number signalling would be
slightly less clear (since you would have to check to see if a particular
release was an LTS release or not).

**Verdict:** while not as clear cut as some previous scenarios, I'm still
calling this one in favour of the scheme in this PEP. Explicit is better than
implicit, and the scheme in this PEP makes a clear split between the two
different kinds of update rather than adding a separate "LTS" tag to an
otherwise ordinary release number. Tagging a particular version as being
special is great for communicating with version control systems and associated
automated tools, but it's a lousy way to communicate information to other
humans.

Python developer, deciding their minimum version dependency
-----------------------------------------------------------

**Status quo:** look for "version added" or "version changed" markers in the
documentation, check against ``sys.version_info``

**This PEP:** look for "version added" or "version changed" markers in the
documentation. If written as a bare Python version, such as "3.3", check
against ``sys.version_info``. If qualified with a standard library version,
such as "3.3 (33.1)", check against ``sys.stdlib_info``.

**PEP 407:** same as status quo

**Verdict:** the scheme in this PEP actually allows third party libraries to be
more explicit about their rate of adoption of standard library features. More
conservative projects will likely pin their dependency to the language
version and avoid features added in the standard library releases. Faster
moving projects could instead declare their dependency on a particular
standard library version. However, since PEP 407 does have the advantage of
preserving the status quo, I'm calling this one for PEP 407 (albeit with a
slim margin).


Python developers, attempting to reproduce a tracker issue
----------------------------------------------------------

**Status quo:** if not already provided, ask the reporter which version of
Python they're using. This is often done by asking for the first two lines
displayed by the interactive prompt or the value of ``sys.version``.

**This PEP:** same as the status quo (as ``sys.version`` will be updated to
also include the standard library version), but may be needed on additional
occasions (where the user knew enough to state their Python version, but that
proved to be insufficient to reproduce the fault).

**PEP 407:** same as the status quo

**Verdict:** another marginal win for PEP 407. The new standard library version
*is* an extra piece of information that users may need to pass back to
developers when reporting issues with Python libraries (or Python itself,
on our own tracker). However, by including it in ``sys.version``, many
fault reports will already include it, and it is easy to request if needed.


CPython release managers, handling a security fix
-------------------------------------------------

**Status quo:** create a new maintenance release incorporating the security
fix and any other bug fixes under source control. Also create source releases
for any branches open solely for security fixes.

**This PEP:** same as the status quo for maintenance branches. Also create a
new standard library release (potentially incorporating new features along
with the security fix). For security branches, create source releases for
both the former maintenance branch and the standard library update branch.

**PEP 407:** same as the status quo for maintenance and security branches,
but handling security fixes for non-LTS releases is currently an open
question.

**Verdict:** until PEP 407 is updated to actually address this scenario, a
clear win for this PEP.


Effects
=======

Effect on development cycle
---------------------------

Similar to PEP 407, this PEP will break up the delivery of new features into
more discrete chunks. Instead of a whole raft of changes landing all at once
in a language release, each language release will be limited to 6 months
worth of standard library changes, as well as any changes associated with
new syntax.


Effect on workflow
------------------

This PEP proposes the creation of a single additional branch for use in the
normal workflow. After the release of 3.3, the following branches would be
in use::

  2.7         # Maintenance branch, no change
  3.3         # Maintenance branch, as for 3.2
  3.3-compat  # New branch, backwards compatible changes
  default     # Language changes, standard library updates that depend on them

When working on a new feature, developers will need to decide whether or not
it is an acceptable change for a standard library release. If so, then it
should be checked in on ``3.3-compat`` and then merged to ``default``.
Otherwise it should be checked in directly to ``default``.

The "version added" and "version changed" markers for any changes made on
the ``3.3-compat`` branch would need to be flagged with both the language
version and the standard library version. For example: "3.3 (33.1)".

Any changes made directly on the ``default`` branch would just be flagged
with "3.4" as usual.

The ``3.3-compat`` branch would be closed to normal development at the
same time as the ``3.3`` maintenance branch. The ``3.3-compat`` branch would
remain open for security fixes for the same period of time as the ``3.3``
maintenance branch.


Effect on bugfix cycle
----------------------

The effect on the bug fix workflow is essentially the same as that on the
workflow for new features - there is one additional branch to pass through
before the change reaches the ``default`` branch.

If critical bugs are found in a maintenance release, then new maintenance and
standard library releases will be created to resolve the problem. The final
part of the version number will be incremented for both the language version
and the standard library version.

If critical bugs are found in a standard library release that do not affect
the associated maintenance release, then only a new standard library release
will be created and only the standard library's version number will be
incremented.

Note that in these circumstances, the standard library release *may* include
additional features, rather than just containing the bug fix. It is
assumed that anyone that cares about receiving *only* bug fixes without any
new features mixed in will already be relying strictly on the maintenance
releases rather than using the new standard library releases.


Effect on the community
-----------------------

PEP 407 has this to say about the effects on the community:

    People who value stability can just synchronize on the LTS releases which,
    with the proposed figures, would give a similar support cycle (both in
    duration and in stability).

I believe this statement is just plain wrong. Life isn't that simple. Instead,
developers of third party modules and frameworks will come under pressure to
support the full pace of the new release cycle with binary updates, teachers
and book authors will receive complaints that they're only covering an "old"
version of Python ("You're only using 3.3, the latest is 3.5!"), etc.

As the minor version number starts climbing 3 times faster than it has in the
past, I believe perceptions of language stability would also fall (whether
such opinions were justified or not).

I believe isolating the increased pace of change to the standard library,
and clearly delineating it with a separate version number will greatly
reassure the rest of the community that no, we're not suddenly
asking them to triple their own rate of development. Instead, we're merely
going to ship standard library updates for the next language release in
6-monthly installments rather than delaying them all until the next language
definition update, even those changes that are backwards compatible with the
previously released version of Python.

The community benefits listed in PEP 407 are equally applicable to this PEP, 
at least as far as the standard library is concerned:

    People who value reactivity and access to new features (without taking the
    risk to install alpha versions or Mercurial snapshots) would get much more
    value from the new release cycle than currently.

    People who want to contribute new features or improvements would be more
    motivated to do so, knowing that their contributions will be more quickly
    available to normal users.

If the faster release cycle encourages more people to focus on contributing
to the standard library rather than proposing changes to the language
definition, I don't see that as a bad thing.


Handling News Updates
=====================


What's New?
-----------

The "What's New" documents would be split out into separate documents for
standard library releases and language releases. So, during the 3.3 release
cycle, we would see:

* What's New in Python 3.3?
* What's New in the Python Standard Library 33.1?
* What's New in the Python Standard Library 33.2?
* What's New in the Python Standard Library 33.3?

And then finally, we would see the next language release:

* What's New in Python 3.4?

For the benefit of users that ignore standard library releases, the 3.4
What's New would link back to the What's New documents for each of the
standard library releases in the 3.3 series.


NEWS
----

Merge conflicts on the NEWS file are already a hassle. Since this PEP
proposes introduction of an additional branch into the normal workflow,
resolving this becomes even more critical. While Mercurial phases may
help to some degree, it would be good to eliminate the problem entirely.

One suggestion from Barry Warsaw is to adopt a non-conflicting
separate-files-per-change approach, similar to that used by Twisted [2_].

Given that the current manually updated NEWS file will be used for the 3.3.0
release, one possible layout for such an approach might look like::

  Misc/
    NEWS  # Now autogenerated from news_entries
    news_entries/
      3.3/
        NEWS # Original 3.3 NEWS file
        maint.1/ # Maintenance branch changes
          core/
            <news entries>
          builtins/
            <news entries>
          extensions/
            <news entries>
          library/
            <news entries>
          documentation/
            <news entries>
          tests/
            <news entries>
        compat.1/ # Compatibility branch changes
          builtins/
            <news entries>
          extensions/
            <news entries>
          library/
            <news entries>
          documentation/
            <news entries>
          tests/
            <news entries>
        # Add maint.2, compat.2 etc as releases are made
      3.4/
        core/
          <news entries>
        builtins/
          <news entries>
        extensions/
          <news entries>
        library/
          <news entries>
        documentation/
          <news entries>
        tests/
          <news entries>
        # Add maint.1, compat.1 etc as releases are made

Putting the version information in the directory heirarchy isn't strictly
necessary (since the NEWS file generator could figure out from the version
history), but does make it easier for *humans* to keep the different versions
in order.


Other benefits of reduced version coupling
==========================================

Slowing down the language release cycle
---------------------------------------

The current release cycle is a compromise between the desire for stability
in the core language definition and C extension ABI, and the desire to get
new features (most notably standard library updates) into user's hands more
quickly.

With the standard library release cycle decoupled (to some degree) from that
of the core language definition, it provides an opportunity to actually 
*slow down* the rate of change in the language definition. The language
moratorium for Python 3.2 effectively slowed that cycle down to *more than 3
years* (3.1: June 2009, 3.3: August 2012) without causing any major
problems or complaints.

The NEWS file management scheme described above is actually designed to
allow us the flexibility to slow down language releases at the same time
as standard library releases become more frequent.

As a simple example, if a full two years was allowed between 3.3 and 3.4,
the 3.3 release cycle would end up looking like::

    3.2.4       # Maintenance release
    3.3.0       # Language release

    3.3.1       # Maintenance release
    3.3 (33.1)  # Standard library release

    3.3.2       # Maintenance release
    3.3 (33.2)  # Standard library release

    3.3.3       # Maintenance release
    3.3 (33.3)  # Standard library release

    3.3.4       # Maintenance release
    3.3 (33.4)  # Standard library release
    3.4.0       # Language release

The elegance of the proposed branch structure and NEWS entry layout is that
this decision wouldn't really need to be made until shortly before the planned
3.4 release date. At that point, the decision could be made to postpone the
3.4 release and keep the ``3.3`` and ``3.3-compat`` branches open after the
3.3.3 maintenance release and the 3.3 (33.3) standard library release, thus
adding another standard library release to the cycle. The choice between
another standard library release or a full language release would then be
available every 6 months after that.


Further increasing the pace of standard library development
-----------------------------------------------------------

As noted in the previous section, one benefit of the scheme proposed in this
PEP is that it largely decouples the language release cycle from the
standard library release cycle. The standard library could be updated every
3 months, or even once a month, without having any flow on effects on the
language version numbering or the perceived stability of the core language.

While that pace of development isn't practical as long as the binary
installer creation for Windows and Mac OS X involves several manual steps
(including manual testing) and for as long as we don't have separate
"<branch>-release" trees that only receive versions that have been marked as
good by the stable buildbots, it's still a useful criterion to keep in mind
when considering proposed new versioning schemes: what if we eventually want
to make standard library releases even *faster* than every 6 months?

If the practical issues were ever resolved, then the separate standard
library versioning scheme in this PEP could handle it. The tagged version
number approach proposed in PEP 407 could not (at least, not without a lot
of user confusion and uncertainty).


Other Questions
===============

Why not use the major version number?
-------------------------------------

The simplest and most logical solution would actually be to map the
major.minor.micro version numbers to the language version, stdlib version
and maintenance release version respectively.

Instead of releasing Python 3.3.0, we would instead release Python 4.0.0
and the release cycle would look like::

    4.0.0  # Language release

    4.0.1  # Maintenance release
    4.1.0  # Standard library release

    4.0.2  # Maintenance release
    4.2.0  # Standard library release

    4.0.3  # Maintenance release
    4.3.0  # Standard library release
    5.0.0  # Language release

However, the ongoing pain of the Python 2 -> Python 3 transition (and
associated workarounds like the ``python3`` and ``python2`` symlinks to
refer directly to the desired release series) means that this simple option
isn't viable for historical reasons.

One way that this simple approach *could* be made to work is to merge the
current major and minor version numbers directly into a 2-digit major
version number::

    33.0.0  # Language release

    33.0.1  # Maintenance release
    33.1.0  # Standard library release

    33.0.2  # Maintenance release
    33.2.0  # Standard library release

    33.0.3  # Maintenance release
    33.3.0  # Standard library release
    34.0.0  # Language release


Why not use a four part version number?
---------------------------------------

Another simple versioning scheme would just add a "standard library" version
into the existing versioning scheme::

    3.3.0.0  # Language release

    3.3.0.1  # Maintenance release
    3.3.1.0  # Standard library release

    3.3.0.2  # Maintenance release
    3.3.2.0  # Standard library release

    3.3.0.3  # Maintenance release
    3.3.3.0  # Standard library release
    3.4.0.0  # Language release

However, this scheme isn't viable due to backwards compatibility constraints
on the ``sys.version_info`` structure.


Why not use a date-based versioning scheme?
-------------------------------------------

Earlier versions of this PEP proposed a date-based versioning scheme for
the standard library. However, such a scheme made it very difficult to
handle out-of-cycle releases to fix security issues and other critical
bugs in standard library releases, as it required the following steps:

1. Change the release version number to the date of the current month.
2. Update the What's New, NEWS and documentation to refer to the new release
   number.
3. Make the new release.

With the sequential scheme now proposed, such releases should at most require
a little tidying up of the What's New document before making the release.


Why isn't PEP 384 enough?
-------------------------

PEP 384 introduced the notion of a "Stable ABI" for CPython, a limited
subset of the full C ABI that is guaranteed to remain stable. Extensions
built against the stable ABI should be able to support all subsequent
Python versions with the same binary.

This will help new projects to avoid coupling their C extension modules too
closely to a specific version of CPython. For existing modules, however,
migrating to the stable ABI can involve quite a lot of work (especially for
extension modules that define a lot of classes). With limited development
resources available, any time spent on such a change is time that could
otherwise have been spent working on features that offer more direct benefits
to end users.

There are also other benefits to separate versioning (as described above)
that are not directly related to the question of binary compatibility with
third party C extensions.


Why no binary compatible additions to the C ABI in standard library releases?
-----------------------------------------------------------------------------

There's a case to be made that *additions* to the CPython C ABI could
reasonably be permitted in standard library releases. This would give C
extension authors the same freedom as any other package or module author
to depend either on a particular language version or on a standard library
version.

The PEP currently associates the interpreter version with the language
version, and therefore limits major interpreter changes (including C ABI
additions) to the language releases.

An alternative, internally consistent, approach would be to link the
interpreter version with the standard library version, with only changes that
may affect backwards compatibility limited to language releases.

Under such a scheme, the following changes would be acceptable in standard
library releases:

* Standard library updates

  * new features in pure Python modules
  * new features in C extension modules (subject to PEP 399 compatibility
    requirements)
  * new features in language builtins

* Interpreter implementation updates

  * binary compatible additions to the C ABI
  * changes to the compilation toolchain that do not affect the AST or alter
    the bytecode magic number
  * changes to the core interpreter eval loop

* bug fixes from the corresponding maintenance release

And the following changes would be acceptable in language releases:

* new language syntax
* any updates acceptable in a standard library release
* new deprecation warnings
* removal of previously deprecated features
* changes to the AST
* changes to the emitted bytecode that require altering the magic number
* binary incompatible changes to the C ABI (although the PEP 384 stable ABI
  must still be preserved)

While such an approach could probably be made to work, there does not appear
to be a compelling justification for it, and the approach currently described
in the PEP is simpler and easier to explain.


Why not separate out the standard library entirely?
---------------------------------------------------

A concept that is occasionally discussed is the idea of making the standard
library truly independent from the CPython reference implementation.

My personal opinion is that actually making such a change would involve a
lot of work for next to no pay-off. CPython without the standard library is
useless (the build chain won't even run, let alone the test suite). You also
can't create a standalone pure Python standard library either, because too
many "standard library modules" are actually tightly linked in to the
internal details of their respective interpreters (for example, the builtins,
``weakref``, ``gc``, ``sys``, ``inspect``, ``ast``).

Creating a separate CPython development branch that is kept compatible with
the previous language release, and making releases from that branch that are
identified with a separate standard library version number should provide
most of the benefits of a separate standard library repository with only a
fraction of the pain.


Acknowledgements
================

Thanks go to the PEP 407 authors for starting this discussion, as well as
to those authors and Larry Hastings for initial discussions of the proposal
made in this PEP.

References
==========

.. [1] PEP 407: New release cycle and introducing long-term support versions
   http://www.python.org/dev/peps/pep-0407/

.. [2] Twisted's "topfiles" approach to NEWS generation
   http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles

Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: