Commits

Nick Coghlan committed 060127a

Clean up definition of local version identifiers

Most notable changes:
- integrator suffix -> local version label
- tuple of integers -> (mostly) arbitrary string
- ordered -> explicitly unordered
- excluded from version specifiers that rely on order
- excluded from prefix matching version specifiers

Issue #31: Make them usable for arbitrary build tags
Issue #33: Clarify interaction with prefix matching

Comments (0)

Files changed (3)

core-metadata.rst

 Version
 -------
 
-The distribution's public version identifier, as defined in PEP 440. Public
-versions are designed for consumption by automated tools and support a
-variety of flexible version specification mechanisms (see PEP 440 for
-details).
+The distribution's public or local version identifier, as defined in PEP 440.
+Version identifiers are designed for consumption by automated tools and
+support a variety of flexible version specification mechanisms (see PEP 440
+for details).
 
 Version identifiers MUST comply with the format defined in PEP 440.
 
 Version identifiers MUST be unique within each project.
 
+Index servers MAY place restrictions on the use of local version identifiers
+as described in PEP 440.
+
 Example::
 
     "version": "1.0a2"

standard-metadata-extensions.rst

 ===================================
 
 Structurally, this extension is largely identical to the ``python.project``
-extension.
+extension (the extension name is the only difference).
 
 However, where the ``project`` metadata refers to the upstream creators
 of the software, the ``integrator`` metadata refers to the downstream
 extension will not be used. However, if the software has been patched (for
 example, backporting compatible fixes from a later version, or addressing
 a platform compatibility issue), then this extension SHOULD be used, and
-an integrator suffix added to the package version number.
+a local version label added to the distribution's version identifier.
 
 If there are multiple redistributors in the chain, each one just overwrites
 this extension with their particular metadata.
 
 Local version identifiers MUST comply with the following scheme::
 
-    <public version identifier>[-N[.N]+]
+    <public version identifier>[-<local version label>]
+
+They consist of a normal public version identifier (as defined in the
+previous section), along with an arbitrary "local version label", separated
+from the public version identifier by a hyphen. Local version labels have
+no specific semantics assigned, but some syntactic restrictions are imposed.
 
 Local version identifiers are used to denote fully API compatible patched
-versions of upstream projects. These are created by application developers
-and system integrators when upgrading to a new upstream release would be too
+versions of upstream projects. For example, these may be created by
+application developers and system integrators by applying specific
+backported bug fixes when upgrading to a new upstream release would be too
 disruptive to the application or other integrated system (such as a Linux
 distribution).
 
-Local version identifiers may be used anywhere a public version identifier
-is expected.
-
-Local version identifiers MUST NOT include leading or trailing whitespace.
-
-Numeric components in the integrator suffix are interpreted in the same way
-as the numeric components of the release segment.
-
-The additional segment after the hyphen is referred to as the "integrator
-suffix", and makes it possible to differentiate upstream releases from
-potentially altered rebuilds by downstream integrators. The inclusion of an
-integrator suffix does not affect the kind of a release, but indicates that
+The inclusion of the local version label makes it possible to differentiate
+upstream releases from potentially altered rebuilds by downstream
+integrators. The use of a local version identifier does not affect the kind
+of a release but, when applied to a source distribution, does indicate that
 it may not contain the exact same code as the corresponding upstream release.
 
+To ensure local version identifiers can be readily incorporated as part of
+filenames and URLs, and to avoid formatting inconsistencies in hexadecimal
+hash representations, local version labels MUST be limited to the following
+set of permitted characters:
+
+* Lowercase ASCII letters (``[a-z]``)
+* ASCII digits (``[0-9]``)
+* underscores (``_``)
+* periods (``.``)
+* plus signs (``+``)
+
+Local version labels MUST start and end with an ASCII letter or digit.
+
+Local version identifiers may be used in most locations where a public
+version identifier is expected, with the exception of any version specifiers
+that explicitly rely on being able to unambiguously order candidate versions.
+
 Public index servers SHOULD NOT allow the use of local version identifiers
-in uploaded distributions. Local version identifiers are intended as a tool
-for software integrators rather than publishers.
+for uploaded *source* distributions, but MAY allow them for uploaded
+*binary* distributions. While local version identifiers are intended as a
+tool for software integrators rather than publishers, publishing binary
+distributions to a public index server blurs the distinction between the
+two roles.
 
-Distributions using a local version identifier SHOULD provide the
+Source distributions using a local version identifier SHOULD provide the
 ``python.integrator`` extension metadata (as defined in :pep:`459`).
 
 
 Source labels
 -------------
 
-Source labels are text strings with minimal defined semantics.
+Source labels are text strings with minimal defined semantics. They are
+intended to allow the original source code to be unambiguously identified,
+even if an integrator has applied additional local modifications to a
+particular distribution.
 
 To ensure source labels can be readily incorporated as part of file names
-and URLs, and to avoid formatting inconsistences in hexadecimal hash
+and URLs, and to avoid formatting inconsistencies in hexadecimal hash
 representations they MUST be limited to the following set of permitted
 characters:
 
 
 Source labels MUST start and end with an ASCII letter or digit.
 
-Source labels MUST be unique within each project and MUST NOT match any
-defined version for the project.
+A source label for a project MUST NOT match any defined version for that
+project. This restriction ensures that there is no ambiguity between version
+identifiers and source labels.
 
 
 Final releases
     1.0.post456
     1.1.dev1
 
-The integrator suffix of local version identifiers that share a common
-public version identifier prefix MUST be sorted in the same order as
-Python's tuple sorting when the integrator suffix is parsed as follows
-(this is the same definition as is used for the release segment)::
-
-    tuple(map(int, integrator_suffix.split(".")))
-
-All integrator suffixes involved in the comparison MUST be converted to a
-consistent length by padding shorter segments with zeroes as needed.
-
-All local version identifiers (even the ``-0`` suffix) are sorted *after*
-the corresponding unqualified public version identifier.
+Relative ordering of local version identifiers that share a common public
+version identifier is NOT defined by this PEP, as versions that share a
+common public version identifier are considered equivalent and
+interchangeable at this level. Particular integration environments SHOULD
+define their own local version ordering scheme for use by tools specific to
+that environment.
 
 
 Version ordering across different metadata versions
 labels to compatible public versions is to use the ``.devN`` suffix to
 specify the appropriate version order.
 
+Specific build information may also be included in local version labels.
+
 .. _Semantic versioning: http://semver.org/
 
 
 uniquely identify such releases for publication, while the source label is
 used to record the original DVCS based version label.
 
+Identifying hash information may also be included in local version labels.
+
 
 Olson database versioning
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 pre-releases are considered as candidate versions SHOULD be handled as
 described in `Handling of pre-releases`_.
 
+Except where specifically noted below, local version identifiers MUST NOT be
+permitted in version specifiers, and local version labels MUST be ignored
+entirely when checking if candidate versions match a given version
+specifier.
+
 
 Compatible release
 ------------------
 to be compatible with the specified version.
 
 The specified version identifier must be in the standard format described in
-`Version scheme`_.
+`Version scheme`_. Local version identifiers are NOT permitted in this
+version specifier.
 
 Automated tools SHOULD report an error when this operator is used in
 conjunction with a date based version identifier, as it assumes the use
 and a version identifier.
 
 The specified version identifier must be in the standard format described in
-`Version scheme`_, but a trailing ``.*`` is permitted as described below.
-
-If the specified version identifier is a public version identifier (no
-integrator suffix), then the integrator suffix of any candidate versions
-MUST be ignored when matching versions.
+`Version scheme`_, but a trailing ``.*`` is permitted on public version
+identifiers as described below.
 
 By default, the version matching operator is based on a strict equality
 comparison: the specified version must be exactly the same as the requested
 version. The *only* substitution performed is the zero padding of the
 release segment to ensure the release segments are compared with the same
-length (and similarly for the integrator suffix, if matching against a
-specified local version identifier).
+length.
 
 Whether or not strict version matching is appropriate depends on the specific
 use case for the version specifier. Automated tools SHOULD at least issue
 a trailing ``.*`` to the version identifier in the version matching clause.
 This means that additional trailing segments will be ignored when
 determining whether or not a version identifier matches the clause. If the
-version includes only a release segment, than trailing components in the
-release segment are also ignored.
+specified version includes only a release segment, than trailing components
+(or the lack thereof) in the release segment are also ignored.
 
 For example, given the version ``1.1.post1``, the following clauses would
 match or not as shown::
     == 1.1.post1  # Equal, so 1.1.post1 matches clause
     == 1.1.*      # Same prefix, so 1.1.post1 matches clause
 
+For purposes of prefix matching, the pre-release segment is considered to
+have an implied preceding ``.``, so given the version ``1.1a1``, the
+following clauses would match or not as shown::
+
+    == 1.1        # Not equal, so 1.1a1 does not match clause
+    == 1.1a1      # Equal, so 1.1a1 matches clause
+    == 1.1.*      # Same prefix, so 1.1a1 matches clause
+
+An exact match is also considered a prefix match (this interpreation is
+implied by the usual zero padding rules for the release segment of version
+identifiers). Given the version ``1.1``, the following clauses would
+match or not as shown::
+
+    == 1.1        # Equal, so 1.1 matches clause
+    == 1.1.0      # Zero padding expands 1.1 to 1.1.0, so it matches clause
+    == 1.1.dev1   # Not equal (dev-release), so 1.1 does not match clause
+    == 1.1a1      # Not equal (pre-release), so 1.1 does not match clause
+    == 1.1.post1  # Not equal (post-release), so 1.1 does not match clause
+    == 1.1.*      # Same prefix, so 1.1 matches clause
+
 The use of ``==`` (without at least the wildcard suffix) when defining
 dependencies for published distributions is strongly discouraged as it
 greatly complicates the deployment of security fixes. The strict version
 dependencies for repeatable *deployments of applications* while using
 a shared distribution index.
 
+If the specified version identifier is a public version identifier (no
+local version label), then the local version label of any candidate versions
+MUST be ignored when matching versions.
+
+If the specified version identifier is a local version identifier, then the
+local version labels of candidate versions MUST be considered when matching
+versions, with the public version identifier being matched as described
+above, and the local version label being checked for equivalence using a
+strict string equality comparison.
+
 
 Version exclusion
 -----------------
 those of the `Version matching`_ operator, except that the sense of any
 match is inverted.
 
-If the specified version identifier is a public version identifier (no
-integrator suffix), then the integrator suffix of any candidate versions
-MUST be ignored when excluding versions.
-
 For example, given the version ``1.1.post1``, the following clauses would
 match or not as shown::
     
 As with version matching, the release segment is zero padded as necessary to
 ensure the release segments are compared with the same length.
 
-Local version identifiers are handled according to the combination of their
-handling by the version matching operator and the consistent ordering
-defined by the standard version scheme.
+Local version identifiers are NOT permitted in this version specifier.
 
 
 Exclusive ordered comparison
 the given version, even if acceptance of pre-releases is enabled as
 described in the section below.
 
-Local version identifiers are handled according to the combination of their
-handling by the version exclusion operator and the consistent ordering
-defined by the standard version scheme.
+Local version identifiers are NOT permitted in this version specifier.
 
 
 Handling of pre-releases
 * Added the "direct reference" concept as a standard notation for direct
   references to resources (rather than each tool needing to invent its own)
 
-* Added the "local version identifier" and "integrator suffix" concepts to
+* Added the "local version identifier" and "local version label" concepts to
   allow system integrators to indicate patched builds in a way that is
-  supported by the upstream tools
+  supported by the upstream tools, as well as to allow the incorporation of
+  build tags into the versioning of binary distributions.
 
 * Added the "compatible release" clause
 
 
 Historically, this practice has been invisible to cross-platform language
 specific distribution tools - the reported "version" in the upstream
-metadata is the same as for the unmodified code. This inaccuracy then
-can then cause problems when attempting to work with a mixture of integrator
+metadata is the same as for the unmodified code. This inaccuracy can then
+cause problems when attempting to work with a mixture of integrator
 provided code and unmodified upstream code, or even just attempting to
 identify exactly which version of the software is installed.
 
-The introduction of local version identifiers and the "integrator suffix"
+The introduction of local version identifiers and "local version labels"
 into the versioning scheme, with the corresponding ``python.integrator``
 metadata extension allows this kind of activity to be represented
 accurately, which should improve interoperability between the upstream
 ``pkg_resources.parse_version`` and ``pkg_resources.parse_requirements``,
 with the main distinction being that where ``pkg_resources`` currently always
 takes the suffix into account when comparing versions for exact matches,
-the PEP requires that the integrator suffix of the candidate version be
-ignored when no integrator suffix is present in the version specifier clause.
+the PEP requires that the local version label of the candidate version be
+ignored when no local version label is present in the version specifier
+clause. Furthermore, the PEP does not attempt to impose any structure on
+the local version labels (aside from limiting the set of permitted
+characters), instead treating them as opaque strings that have no inherent
+order and can only be compared for strict equality.
 
 The hyphen is chosen primarily for readability of local version identifiers.
 While the wheel format also uses hyphens as separators between components,
 the escaping rules defined in PEP 427 will convert the hyphen in a local
-version identifier to an underscore before using it in a wheel filename.
+version identifier to an underscore before using it in a wheel filename
+(alternatively, a future iteration of the wheel specification may map the
+optional "build tag" component of the wheel file naming scheme to the
+"local version label" component in this PEP).
 
 This change is designed to ensure that an integrator provided version like
-``pip 1.5-1`` will still satisfy a version specifier like ``pip (== 1.1)``.
+``pip 1.5-1`` or ``pip-1.5-1.git.abc123de`` will still satisfy a version
+specifier like ``pip (>= 1.5)``.
 
 
 References