1. PyPA
  2. Python Packaging Authority Projects
  3. PyPI Metadata Formats
  4. Issues
Issue #48 resolved

Come up with a solution to Mangle + Windows Issues

Donald Stufft
created an issue

sdist:         No Mangle
bdist_dumb:    No Mangle
bdist_rpm:     Illegal Character (:)
bdist_wininst: Invalid Directory
bdist_msi:     Invalid Directory

None of these work on Windows, my assumption is that the : is an illegal
character to have in a Windows directory.


sdist:         : and + mangled to - (hyphen)
bdist_dumb:    : and + mangled to - (hyphen)
bdist_rpm:     : and + mangled to _ (underscore)
bdist_wininst: : and + mangled to - (hyphen)
bdist_msi:     Invalid Error
bdist_egg:     : and + mangled to _ (underscore)
bdist_wheel:   : and + mangled to _ (underscore)

So the most complex version is not a valid MSI version at all. I think that is OK because it appears the restrictions for MSI is [0-255].[0-255].[0-65535] so this is just something that someone who is creating a MSI has to live with.

As far as mangling goes, I think the easiest thing to say is to get setuptools to not mangle the filename for our characters and say that if you're creating a version with an epoch or a local version you need to have a setuptools greater >= <whatever verison does that>.

The most concerning thing to me here is that our most complex version is a completely invalid directory on Windows. My guess is that it's the : character which is a directory separator on windows which seems to be supported by http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29. Our packaging formats put the version into the filename and directory names inside of that, so we're going to need to either define a normalization for the : character, or we're going to need to pick a different character that is allowed on Windows.

Comments (7)

  1. Donald Stufft reporter

    It looks like Windows allows ! in a directory name... I don't particularly like it, but I don't think we have a whole lot of choices left in the ascii plane. I'm asking the person with a windows machine to see if that works there.

  2. Donald Stufft reporter

    Ok, everything but bdist_msi works on Windows and that's what I expected. The end result looks like distutils always uses version without any mangling whereas setuptools will mangle with either _ or - depending on the distribution type that is being built.

    So my recommendations are:

    • Switch from : to ! for epoch separation (Compatibility Numbers coming shortly)
    • Try to get setuptools to not mangle ! and + in a version, and tell people to upgrade to setuptools >= <whatever> if they want to use those.
    • Try to do the same for Wheel, this might require an update to the Wheel spec.
  3. Donald Stufft reporter

    It looks like switching from : to ! does not have any impact on our compatibility numbers, which I didn't expect it would.

    $ invoke check.pep440 --cached
    Total Version Compatibility:              230888/238504 (96.81%)
    Total Sorting Compatibility (Unfiltered): 42928/45328 (94.71%)
    Total Sorting Compatibility (Filtered):   45304/45328 (99.95%)
    Projects with No Compatible Versions:     797/45328 (1.76%)
    Projects with Differing Latest Version:   1156/45328 (2.55%)
  4. Donald Stufft reporter

    Alternative is continue to use : but escape it as ! in directories and filenames, that's trading a slightly nicer user editable/displayed (PyPI, pip list etc) for a confusing "sometimes it's : sometimes it's !" model.

  5. Nick Coghlan

    The issue I forgot for Windows is that ":" is the drive separator. RPM and deb files are POSIX only, so ":" is just a normal character there.

    I suggest we use "!" as the upstream epoch separator, then distros can translate that to ":" for their native packaging systems.

  6. Log in to comment