Issue #48 resolved

Come up with a solution to Mangle + Windows Issues

Donald Stufft
created an issue

sdist:         No Mangle
bdist_dumb:    No Mangle
bdist_rpm:     Illegal Character (:)
bdist_wininst: Invalid Directory
bdist_msi:     Invalid Directory

None of these work on Windows, my assumption is that the : is an illegal
character to have in a Windows directory.


sdist:         : and + mangled to - (hyphen)
bdist_dumb:    : and + mangled to - (hyphen)
bdist_rpm:     : and + mangled to _ (underscore)
bdist_wininst: : and + mangled to - (hyphen)
bdist_msi:     Invalid Error
bdist_egg:     : and + mangled to _ (underscore)
bdist_wheel:   : and + mangled to _ (underscore)

So the most complex version is not a valid MSI version at all. I think that is OK because it appears the restrictions for MSI is [0-255].[0-255].[0-65535] so this is just something that someone who is creating a MSI has to live with.

As far as mangling goes, I think the easiest thing to say is to get setuptools to not mangle the filename for our characters and say that if you're creating a version with an epoch or a local version you need to have a setuptools greater >= <whatever verison does that>.

The most concerning thing to me here is that our most complex version is a completely invalid directory on Windows. My guess is that it's the : character which is a directory separator on windows which seems to be supported by Our packaging formats put the version into the filename and directory names inside of that, so we're going to need to either define a normalization for the : character, or we're going to need to pick a different character that is allowed on Windows.

Comments (7)

  1. Donald Stufft reporter

    It looks like Windows allows ! in a directory name... I don't particularly like it, but I don't think we have a whole lot of choices left in the ascii plane. I'm asking the person with a windows machine to see if that works there.

  2. Donald Stufft reporter

    Ok, everything but bdist_msi works on Windows and that's what I expected. The end result looks like distutils always uses version without any mangling whereas setuptools will mangle with either _ or - depending on the distribution type that is being built.

    So my recommendations are:

    • Switch from : to ! for epoch separation (Compatibility Numbers coming shortly)
    • Try to get setuptools to not mangle ! and + in a version, and tell people to upgrade to setuptools >= <whatever> if they want to use those.
    • Try to do the same for Wheel, this might require an update to the Wheel spec.
  3. Donald Stufft reporter

    It looks like switching from : to ! does not have any impact on our compatibility numbers, which I didn't expect it would.

    $ invoke check.pep440 --cached
    Total Version Compatibility:              230888/238504 (96.81%)
    Total Sorting Compatibility (Unfiltered): 42928/45328 (94.71%)
    Total Sorting Compatibility (Filtered):   45304/45328 (99.95%)
    Projects with No Compatible Versions:     797/45328 (1.76%)
    Projects with Differing Latest Version:   1156/45328 (2.55%)
  4. Donald Stufft reporter

    Alternative is continue to use : but escape it as ! in directories and filenames, that's trading a slightly nicer user editable/displayed (PyPI, pip list etc) for a confusing "sometimes it's : sometimes it's !" model.

  5. Nick Coghlan

    The issue I forgot for Windows is that ":" is the drive separator. RPM and deb files are POSIX only, so ":" is just a normal character there.

    I suggest we use "!" as the upstream epoch separator, then distros can translate that to ":" for their native packaging systems.

  6. Log in to comment