Issue #134 wontfix

Spaces in file names make it impossible to package for Solaris 9 and 10

Maciej Bliziński
created an issue

Comments (10)

  1. Jason R. Coombs

    Maciej, thanks for reporting this issue. I sympathize with your concerns. It appears you're reporting a defect/limitation in the Solaris packaging tools and seeking to extend that limitation to setuptools.

    Your report states that the spaces make it "impossible" to package for Solaris 9, but your references suggest that there is already a workaround implemented.

    In my opinion, that's the optimal solution. While it would be trivial for setuptools omit the use of spaces in the filename, it would also remove the demonstration that setuptools supports the use of spaces in package resources. Using spaces in the filename serves as a signal that this use-case is supported and works. It furthermore provides for a better aesthetic and is more congruent with the canonical word delimiter in English.

    Consider viewing the files in Windows Explorer, for example. Without spaces, the file names are clumsily displayed:

    But if spaces are employed, they appear much more naturally:

    More than just aesthetics, there are cases where allowing spaces means that resources can be presented to users without transformation (whereas if the names are encoded with a spaces as underscores encoding, it must be decoded prior to user consumption).

    Therefore, I believe there are tangible benefits to allowing resources with spaces in the names and having that use-case proven in the setuptools package has value. Part of this value is the exhibition of the edge cases such as the limitation in the Solaris package manager.

    In my opinion, it's embarrassing that a first-class package manager could exist without having support for such file names, especially considering that most file systems have had support for these names for most of two decades.

    Nevertheless, I don't want to press the issue if there is a serious impact to portability. Can you share more detail about the impact of this limitation? Is the workaround that you linked not adequate? Are there possibly other more elegant solutions? Could the packager instead use a zip-egg distribution, which would contain only that one file and thus wouldn't be subject to the disallowed filenames limitation?

  2. Maciej Bliziński reporter

    Hi Jason,

    I'm a package maintainer at OpenCSW, a community project dedicated to working closely with upstream projects to improve portability of open source software. I hope you and I are on the same page with the aim of setuptools being portable, meaning that you can build, install and package unmodified setuptools code on all platforms users care about.

    I'm trying to push the fix upstream so that I don't need to maintain a downstream set of patches. It'll also help other Solaris users who won't run into this problem in the future.

    The best place for checking if things work as required are unit tests. A unit test can prove that resources can be presented to users without transformation, without installing any troublesome files in the filesystem.

    I'm not sure about a zip-egg distribution. As long as files in the package file map have whitespace-free names, it should be fine.

    The simplest test to verify if the file names are OK is this:

    cd <setuptools-install-image>
    if [[ $(find . -name '* *' | wc -l) -gt 0 ]]; then echo BAD; else echo GOOD; fi
    

    Otherwise I'm getting this:

    ## Building pkgmap from package prototype file.
    ERROR in /home/maciej/src/opencsw/pkg/lang-python/pysetuptools/trunk/work/solaris9-sparc/build-global/CSWpy-setuptools.prototype-sparc:
        garbled entry
        - pathname: /opt/csw/lib/python2.6/site-packages/setuptools/script
        - problem: mode is not numeric.
    pkgmk: ERROR: unable to build pkgmap from prototype file
    ## Packaging was not successful.
    

    I agree that it's a silly limitation and you're right from software design point of view. In general, SVR4 package manager is a popular target of complaints and ridicule, for many reasons. I'm sending the bug report from a pragmatic standpoint, trying to save users and packagers headache.

    Maciej

  3. Jason R. Coombs

    Maciej, how does the package manager convert the setuptools tarfile into the files built for a particular target OS/Python version? I don't see anything in that Makefile that invokes setup.py to build or install setuptools. Does that happen implicitly due to the presence of a setup.py file in the source distribution?

  4. Maciej Bliziński reporter

    Hi Jason, the CATEGORIES variable is set to "python", so the build system runs python setup.py build and python setup.py --root=/path/to/install/image install, then runs the find utility to collect the files. It boils down to:

    $ touch example.txt  # the file must exist
    $ find . -print | pkgproto
    f none example.txt 0644 maciej other
    

    The resulting line is then used to create the package. A similar line is put into the package itself, in the pkgmap file.

  5. Jason R. Coombs

    Thanks for that information. Thanks also for pull request 33.

    I was hoping there might be a way to use a zip egg, but given that's the way the packaging routine handles Python packages, I don't see a straightforward way to produce a zip egg (with about as much customization as is already in the existing patch).

    If the issue were inherent in Solaris, I'd say that Setuptools should adjust to account for the issue. However, since the issue is in a packaging tool for Solaris, I believe the workaround in place is the best place for it. The patch serves as a signal to the limitation in the packaging system. If Setuptools were to adjust instead, there would be no good place to document the limitation in the system. That is, one couldn't provide hyperlinks to resources as you've done in this ticket to describe the impact of the limitation, because it would be subtly suppressed.

    That, along with my previously-mentioned benefits to providing the resource with spaces, I'll elect not to apply the patch. I make this choice with pained reservation because the tradeoffs don't suggest an obvious choice.

    I have added a comment to the code to reference this issue, which should bring attention to this issue if the script names are ever changed (so that the SVR4 patches can be considered).

  6. Martin Mokrejs

    I just came here because the spaces and round brackets break simple searches on the filesystem.

    Conside a command like:

    $ find /usr -name *blah* | while read f; do ls -latr $f; done

    This breaks on files containing spaces.

    Honestly, I am shocked \what you say here. Your intentionally invented this commit: https://bitbucket.org/pypa/setuptools/commits/90d6e15a0f5601f1f086d456bd01c835a9a10d65 Seriously, do you want to create troubles to all of us to show setuptools can handle spaces and maybe brackets in filenames and dirnames? Why isn't the first criterion whether can the file can be placed on a filesystem and whether it is searchable? Anyway, please generate least troubles to the rest of the world. sorry to say that.

    BTW, I do not mind this would have worked around in the particular example:

    $ find /usr -name *blah* | while read f; do ls -latr "$f"; done

  7. Log in to comment