Issue #188 resolved

NT escaping in shebang in easy_install

Anonymous created an issue

Hi. As far as I can see, there is still escaping of spaces in the shebang in the current version: I find this quite unfortunate, as this is of very limited use, but prevents some valid use cases.

The current code quotes all shebang strings that contain spaces. On all POSIX systems that I tried (current Ubuntu, ancient redhat, current OS X) this is illegal and simply doesn't work. Given the name of the function, it seems this was intended for windows NT (?).

However, this escaping prevents valid use cases, for example it doesn't allow the specification of "/bin/env /bin/python". I think there is a workaround for this specific case. However I have a use case where I need a different command to produce the correct python executable. This would be easy, except for the escaping.

If it is possible to detect being on a posix platform (probably not that easy?), we could just skip the escaping, as it is never beneficial. If this is not possible, maybe just skip the escaping all together? The benefit seems very platform specific.

Thanks, Andy

Comments (27)

  1. Guy Rozendorn

    well, escaping doesn't work either:

    11:55:09  Guys-MacBook-Air  ~/infinigit  easy_install
    zsh: /Users/guy/.virtualenvs/with space/bin/easy_install: bad interpreter: /Users/guy/.virtualenvs/with\: no such file or directory
    error: No urls, filenames, or requirements specified (see --help)
    11:55:32  Guys-MacBook-Air  ~/infinigit  cat ~/.virtualenvs/with\ space/bin/easy_install
    #!/Users/guy/.virtualenvs/with\ space/bin/python
    # EASY-INSTALL-ENTRY-SCRIPT: 'setuptools==3.4.4','console_scripts','easy_install'
    __requires__ = 'setuptools==3.4.4'
    import sys
    from pkg_resources import load_entry_point
    if __name__ == '__main__':
            load_entry_point('setuptools==3.4.4', 'console_scripts', 'easy_install')()
  2. Guy Rozendorn

    escaping does not work either; I don't know of a solution to white-spaces in shebang lines on Unix. if there is one, re-open and we'll see how to implement it. but from a quick search spaces are not supported in shebangs (again, on Unix).

  3. Jason R. Coombs

    Guy, I think the OP is not requesting that Setuptools support spaces in filenames in shebang lines, but rather requesting that Setuptools not escape the shebang so that one can provide multiple entities in the shebang line.

  4. Jason R. Coombs

    Reading through the source, it appears to me as if "executable" is allowed to be supplied, but options to that executable should be present as options to the original script text.

    What this ticket is requesting is that "executable" be interpreted to allow multiple entities on Unix and be considered to be a single entity on Windows. That would be a bad choice.

    t3kcit would it be possible in your use case to instead pass a script with something like script_text="#!/usr/bin/env script param", executable="/usr/bin/env" ?

    The way I read the code, the 'script' and 'param' would be included in the resulting script and /usr/bin/env would be replaced by /usr/bin/env and there wouldn't be any quoted characters.

    Let me know what you find.

  5. t3kcit

    Thanks a lot for your replies and sorry for being unclear. I think Jason got it right. I basically want to pass parameters to the shebang executable. Unfortunately I didn't understand the proposed solution. Could you please point me to the documentation of "script_text" and which version it was added in?

    Thanks a lot!


  6. Jason R. Coombs

    When I mention 'script_text', I'm referring to the parameter to the get_script_header function.

    It's hard to infer from the description how you're using that call, but because you referenced it explicitly, I'm presuming you're calling that function directly or you've at least traced the behavior in your usage to that function call.

    t3kcit can you provide a description of what you're trying to accomplish (i.e. what are your inputs)?

  7. t3kcit

    Sorry to be unclear. I use (or want to use) something like the below:

           'build_scripts': {
               'executable': '/my/own/env give-me-python',

    in the call to setuptools setup function. The intention is to have the shebang in my scripts be "#!/my/own/env give-me-python". (similar to "/bin/env python").

    This ends up in the function "get_script_header" function, where it is escaped. Basically what I want is to set the shebang at setup time to be the above. I thought the options I give above were the way to achieve this, but I might be mistaken.

  8. Jason R. Coombs

    t3kcit: what if you set executable to '/my/own/env', but then in the scripts themselves, have a shebang:

    #!/any/executable/will/become/my/own/env give-me-python

    Will that work for your purposes?

  9. t3kcit

    I would prefer not to change the scripts if possible. I have to do this to a lot of packages and limiting my changes to the script will hopefully make it easier to maintain forks.

  10. Jason R. Coombs

    That clarifies things. So it seems that the issue is that you want to provide "options" (a.k.a. parameters) to the executable and simply passing those options as the executable is causing them to be escaped (as one might expect). In any case, setuptools can probably support "executable" meaning more than simply an executable.

  11. t3kcit

    Sorry for being unclear in the beginning. I disagree with escaping being expected, as this is a non-feature on basically all platforms. How would would you like to add this functionality? Maybe I can come up with a patch.

  12. Marcin Wojdyr

    I think I have the same problem, but I'd put it differently. Handling of the "executable" option in setuptools (in is not compatible with handling of this option in distutils.

    For example, if I have in ~/.pydistutils.cfg:

    executable = /usr/bin/env my-python

    distutils will, as expected, change shebang in scripts to:

    #!/usr/bin/env my-python

    but setuptools will start script with:

    #!"/usr/bin/env my-python"

    Could it be changed just for the sake of consistency? Simply don't use nt_quote_arg() on unix in

  13. t3kcit

    Another good point. I really don't see any reason for this call, as it clearly breaks things, but does not seem to have an beneficial effect on any platform. Can someone explain why these lines are there? As I said above, they don't work for any OS I tried.

  14. Jason R. Coombs

    I believe I have a fix for this. In the latest commits, I've updated Setuptools to allow for the build_scripts parameter to be a CommandSpec instance, which allows specifying the command line entities explicitly:

    'executable': easy_install.CommandSpec(['/my/own/env', 'give-me-python'])

    I quickly realized that this verbose syntax isn't particularly friendly, so I allowed it to also accept just a simple list:

    'executable': ['/my/own/env', 'give-me-python']

    I then realized that still won't fix the situation where the executable is specified in a metadata file (such as pydistutils.cfg), so I made a backward-incompatible change which treats the executable if it's a string as a command line, requiring quoting or escaping if spaces are expected in the path. So now the syntax should be exactly as users have expected:

    'executable': '/my/own/env give-me-python'

    I haven't made a release, but do please download and install the latest source and report your experience with this change.

  15. Joe Esposito

    I think this fix broke something else.

    I have Python installed in C:\Program Files (x86)\Python (for legacy reasons--it's kind of late to move it now), and any console_script I install from PyPI breaks now. Running the script causes a "failed to create process" error.

    I noticed that older console_scripts were working, and turns out it's because they were quoted.

    #!"C:\Program Files (x86)\Python\python.exe"

    Adding quotes around the newly installed scripts fixes it.

    Any suggestions? Should I open a new issue? Thanks.

  16. Log in to comment