#85 Merged at 169574b

Fix command expansion and parsing.

  1. Clark Boylan

This pull request fixes issue 150 and at least one other problem in the command parser (multiword variable expansions are treated as a single argv item when they may not be).

Comments (8)

  1. Chris

    With this change, how do you propose passing in posargs that contains spaces? Can you include a test case that demonstrates how to do that?

      1. Clark Boylan author

        It breaks the old behavior in the tests that were updated (test_config.py) lines 281 and 307. In 1.6.1 all substitutions but posargs were apparently placed as distinct entries in argv despite spaces and other shell special characters. Posargs was different and parsed (by shlex iirc) before becoming argv entries. 1.7.0 made everything a distinct argv entry with minimal parsing. This PR goes to the other extreme of making everything parsed by shlex which is much more flexible but not backward compatible.

  2. Chris

    The way you characterized it on IRC nails my actual concern as expressed on #150, Ronny; posargs is a list, not a string, and should be treated as such. That said, we could look at inserting it quoted; that might be a better choice.

  3. Marc Abramowitz

    Here's an example:

    # testr.ini
    skipsdist = true
    whitelist_externals = echo
    commands =
        echo "python setup.py test --slowest --testr-args='{posargs}'"
    $ tox -c testr.ini -- --failing --parallel
    python runtests: PYTHONHASHSEED='1235942326'
    python runtests: commands[0] | echo python setup.py test --slowest --testr-args='--failing --parallel'
    python setup.py test --slowest --testr-args='--failing --parallel'

    Note that the quotes are preserved. This fails hard on the hg tip without this PR:

    ❯ tox -c testr.ini -- --failing --parallel
    Traceback (most recent call last):
      File "/Users/marca/python/virtualenvs/tox/bin/tox", line 9, in <module>
        load_entry_point('tox==1.7.0', 'console_scripts', 'tox')()
      File "/Users/marca/dev/hg-repos/tox/tox/_cmdline.py", line 25, in main
        config = parseconfig(args, 'tox')
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 47, in parseconfig
        parseini(config, inipath)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 285, in __init__
        self._makeenvconfig("python", "_xz_9", reader._subs, config)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 346, in _makeenvconfig
        vc.commands = reader.getargvlist(section, "commands")
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 518, in getargvlist
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 550, in _processcommand
        new_word = self._replace(word)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 666, in _replace
        return RE_ITEM_REF.sub(self._replace_match, x)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 662, in _replace_match
        return handler(match)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 633, in _replace_substitution
        val = self._substitute_from_other_section(sub_key)
      File "/Users/marca/dev/hg-repos/tox/tox/_config.py", line 627, in _substitute_from_other_section
        "substitution key %r not found" % key)
    tox.ConfigError: ConfigError: substitution key 'posargs' not found

    The current hg tip gets very confused when {posargs} appears inside quotes, apparently.