1. Holger Krekel
  2. tox
  3. Issues

Issues

Issue #99 on hold

Environment variable usage schema {env:XXX} is not working

jenisys
created an issue

Version info: tox-1.4.3, tox-1.5.dev7 (current head/tip)

The documentation describes that environment variables can be used/referenced as shown above. But there seems to be a number of issues.

PROBLEMS:

  • Environment variables that are setup via "setenv= ..." can not be referenced in "commands= ..." part.
  • Option --showconfig does not evaluate "setenv= ..." parts in testenv section before the output is shown.
[testenv:example]
commands=
    echo ENVIRONMENT-VAR foo: {env:foo}
    echo ENVIRONMENT-VAR bar: {env:bar}
setenv =
    foo = Alice
    bar = Bob
deps=

ADDITIONAL NOTE: Current repository tip has a problem with {homedir} placeholder when indexserver= local=file://{homedir}/.pip_downloads is used. This is working in box-1.4.3, but no longer working in tox-1.5.dev7.

Comments (17)

  1. Krisztian Fekete

    Re ADDITIONAL NOTE

    Could you describe how the indexserver= local=file://{homedir}/.pip_downloads worked?

    Could you give a minimal, but complete tox.ini exposing the problem and whatever setup needed - e.g. what exactly is in .pip_downloads?

    What are the outputs of tox -r -v (both 1.4.3 and 1.5dev7)?

  2. jenisys reporter

    The problem is already with tox --showconfig. The 1.5dev version just pukes on me when "{homedir}" is used in "tox.ini".

    shell:xxx [525]. tox --version
    1.5.dev7 imported from /.../tox/tox/__init__.pyc
    
    shell:xxx [526]. tox --showconfig
    Traceback (most recent call last):
      File "/usr/local/bin/tox", line 9, in <module>
        load_entry_point('tox==1.5.dev7', 'console_scripts', 'tox')()
      File "/.../tox/tox/_cmdline.py", line 24, in main
        config = parseconfig(args, 'tox')
      File "/.../tox/tox/_config.py", line 42, in parseconfig
        parseini(config, inipath)
      File "/.../tox/tox/_config.py", line 202, in __init__
        for line in reader.getlist(toxsection, "indexserver"):
      File "/.../tox/tox/_config.py", line 371, in getlist
        s = self.getdefault(section, name, None)
      File "/.../tox/tox/_config.py", line 463, in getdefault
        x = self._replace(x)
      File "/.../tox/tox/_config.py", line 570, in _replace
        return rexpattern.sub(replace_func, x)
      File "/.../tox/tox/_config.py", line 561, in _replace_match_no_quote
        return self._replace_match(match, quote=False)
      File "/.../tox/tox/_config.py", line 552, in _replace_match
        raise tox.exception.ConfigError("No support for the %s substitution type" % sub_type)
    tox.ConfigError: ConfigError: No support for the homedir}/.pip/downloads/simple
    local1  = file substitution type
    

    NOTES: I am basically using a local PyPI directory structure with all required, already downloaded packages. The corrected value is indexserver= local=file://{homedir}/.pip_downloads/simple.

    For a real-world example, see: https://github.com/behave/behave/blob/master/tox.ini There is a also the script that builds the local package index for me.

  3. Krisztian Fekete

    Thanks! From this info I was able to reproduce the regression.

    My previous fix introduced a bug, that causes multiple substitutions on same line to fail (non-greedy -> greedy re). This should be easy to fix (on the way).

    (BTW your tox.ini fails until the indexserver default entry is edited for online use - at least for the first time both tox -e init and tox -e py27 fails)

    NOTE: the discussion so far was about a regression, not about the main issue regarding substitutions of the format {env:variable}, so the issue is NOT resolved even if the regression is fixed.

  4. jenisys reporter

    Krisztian Puska Well actually my tox -e init is working as expected with tox 1.4.3. The reason for this is in the setenv= PIP_INDEX_URL = ... in the testenv:init. It provides pip with the pypi URL (as needed) for online usage.

    I assume you just inspected the "tox.ini" (or tried it out with current tip).

    Otherwise, thx for the fix.

  5. itxaka serrano

    This is still open and not working on current tox:

    $ tox --version
    2.1.1 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc
    
    $ cat tox.ini 
    [tox]
    envlist = py27
    skipsdist=True
    
    [testenv:py27]
    setenv =
        PYTHONPATH={toxinidir}
    commands =
        ls {env:PYTHONPATH}%                                                                                                                                                                                     
    
    $ tox
    Traceback (most recent call last):
      File "/usr/local/bin/tox", line 11, in <module>
        sys.exit(cmdline())
      File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 38, in main
        config = prepare(args)
      File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 26, in prepare
        config = parseconfig(args)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 219, in parseconfig
        parseini(config, inipath)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 669, in __init__
        self.make_envconfig(name, section, reader._subs, config)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 697, in make_envconfig
        res = meth(env_attr.name, env_attr.default)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 864, in getargvlist
        return _ArgvlistReader.getargvlist(self, s)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 1015, in getargvlist
        commands.append(cls.processcommand(reader, current_command))
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 1045, in processcommand
        new_word = reader._replace(word)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 981, in _replace
        return RE_ITEM_REF.sub(self._replace_match, x)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 977, in _replace_match
        return handler(match)
      File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 921, in _replace_env
        (envkey, envkey))
    tox.ConfigError: ConfigError: substitution env:'PYTHONPATH': unknown environment variable 'PYTHONPATH'
    

    Also happens if used anywhere else as far as my tests go.

    Its been reported since 2 years ago and there has been no advance on it, do you need any help on it? Been looking at the code myself but found it quite difficult to wrap my head around it on how to fix it properly.

    Im guessing that _replace_env is failing because is trying to check if the requested env is on os.environ while parsing the config, but the env variables are not set until the runtests action is launched (line 330 in venv.py: env = self._getenv(testcommand=True)

    Will try to see if there is a good solution to fix this :)

  6. Holger Krekel repo owner

    Manjunath Kumatagi FYI tox doesn't have a lot of regular contributors, i am afraid. myself, I only get to look at things from time to time and prioritize things according to my own needs and customers.

    itxaka serrano yes, your analysis is correct i think. Haven't looked if there maybe is a quick way to fix it and have setenv be honored during configuration reading time already. Happy if you find something and can do a PR -- the functional test you basically already have.

  7. Log in to comment