Problem with variables in env_file

Issue #2 resolved
Anonymous created an issue

Hi. I'm trying your package and had a lot of trouble to get it to work right until I found out that variables inside quotes is ignored. The regex that catches variables inside files is not catching when using quotes.

This is ignored by the regex:

export VAR="/home/path"

while this is not:

export VAR=/home/path

In this case the quotes is not needed but I write quotes without even thinking about it since I work in a filesystem with many folders with spaces in them.

It's not a big deal since it's possible to get it to work without the quotes but pretty frustrating until you know whats going on. If you don't improve the regex maybe you could include a note about this in the documentation.

Thanks anyway for a very useful package!

Gustaf Ekeberg

Comments (14)

  1. Daniele Niero repo owner

    Hi Gustaf.

    Indeed! Thank you for reporting this.

    I have change the regular expression so that it should catch variables in quotes. I think this is the right thing to do, otherwise it will be hard to set paths with spaces.

    I'm going to submit a new patch version. Please let me know if it fixes the problem. It should but you never know.

    • Dan
  2. Gustaf Ekeberg

    Hi Daniele.

    Thank you for the fix, it's almost working now.

    The variable is captured, but something is wrong when it's passed to the build script. I guess that the quotes are quoted. When my export VAR="/project/folder" is sent to the build script it's seen by the build script as "/project/folder", not /project/folder. I don't know where the problem occurs.

    This indicates to me that the quotes are passed to the build script but they should not be. I guess that they are not needed anymore when the variable is processed as a python variable. You should probably remove the quotes from the regex, and instead only capture what's inside them.

    Maybe the regex could be like this:

    ^(?:(?i)export|(?i)set)\s([a-zA-Z0-9%_$/]*)\=("|'|)(.+?)\2$

    This will catch the outermost quotes or the entire variable if no matching pair of quotes is found. The key is in group 1, and the variable in group 3, group 2 is used to match quotes.

    I'm not very familiar with python so this could be totally wrong.

  3. Gustaf Ekeberg

    One more: ^(?:(?i)export|(?i)set)\s([a-zA-Z0-9%_$/]*)\=(["']?)([^\2]*?)\2.*$

    This one will match the first pair of single or double quote symbols, making it possible to have a comment after the variable is set:

    export VAR="'my var'" #comment
    export VAR_II="/my folder/here" # comment
    

    The first capture will be: VAR, 'my var', the second capture: VAR_II, /my folder/here

  4. Daniele Niero repo owner

    Hey Gustaf,

    That's great.

    However I'm not sure about allowing comments after the variables set. Do they work on Windows? Windows uses rem or :: so those have to be handled as well.

    Instinctively I wouldn't allow them. What is your thought?

    And thanks again for taking your time to improve this package, I really appreciate it. Have you thought about making a push request?

  5. Daniele Niero repo owner

    I tried the regex you posted, but unfortunately they seem not working properly. I'm sure it is a python's regex limitations or implementation thing.

    the best result I could get is this: ^(?:(?i)export|(?i)set)\s([a-zA-Z0-9%_$/]*)\=[\"\']?(.+?)[\"\']?$

    Please if you have other ideas let me know, otherwise I can submit the regular expression above.

    You can test yours with this snippet code:

    import re, os
    
    envs_file = "" # put here the path to your file
    
    variables_set = []
    
    regex = re.compile("^(?:(?i)export|(?i)set)\s([a-zA-Z0-9%_$/]*)\=[\"\']?(.+?)[\"\']?$", re.M|re.X|re.S)
    envf = open(os.path.abspath(envs_file), 'r')
    lines = envf.read()
    envf.close()
    
    it = re.finditer(regex, lines)
    for m in it:
        key, value = m.groups()
        variables_set.append((key, value))
    
    
    max_key_length = 0
    for pair in variables_set:
        if len(pair[0]) > max_key_length:
            max_key_length = len(pair[0])
    
    log_format = '{:>'+str(max_key_length)+'} = {}'
    
    for pair in variables_set:
        print( log_format.format(pair[0], pair[1]) )
    
  6. Gustaf Ekeberg

    I think I found a working solution to this now. The regex in my pull request will match the variable after the equal sign until first space character, the rest of the line is ignored. If matching quotes is found then only the part inside the quotes will be used as variable. If you enter something after the variable it will always be ignored.

  7. Gustaf Ekeberg

    I was also thinking about a different way to store the env vars. Why do you need to store it in a sh script? Is it to be able to source the file when working in terminal? You could actually store the vars in a JSON-file like you do in the sublime-project file. If you do it that way it would be possible to set different vars depending on OS.

    All of this depends on how you use env vars. For me it would work great if I stored the env vars in a JSON-file.

  8. Daniele Niero repo owner

    Hi Gustaf.

    It is exactly to use it in Terminal/Command Line. I did in in this way because I have some software and software launchers that need a bat/sh file to initialise their environment. I wanted to use the same file to set my environment in Sublime.

    What can be done is allow ALSO a json file. The file extension could be checked and then call a function accordingly.

    I'm going to review your pull request.

  9. Log in to comment