Nobs should request Python 2

[Benjamin Brock]

/nobs/tool.py in the install package should specifically request Python 2. python is Python 3 on bleeding edge systems, and the nobs code is not Python 3 compatible.

Here's my recommended diff:

nobs/tool.py:
1c1
< #!/usr/bin/env python2
---
> #!/usr/bin/env python

[Dan Bonachea]

Ben - this is intentional, to accommodate systems where the command "python2" might not exist. If you look at line 10 you'll see where we automate fallback to "python2" if we got python 3.

If this fallback is not working correctly on your system, please post a complete trace of the commands you ran and the error messages.

[Benjamin Brock]

Gotcha. Here's the trace:

[xiii@shini upcxx-2018.9.0]$ ./install $HOME/pkg/upcxx-2018.9.0-dummy-install
System: Linux shini 4.19.8-arch1-1-ARCH #1 SMP PREEMPT Sat Dec 8 13:49:11 UTC 2018 x86_64 GNU/Linux

Date: Mon Dec 17 10:17:32 PST 2018
Current directory: /home/xiii/pkg/upcxx-2018.9.0
Install directory:
Settings:

/usr/bin/g++
g++ (GCC) 8.2.1 20181127
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/usr/bin/gcc
gcc (GCC) 8.2.1 20181127
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


  File "/home/xiii/pkg/upcxx-2018.9.0/nobs/tool.py", line 15
    from nobs.async import shutdown
                  ^
SyntaxError: invalid syntax
GASNET must point to local tarball file, tarball url, or gasnet source tree.

[Dan Bonachea]

When I try on a system with python=Python 3.5.0 and python2=Python 2.7.10, the auto fallback works as intended.

Your error might indicate you don't have a working python2 in your PATH (we should probably do a better job diagnosing that error). Note that UPC++ requires Python2 v2.7.5 or greater, available in PATH as either python2 or python.

Can you please post the output from python --version and python2 --version? You could also try the current HEAD of develop, which includes this information in the install script output.

[Benjamin Brock]

Here's the output with HEAD of develop:

[xiii@shini upcxx]$ ./install $HOME/src/upcxx-install/
UPCXX revision: upcxx-2018.9.3-6-g842a57d
System: Linux shini 4.19.8-arch1-1-ARCH #1 SMP PREEMPT Sat Dec 8 13:49:11 UTC 2018 x86_64 GNU/Linux

Date: Tue Dec 18 21:14:05 PST 2018
Current directory: /home/xiii/src/upcxx
Install directory:
Settings:

/usr/bin/python:  Python 3.7.1
/usr/bin/python2:  Python 2.7.15

/usr/bin/g++
g++ (GCC) 8.2.1 20181127
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/usr/bin/gcc
gcc (GCC) 8.2.1 20181127
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


  File "/home/xiii/src/upcxx/nobs/tool.py", line 15
    from nobs.async import shutdown
                  ^
SyntaxError: invalid syntax
GASNET must point to local tarball file, tarball url, or gasnet source tree.

[Dan Bonachea]

Hmm - this is a newer Python3 than I have handy for testing. The newest I have (on cori) is Python 3.6.5, where module load python/3.6-anaconda-5.2 still results in correct fall-back behavior.

It may be that Python 3.7 has broken the auto fall-back feature.

Can you please try setting envvar UPCXX_PYTHON=python2

[john bachan]

The offending line from nobs/tool.py has been removed as a matter of refactoring in commit 3297eb523760336ec49b71afe0f4f7301ee3ef8e. Perhaps that'll fix this. Don't have a Py3.7 to try it on yet.

[Dan Bonachea]

I've reproduced the original problem on Python 3.7.2.

The refactoring does indeed hide the problem, but the fallback mechanism still fails (at least on our test mac) because on at least that system the fallback needs to use python2.7 not python2.

Proposed solution in pull request #62 replaces the python-based fallback logic with more robust shell code, avoiding the whole issue of trying to find a working python interpreter from within python.

[Dan Bonachea]

nobs: Improve python detection logic

Replace problematic python-based fallback logic with more robust shell code that works harder to find a working python binary that reports a version number within our required range. The auto defaulting logic now checks all of the following commands, in order:

python2.7 python2.8 python2 python

Users can still set $UPCXX_PYTHON to force use of a particular python.

Resolves issue 187.

→ <<cset 646bd155235d>>