BLOCKER: Nobs crash with Python 2.6.0/2.7.0 on Dirac
We currently enforce a Python requirement of 2.6 in our scripts, but using Python 2.6.0 OR 2.7.0 on dirac (pcp-d-6 : module load python/2.x.0) currently gives nobs crashes:
$ git describe
upcxx-2017.9.1-61-g016ca07
$ module switch python python/2.6.0
$ python --version
Python 2.6
$ rm -Rf .nobs/ ; ./install poo
UPCXX revision: upcxx-2017.9.1-61-g016ca07
System: Linux pcp-d-6 3.10.0-693.1.1.el7.x86_64 #1 SMP Tue Aug 15 08:36:44 CDT 2017 x86_64 x86_64 x86_64 GNU/Linux
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: Scientific
Description: Scientific Linux release 7.4 (Nitrogen)
Release: 7.4
Codename: Nitrogen
Date: Fri Jan 26 08:26:27 PST 2018
Current directory: /home/pcp1/bonachea/UPC/upcxx
Install directory:
Settings: UPCXX_CODEMODE='debug'
/usr/local/pkg/gcc/7.2.0/bin/g++
g++ (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/local/pkg/gcc/7.2.0/bin/gcc
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Downloading https://gasnet.lbl.gov/EX/GASNet-2017.12.0.tar.gz
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Uncaught exception
Traceback (most recent call last):
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/tool_main.py", line 86, in main
printed.wait()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 447, in wait
return me.wait_futurized().value()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1085, in execute
kind, dir_path = yield me.get_config()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 601, in fire1
proxied = lam(arg_result)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 549, in result
return resultoid()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1034, in execute
cc, cxx, debug, config, source_dir, user_args = yield me.get_config()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 601, in fire1
proxied = lam(arg_result)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 549, in result
return resultoid()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1020, in get_config
source_dir = yield cxt.gasnet_source()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 879, in execute
yield download()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 604, in fire1
proxied = lam(*arg_result._val_seq, **arg_result._val_kws)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 876, in download
urllib.urlretrieve(url, tgz)
File "/usr/local/pkg/python/2.6.0/lib/python2.6/urllib.py", line 94, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "/usr/local/pkg/python/2.6.0/lib/python2.6/urllib.py", line 235, in retrieve
fp = self.open(url, data)
File "/usr/local/pkg/python/2.6.0/lib/python2.6/urllib.py", line 200, in open
return self.open_unknown(fullurl, data)
File "/usr/local/pkg/python/2.6.0/lib/python2.6/urllib.py", line 212, in open_unknown
raise IOError, ('url error', 'unknown url type', type)
IOError: [Errno url error] unknown url type: 'https'
UPC++ Installation failed. Please report the entire log above to: upcxx@googlegroups.com
$ module switch python python/2.7.0
$ rm -Rf .nobs/ ; ./install poo
UPCXX revision: upcxx-2017.9.1-61-g016ca07
System: Linux pcp-d-6 3.10.0-693.1.1.el7.x86_64 #1 SMP Tue Aug 15 08:36:44 CDT 2017 x86_64 x86_64 x86_64 GNU/Linux
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: Scientific
Description: Scientific Linux release 7.4 (Nitrogen)
Release: 7.4
Codename: Nitrogen
Date: Fri Jan 26 08:26:34 PST 2018
Current directory: /home/pcp1/bonachea/UPC/upcxx
Install directory:
Settings: UPCXX_CODEMODE='debug'
/usr/local/pkg/gcc/7.2.0/bin/g++
g++ (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/local/pkg/gcc/7.2.0/bin/gcc
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Downloading https://gasnet.lbl.gov/EX/GASNet-2017.12.0.tar.gz
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Uncaught exception
Traceback (most recent call last):
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/tool_main.py", line 86, in main
printed.wait()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 447, in wait
return me.wait_futurized().value()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1085, in execute
kind, dir_path = yield me.get_config()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 601, in fire1
proxied = lam(arg_result)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 549, in result
return resultoid()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1034, in execute
cc, cxx, debug, config, source_dir, user_args = yield me.get_config()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 601, in fire1
proxied = lam(arg_result)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 549, in result
return resultoid()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 1020, in get_config
source_dir = yield cxt.gasnet_source()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/memodb.py", line 467, in memo_execute
yield result
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 751, in fire
arg = gen_throw(arg.exception, None, arg.traceback)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 879, in execute
yield download()
File "/home/pcp1/bonachea/UPC/upcxx/nobs/nobs/async.py", line 604, in fire1
proxied = lam(*arg_result._val_seq, **arg_result._val_kws)
File "/home/pcp1/bonachea/UPC/upcxx/nobsrule.py", line 876, in download
urllib.urlretrieve(url, tgz)
File "/usr/local/pkg/python/2.7.0/lib/python2.7/urllib.py", line 91, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "/usr/local/pkg/python/2.7.0/lib/python2.7/urllib.py", line 237, in retrieve
fp = self.open(url, data)
File "/usr/local/pkg/python/2.7.0/lib/python2.7/urllib.py", line 202, in open
return self.open_unknown(fullurl, data)
File "/usr/local/pkg/python/2.7.0/lib/python2.7/urllib.py", line 214, in open_unknown
raise IOError, ('url error', 'unknown url type', type)
IOError: [Errno url error] unknown url type: 'https'
UPC++ Installation failed. Please report the entire log above to: upcxx@googlegroups.com
$ module switch python python/sys-default
$ python --version
Python 2.7.5
$ rm -Rf .nobs/ ; ./install poo
UPCXX revision: upcxx-2017.9.1-61-g016ca07
System: Linux pcp-d-6 3.10.0-693.1.1.el7.x86_64 #1 SMP Tue Aug 15 08:36:44 CDT 2017 x86_64 x86_64 x86_64 GNU/Linux
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: Scientific
Description: Scientific Linux release 7.4 (Nitrogen)
Release: 7.4
Codename: Nitrogen
Date: Fri Jan 26 08:30:28 PST 2018
Current directory: /home/pcp1/bonachea/UPC/upcxx
Install directory:
Settings: UPCXX_CODEMODE='debug'
/usr/local/pkg/gcc/7.2.0/bin/g++
g++ (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/local/pkg/gcc/7.2.0/bin/gcc
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Downloading https://gasnet.lbl.gov/EX/GASNet-2017.12.0.tar.gz
Finished https://gasnet.lbl.gov/EX/GASNet-2017.12.0.tar.gz
Configuring GASNet...
(in /home/pcp1/bonachea/UPC/upcxx/.nobs/art/1a1cfc7cebb7e2113a4184368fd5ca28074c74a0)
/home/pcp1/bonachea/UPC/upcxx/.nobs/art/35f64cafbe12c820f471b72fee18d916c31295c9/GASNet-2017.12.0/configure --enable-debug --disable-psm --disable-mxm --disable-portals4 --disable-ofi --disable-parsync
<works>
For reference, here are some relevant release dates from https://www.python.org/downloads/:
- Python 2.7.9 2014-12-10
- Python 2.7.5 2013-05-12
- Python 2.7.0 2010-07-03
- Python 2.6.0 2008-10-02
IMO this is a show-stopper that needs to be dealt with (using minimal code changes) before release. My vote is that we document 2.7.5 as the minimum version for this beta release (using the patch below) and post-release investigate further.
--- a/README.md
+++ b/README.md
@@ -40,7 +40,7 @@ The current release is known to work on the following configurations:
Miscellaneous software requirements:
-* Python version 2.6 or newer
+* Python version 2.7.5 or newer
* Perl version 5.005 or newer
diff --git a/nobs/tool.py b/nobs/tool.py
index c061408..72f13b5 100755
--- a/nobs/tool.py
+++ b/nobs/tool.py
@@ -2,8 +2,8 @@
import sys
-if sys.version_info < (2,6):
- sys.stdout.write('ERROR: Python 2.6 required.\n')
+if sys.version_info < (2,7,5):
+ sys.stdout.write('ERROR: Python >= 2.7.5 required.\n')
exit(1)
if sys.version_info[0] != 2:
diff --git a/utils/upcxx-run b/utils/upcxx-run
index 78c8a5e..e204757 100755
--- a/utils/upcxx-run
+++ b/utils/upcxx-run
@@ -176,8 +176,8 @@ def set_ssh_servers(ssh_servers, nranks, conduit):
def main():
- if sys.version_info < (2, 6):
- print('Error: Python version >= 2.6 required')
+ if sys.version_info < (2, 7, 5):
+ print('Error: Python version >= 2.7.5 required')
return 1
args = load_args()
Note the version checks in both scripts above are NOT printing the expected message to the console for Python 2.6.0, due to other errors:
$ python --version
Python 2.6
$ ./utils/upcxx-run --help
Traceback (most recent call last):
File "./utils/upcxx-run", line 8, in <module>
import argparse
ImportError: No module named argparse
$ rm -Rf .nobs/ ; ./install poo
UPCXX revision: upcxx-2017.9.1-61-g016ca07
System: Linux pcp-d-6 3.10.0-693.1.1.el7.x86_64 #1 SMP Tue Aug 15 08:36:44 CDT 2017 x86_64 x86_64 x86_64 GNU/Linux
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: Scientific
Description: Scientific Linux release 7.4 (Nitrogen)
Release: 7.4
Codename: Nitrogen
Date: Fri Jan 26 08:40:22 PST 2018
Current directory: /home/pcp1/bonachea/UPC/upcxx
Install directory:
Settings: UPCXX_CODEMODE='debug'
/usr/local/pkg/gcc/7.2.0/bin/g++
g++ (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
/usr/local/pkg/gcc/7.2.0/bin/gcc
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
GASNET must point to local tarball file, tarball url, or gasnet source tree.
Comments (3)
-
reporter -
reporter Proposed resolution in pull request #10
After the beta we can explore lowering the minimum version provided we fix the problems preventing older versions from working.
-
reporter - changed status to resolved
- Log in to comment
I was able to make the nobs version check reach the install console by also sending the message to stderr: