1. ebo
  2. tstat
  3. Issues
Issue #1 resolved

Install not functional on CentOS 6.3 with Python 2.6.6

Andrew Ring
created an issue

Hello,

I am having some difficulty getting this to work on CentOS 6.3 with Python 2.6.6. Following the directions on the overview page, I could not get the software to work.

Here is what I did and the results. Lines starting with "$" are things I did. The other lines are output.

$ hg clone https://bitbucket.org/ebo/tstat
destination directory: tstat
requesting all changes
adding changesets
adding manifests
adding file changes
added 30 changesets with 46 changes to 13 files
updating to branch default
11 files updated, 0 files merged, 0 files removed, 0 files unresolved

$ cd tstat/

$ sudo python setup.py install
running install
running build
running build_py
running build_scripts
running install_lib
creating /usr/lib/python2.6/site-packages/tstat
copying build/lib/tstat/init.py -> /usr/lib/python2.6/site-packages/tstat
copying build/lib/tstat/pbs.py -> /usr/lib/python2.6/site-packages/tstat
byte-compiling /usr/lib/python2.6/site-packages/tstat/init.py to init.pyc
byte-compiling /usr/lib/python2.6/site-packages/tstat/pbs.py to pbs.pyc
running install_scripts
copying build/scripts-2.6/tstat -> /usr/bin
copying build/scripts-2.6/ttop -> /usr/bin
changing mode of /usr/bin/tstat to 755
changing mode of /usr/bin/ttop to 755
running install_egg_info
Writing /usr/lib/python2.6/site-packages/tstat-1.0-py2.6.egg-info

$ /usr/bin/tstat
Traceback (most recent call last):
File "/usr/bin/tstat", line 3, in <module>
from tstat import pbs, version
ImportError: No module named tstat

$ /usr/bin/ttop
Traceback (most recent call last):
File "/usr/bin/ttop", line 3, in <module>
from tstat import pbs, version
ImportError: No module named tstat

Thank you,
Andrew

Edit: fixed line breaks

Comments (12)

  1. ebo repo owner

    Can you confirm that all files are in the correct place (/usr/lib/python2.6/site-packages/tstat) and that /usr/bin/env python calls the same python interpreter as a plain python and that the site-packages are included in the sys.path of this interpreter?

  2. Andrew Ring reporter

    Uncertain

    $ sudo ls /usr/lib/python2.6/site-packages/tstat
    init.py init.pyc pbs.py pbs.pyc

    $ ls -al /usr/lib/python2.6/site-packages/
    total 72
    drwxr-xr-x. 11 root root 4096 Nov 22 11:09 .
    drwxr-xr-x. 3 root root 4096 Sep 11 2012 ..
    drwxr-xr-x. 2 root root 4096 Mar 7 2012 iniparse
    -rw-r--r--. 1 root root 1085 Aug 16 2010 iniparse-0.3.1-py2.6.egg-info
    drwxr-xr-x 2 root root 4096 Jan 9 2013 iotop
    drwxr-xr-x 2 root root 4096 Jan 9 2013 iotop-0.3.2-py2.6.egg-info
    -rw-r--r-- 1 root root 4330 Aug 14 2009 libproxy.py
    -rw-r--r-- 2 root root 3990 Nov 14 2012 libproxy.pyc
    -rw-r--r-- 2 root root 3990 Nov 14 2012 libproxy.pyo
    drwxr-xr-x. 2 root root 4096 Nov 7 2012 rpmUtils
    drwxr-xr-x. 3 root root 4096 Nov 7 2012 sos
    drwx------ 2 root root 4096 Nov 22 10:04 tstat
    -rw------- 1 root root 239 Nov 22 11:09 tstat-1.0-py2.6.egg-info
    drwxr-xr-x. 2 root root 4096 Mar 7 2012 urlgrabber
    -rw-r--r--. 1 root root 2285 Jul 19 2011 urlgrabber-3.9.1-py2.6.egg-info
    drwxr-xr-x. 2 root root 4096 Nov 7 2012 yum
    drwxr-xr-x. 2 root root 4096 Nov 7 2012 yumutils

    $ /usr/bin/env python --version
    Python 2.6.6

    $ python --version
    Python 2.6.6

  3. ebo repo owner

    The permissions of the tstat directory in site-packages are wrong. The directory and all files in it have to be world accessible.

    This is likely caused by having strict permissions on the original clone.

  4. Andrew Ring reporter

    Thank you. That advanced things a bit. Unfortuantly, I am still missing something.

    /usr/lib/python2.6/site-packages]$ sudo chmod -R oug+xr tstat
    /usr/lib/python2.6/site-packages]$ sudo chmod uog+r tstat-1.0-py2.6.egg-info

    $ ls -lrta
    total 32
    -rwxr-xr-x 1 root root 4659 Nov 22 10:01 pbs.py
    -rwxr-xr-x 1 root root 21 Nov 22 10:01 init.py
    -rwxr-xr-x 1 root root 167 Nov 22 10:04 init.pyc
    -rwxr-xr-x 1 root root 6207 Nov 22 10:04 pbs.pyc

    $ /usr/bin/tstat
    Traceback (most recent call last):
    File "/usr/bin/tstat", line 3, in <module>
    from tstat import pbs, version
    File "/usr/lib/python2.6/site-packages/tstat/pbs.py", line 38, in <module>
    pbs_connect = libtorque.pbs_connect
    File "/usr/lib64/python2.6/ctypes/init.py", line 366, in getattr
    func = self.getitem(name)
    File "/usr/lib64/python2.6/ctypes/init.py", line 371, in getitem
    func = self._FuncPtr((name_or_ordinal, self))
    AttributeError: /usr/bin/python: undefined symbol: pbs_connect

  5. ebo repo owner

    Now we are getting deep into ctypes territory, which is kind of hard for me to debug, because I do not have the CentOS environment.

    Try changing into your clones source directory and import pbs.py directly into a python cli.

    The fact that the exceptions shows to the python executable seems to imply, that there is a problem with dynamic loading and lookup.

    Please try to change the first line to the second line in pbs.py

    libtorque = ctypes.cdll.LoadLibrary(torque_name)                                                                                                             
    libtorque = ctypes.CDLL(torque_name, ctypes.RTLD_GLOBAL)
    
  6. Andrew Ring reporter

    ebo, thank you again.
    We may be stuck as far as troubleshooting goes then, as I do not know python.
    Is it possible there are additional dependencies?

    I found and replaced the line referenced above. I did the import in an interactive session with verboseness turned on.

    edit: fixed formatting

    $ python -v
    # installing zipimport hook
    import zipimport # builtin
    # installed zipimport hook
    # /usr/lib64/python2.6/site.pyc matches /usr/lib64/python2.6/site.py
    import site # precompiled from /usr/lib64/python2.6/site.pyc
    # /usr/lib64/python2.6/os.pyc matches /usr/lib64/python2.6/os.py
    import os # precompiled from /usr/lib64/python2.6/os.pyc
    import errno # builtin
    import posix # builtin
    # /usr/lib64/python2.6/posixpath.pyc matches /usr/lib64/python2.6/posixpath.py
    import posixpath # precompiled from /usr/lib64/python2.6/posixpath.pyc
    # /usr/lib64/python2.6/stat.pyc matches /usr/lib64/python2.6/stat.py
    import stat # precompiled from /usr/lib64/python2.6/stat.pyc
    # /usr/lib64/python2.6/genericpath.pyc matches /usr/lib64/python2.6/genericpath.py
    import genericpath # precompiled from /usr/lib64/python2.6/genericpath.pyc
    # /usr/lib64/python2.6/warnings.pyc matches /usr/lib64/python2.6/warnings.py
    import warnings # precompiled from /usr/lib64/python2.6/warnings.pyc
    # /usr/lib64/python2.6/linecache.pyc matches /usr/lib64/python2.6/linecache.py
    import linecache # precompiled from /usr/lib64/python2.6/linecache.pyc
    # /usr/lib64/python2.6/types.pyc matches /usr/lib64/python2.6/types.py
    import types # precompiled from /usr/lib64/python2.6/types.pyc
    # /usr/lib64/python2.6/UserDict.pyc matches /usr/lib64/python2.6/UserDict.py
    import UserDict # precompiled from /usr/lib64/python2.6/UserDict.pyc
    # /usr/lib64/python2.6/_abcoll.pyc matches /usr/lib64/python2.6/_abcoll.py
    import _abcoll # precompiled from /usr/lib64/python2.6/_abcoll.pyc
    # /usr/lib64/python2.6/abc.pyc matches /usr/lib64/python2.6/abc.py
    import abc # precompiled from /usr/lib64/python2.6/abc.pyc
    # /usr/lib64/python2.6/copy_reg.pyc matches /usr/lib64/python2.6/copy_reg.py
    import copy_reg # precompiled from /usr/lib64/python2.6/copy_reg.pyc
    import encodings # directory /usr/lib64/python2.6/encodings
    # /usr/lib64/python2.6/encodings/init.pyc matches /usr/lib64/python2.6 /encodings/init.py
    import encodings # precompiled from /usr/lib64/python2.6/encodings/init.pyc
    # /usr/lib64/python2.6/codecs.pyc matches /usr/lib64/python2.6/codecs.py
    import codecs # precompiled from /usr/lib64/python2.6/codecs.pyc
    import _codecs # builtin
    # /usr/lib64/python2.6/encodings/aliases.pyc matches /usr/lib64/python2.6 /encodings/aliases.py
    import encodings.aliases # precompiled from /usr/lib64/python2.6/encodings /aliases.pyc
    # /usr/lib64/python2.6/encodings/utf_8.pyc matches /usr/lib64/python2.6/encodings /utf_8.py
    import encodings.utf_8 # precompiled from /usr/lib64/python2.6/encodings/utf_8.pyc
    Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
    [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    dlopen("/usr/lib64/python2.6/lib-dynload/readline.so", 2);
    import readline # dynamically loaded from /usr/lib64/python2.6/lib-dynload /readline.so

    >>> import pbs.py
    # pbs.pyc has bad mtime
    import pbs # from pbs.py
    # can't create pbs.pyc
    import ctypes # directory /usr/lib64/python2.6/ctypes
    # /usr/lib64/python2.6/ctypes/init.pyc matches /usr/lib64/python2.6/ctypes /init.py
    import ctypes # precompiled from /usr/lib64/python2.6/ctypes/init.pyc
    dlopen("/usr/lib64/python2.6/lib-dynload/_ctypes.so", 2);
    import _ctypes # dynamically loaded from /usr/lib64/python2.6/lib-dynload /_ctypes.so
    # /usr/lib64/python2.6/struct.pyc matches /usr/lib64/python2.6/struct.py
    import struct # precompiled from /usr/lib64/python2.6/struct.pyc
    dlopen("/usr/lib64/python2.6/lib-dynload/_struct.so", 2);
    import _struct # dynamically loaded from /usr/lib64/python2.6/lib-dynload/_struct.so
    # /usr/lib64/python2.6/ctypes/_endian.pyc matches /usr/lib64/python2.6/ctypes /_endian.py
    import ctypes._endian # precompiled from /usr/lib64/python2.6/ctypes/_endian.pyc
    # /usr/lib64/python2.6/ctypes/util.pyc matches /usr/lib64/python2.6/ctypes/util.py
    import ctypes.util # precompiled from /usr/lib64/python2.6/ctypes/util.pyc
    # /usr/lib64/python2.6/re.pyc matches /usr/lib64/python2.6/re.py
    import re # precompiled from /usr/lib64/python2.6/re.pyc
    # /usr/lib64/python2.6/sre_compile.pyc matches /usr/lib64/python2.6/sre_compile.py
    import sre_compile # precompiled from /usr/lib64/python2.6/sre_compile.pyc
    import _sre # builtin
    # /usr/lib64/python2.6/sre_parse.pyc matches /usr/lib64/python2.6/sre_parse.py
    import sre_parse # precompiled from /usr/lib64/python2.6/sre_parse.pyc
    # /usr/lib64/python2.6/sre_constants.pyc matches /usr/lib64/python2.6 /sre_constants.py
    import sre_constants # precompiled from /usr/lib64/python2.6/sre_constants.pyc
    # /usr/lib64/python2.6/tempfile.pyc matches /usr/lib64/python2.6/tempfile.py
    import tempfile # precompiled from /usr/lib64/python2.6/tempfile.pyc
    # /usr/lib64/python2.6/random.pyc matches /usr/lib64/python2.6/random.py
    import random # precompiled from /usr/lib64/python2.6/random.pyc
    # /usr/lib64/python2.6/future.pyc matches /usr/lib64/python2.6/future.py
    import future # precompiled from /usr/lib64/python2.6/future.pyc
    dlopen("/usr/lib64/python2.6/lib-dynload/mathmodule.so", 2);
    import math # dynamically loaded from /usr/lib64/python2.6/lib-dynload /mathmodule.so
    dlopen("/usr/lib64/python2.6/lib-dynload/binascii.so", 2);
    import binascii # dynamically loaded from /usr/lib64/python2.6/lib-dynload/binascii.so
    dlopen("/usr/lib64/python2.6/lib-dynload/_randommodule.so", 2);
    import _random # dynamically loaded from /usr/lib64/python2.6/lib-dynload /_randommodule.so
    dlopen("/usr/lib64/python2.6/lib-dynload/cStringIO.so", 2);
    import cStringIO # dynamically loaded from /usr/lib64/python2.6/lib-dynload /cStringIO.so
    dlopen("/usr/lib64/python2.6/lib-dynload/fcntlmodule.so", 2);
    import fcntl # dynamically loaded from /usr/lib64/python2.6/lib-dynload /fcntlmodule.so
    import thread # builtin
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "pbs.py", line 39, in <module>
    pbs_connect = libtorque.pbs_connect( pbs_server )
    File "/usr/lib64/python2.6/ctypes/init.py", line 366, in getattr
    func = self.getitem(name)
    File "/usr/lib64/python2.6/ctypes/init.py", line 371, in getitem
    func = self._FuncPtr((name_or_ordinal, self))
    AttributeError: python: undefined symbol: pbs_connect

  7. sazs

    Hi Andrew,

    I had the same error.

    The problem is that environmet you are running "tstat" can't see libtorque.so.

    You can confirm this with the following test: /sbin/ldconfig -p |grep torque If it is not giving you results the library is not there.

    The workaroung is either to add libtroque.so path into /etc/ld.so.conf.d/ or hardcode the library in pbs.py:

    libtorque = ctypes.cdll.LoadLibrary("/full_path_to_your_so_file/libtorque.so")

  8. Andrew Ring reporter

    Sazs,

    Great find. Just tried things again, and things are working now out of the box.

    I would guess that the difference was a recent upgrade from Torque 2.5.10 to 4.2.6.1 on my test system. In that change the default location of the torque libraries moved form /opt/torque/lib/ to /usr/lib64/.

    Thank you, Andrew

  9. ebo repo owner

    Nice find. Having libtorque at a position where it can not be found by the dynamic loader is obviously a problem. I don't know if this is something fixable within tstat.

  10. sazs

    Maybe to change the pbs.py to something like?

    if torque_name:
        libtorque = ctypes.cdll.LoadLibrary(torque_name)
    else:
        print "No libtorque.so found"
        sys.exit(1)
    
  11. Log in to comment