1. litmis Avatar litmis
  2. Open Source Projects for IBM i
  3. ibmichroot
  4. Issues

python sym links

Issue #1 resolved
Aaron Bartell created an issue

Working on python chroot and one of the things pip3 binary requires is /QOpenSys/QIBM/ProdData/SC1/zlib/lib/libz.so which is sym linked via /QOpenSys/usr/lib/libz.so.

The symlink is relative:

$  ls -all  /QOpenSys/usr/lib/libz.so
lrwxrwxrwx    1 qsys     0                80 May 21 22:34 /QOpenSys/usr/lib/libz.so -> ../../QIBM/ProdData/SC1/zlib/lib/libz.so

Need to resolve all required symbolic links not only of supporting libs but also binary files. Here's a partial list:

  • /QOpenSys/usr/bin/2to3
  • /QOpenSys/usr/bin/2to3-3.4
  • /QOpenSys/usr/bin/easy_install-3.4
  • /QOpenSys/usr/bin/idle3
  • /QOpenSys/usr/bin/idle3.4
  • /QOpenSys/usr/bin/pip3
  • /QOpenSys/usr/bin/pip3.4
  • /QOpenSys/usr/bin/pydoc3
  • /QOpenSys/usr/bin/pydoc3.4
  • /QOpenSys/usr/bin/python3
  • /QOpenSys/usr/bin/python3-config
  • /QOpenSys/usr/bin/python3.4
  • /QOpenSys/usr/bin/python3.4-config
  • /QOpenSys/usr/bin/python3.4m
  • /QOpenSys/usr/bin/python3.4m-config
  • /QOpenSys/usr/bin/pyvenv
  • /QOpenSys/usr/bin/pyvenv-3.4
  • /QOpenSys/usr/lib/libpython3.4m.so
  • /QOpenSys/usr/lib/libz.so

Comments (19)

  1. Former user Account Deleted
    • assigned issue to

    This has a "policy" decision. Issue: if you force 'chase symbolic links' bin/lib in SC1, likely sets precedent to 'chase symbolic links' every thing
    choice 1) Force user to change his PATH, LIBPATH ... aka ... want to use SC1, then set you own PATH/LIBPATH Mr. Customer choice 2) Add new method to chroot_etup.sh to 'chase' SC1 symlinks /QOpenSys/QIBM/ProdData/SC1/..bin-or-lib to /QOpenSys/usr/bin-or-lib BTW -- not sure a saying 'chase lib/bin for IBM stuff will work'

  2. Aaron Bartell reporter

    Thinking through this again now that I know about the :file feature. Maybe the Python chroot needs to have the following and then in chroot_OPS_SC1.lst it takes care of any symbolic links associated with itself.

    ---chroot_OPS_PYTHON.lst---

    :file
chroot_OPS_SC1.lst

    Specifically, the following would be added to the yet-to-be-created :ln section of chroot_OPS_SC1.lst:

    $ ls -la /QOpenSys/usr/bin | grep 'SC1' | awk '{print $0;}'
lrwxrwxrwx    1 qsys     0                88 May 21 22:34 c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/bin/c_rehash
lrwxrwxrwx    1 qsys     0                86 May 21 22:34 openssl -> ../../QIBM/ProdData/SC1/OpenSSL/bin/openssl
lrwxrwxrwx    1 qsys     0                78 May 21 22:34 scp -> ../../QIBM/ProdData/SC1/OpenSSH/bin/scp
lrwxrwxrwx    1 qsys     0                80 May 21 22:34 sftp -> ../../QIBM/ProdData/SC1/OpenSSH/bin/sftp
lrwxrwxrwx    1 qsys     0                78 May 21 22:34 slogin -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh
lrwxrwxrwx    1 qsys     0                78 May 21 22:34 ssh -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh
lrwxrwxrwx    1 qsys     0                86 May 21 22:34 ssh-add -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh-add
lrwxrwxrwx    1 qsys     0                90 May 21 22:34 ssh-agent -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh-agent
lrwxrwxrwx    1 qsys     0                92 May 21 22:34 ssh-keygen -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh-keygen
lrwxrwxrwx    1 qsys     0                94 May 21 22:34 ssh-keyscan -> ../../QIBM/ProdData/SC1/OpenSSH/bin/ssh-keyscan
$ ls -la /QOpenSys/usr/lib | grep 'SC1' | awk '{print $0;}'
lrwxrwxrwx    1 qsys     0                94 May 21 22:34 libcrypto.a -> ../../QIBM/ProdData/SC1/OpenSSL/lib/libcrypto.a
lrwxrwxrwx    1 qsys     0                88 May 21 22:34 libssl.a -> ../../QIBM/ProdData/SC1/OpenSSL/lib/libssl.a
lrwxrwxrwx    1 qsys     0                80 May 21 22:34 libz.so -> ../../QIBM/ProdData/SC1/zlib/lib/libz.so

    Is that basically what you are getting at? Thoughts?

  3. Former user Account Deleted

    First, yes, i agree completely chroot_OPS_SC1.lst should take care of itself, including, exporting any symbolic links into the 'public domain'.

    Second, ok, you voted down my idea IBM i customers can handle setting PATH and LIBPATH on thier own to reach things like SC1 functions (*).

    ()BTW -- Cough, cough, gasp, IBM i peachy fine to set LIBL (normal), but fatal flaw in IBM i product set LIBPATH (black hole of dispair) ... so ... cough ... please allow just a little wine and cheese here.

  4. Aaron Bartell reporter

    Second, ok, you voted down my idea IBM i customers can handle setting PATH and LIBPATH on thier own to reach things like SC1 functions (*).

    I think developers would be best served if we made chroot environments as much like non-chroot as possible right out of the gates. Because if they are using chroot to do automated testing then portions of the testing could give false positives based on differences in chroot vs. non-chroot.

    With that said, I do agree that PATH isn't as common knowledge (yet) as *LIBL.

    With that said (part II), if we do go down the route of setting PATH and LIBPATH then we should also include .profile configuration so things can be automatically set upon chroot'ing or ssh'ing into a chroot.

  5. Former user Account Deleted

    Oh boy, my machine has a openssl version in the symbolic link openssl-0.9.8j (see below). 

     > ls -l /QOpenSys/usr/bin/c_rehash
   lrwxrwxrwx    1 qsys     0               118 Aug 22 2011  /QOpenSys/usr/bin/c
   _rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
   $

    Therefore we can not build a 'generic' chroot_OPS_SC1.lst.

    #
# links list (must occur after minimal copy above)
# ( chroot $CHROOT_DIR /bin/sh -c "ln -sf $1 $2" )
#
:ln
/QOpenSys/usr/bin/c_rehash /QOpenSys/QIBM/ProdData/SC1/OpenSSL/bin/c_rehash
/QOpenSys/usr/bin/openssl /QOpenSys/QIBM/ProdData/SC1/OpenSSL/bin/openssl

    Mmm ...

  6. Aaron Bartell reporter

    My v7.2 machine:

    $ ls -l /QOpenSys/usr/bin/c_rehash
lrwxrwxrwx    1 qsys     0                88 May 21 22:34 /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/bin/c_rehash

    My v7.1 machine:

    $ ls -l /QOpenSys/usr/bin/c_rehash
lrwxrwxrwx    1 qsys     0               118 Jan 16 2012  /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehas

    Wondering if we can use uname (or other) to determine IBM i version.release and then occupy a variable based on that. For example:

    :ln
/QOpenSys/usr/bin/c_rehash /QOpenSys/QIBM/ProdData/SC1/OpenSSL$openssl_path/bin/c_rehash

    And then in chroot_setup.sh have something like 

    if [os_version -eq '72']
  openssl_path = ''
else
  openssl_path = '/openssl-0.9.8j'
fi
  7. Former user Account Deleted

    Alternative: :ln_rel (link relative ../../), therein perform 'relative links' without entering chroot.

    I am thinking we can use utility 'find', using only utility 'cd' to move around 

    bash-4.3$ ls -l /QOpenSys/usr/bin/c_rehash
lrwxrwxrwx    1 qsys     0               118 Jul 11 2014  /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash

bash-4.3$ cd /QOpenSys/ranger
bash-4.3$ find QOpenSys/QIBM/ProdData/SC1 -name "c_rehash" 
QOpenSys/QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
  8. Former user Account Deleted

    Or maybe 'ls -l' is enough all by itself ...

    bash-4.3$ ls -l /QOpenSys/usr/bin/c_rehash
lrwxrwxrwx    1 qsys     0               118 Jul 11 2014  /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
  9. Former user Account Deleted

    Something like this should work (maybe) ...

    #
# relative links list
# (
#   ls -l /QOpenSys/usr/bin/c_rehash
#   /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
#   cd $CHROOT_DIR/QOpenSys/usr/bin/
#   ln ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash c_rehash
# )
#
:ln_rel
/QOpenSys/usr/bin/c_rehash
  10. Aaron Bartell reporter

    It's possible ls -l could work. My only concern is if /QOpenSys/usr/bin|lib has been altered with other symlinks (i.e. PowerRuby ships their own openssl now). That's why it's kind of nice to exactly spell out symbolic linkage. Though with that said I've now created a double standard because what's in chroot will be different than non-chroot. ug. Time to go find a bag of sour gummy worms (my vice). [thinking] So the issue would be if you were creating a Python chroot environment and blindly resolved ...SC1/OpenSSL... symlinks, which in turn could create a symlink to /PowerRuby/... and would break once inside chroot because we didn't copy /PowerRuby.

  11. Former user Account Deleted

    Bingo ... it works ...

    $ which ssh
/QOpenSys/usr/bin/ssh
$ ls -l /QOpenSys/usr/bin/ssh
lrwxrwxrwx    1 adc      0               106 Jul 27 16:10 /QOpenSys/usr/bin/ssh -> ../../QIBM/ProdData/SC1/OpenSSH/openssh-4.7p1/bin/ssh
$ ssh adc@lp0264d
Could not create directory '/QOpenSys/ranger/./home/RANGER//.ssh'.
The authenticity of host 'lp0264d (9.5.158.38)' can't be established.
ECDSA key fingerprint is c9:7d:51:ea:19:52:20:15:d9:9e:5d:57:b7:8d:12:b5.
Are you sure you want to continue connecting (yes/no)? yes
Failed to add the host to the list of known hosts (/QOpenSys/ranger/./home/RANGER/.ssh/known_hosts).
adc@lp0264d's password: 
Welcome to LP0264D
$ exit
Connection to lp0264d closed.

    New :ln_rel durective ...

    # Command:
#   > ./chroot_setup.sh chroot_OPS_SC1.lst /QOpenSys/root_path

#
# copy dirs list
# (
#  mkdir -p $CHROOT_DIR/path
#  cp -R  /path/* $CHROOT_DIR/path/.
# )
#
:cp_dir
/QOpenSys/QIBM/ProdData/SC1

#
# relative links list
# (
#   ls -l /QOpenSys/usr/bin/c_rehash
#   /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
#   cd $CHROOT_DIR/QOpenSys/usr/bin/
#   ln ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash c_rehash
# )
#
:ln_rel
/QOpenSys/usr/bin/c_rehash
/QOpenSys/usr/bin/openssl
/QOpenSys/usr/bin/scp
/QOpenSys/usr/bin/sftp
/QOpenSys/usr/bin/slogin
/QOpenSys/usr/bin/ssh
/QOpenSys/usr/bin/ssh-add
/QOpenSys/usr/bin/ssh-agent
/QOpenSys/usr/bin/ssh-keygen
/QOpenSys/usr/bin/ssh-keyscan
/QOpenSys/usr/lib/libcrypto.a
/QOpenSys/usr/lib/libssl.a
/QOpenSys/usr/lib/libz.so
  12. Former user Account Deleted

    Looks like this when it is running (OUTSIDE chroot) ...

    bash-4.3$ ./chroot_setup.sh chroot_OPS_SC1.lst /QOpenSys/ranger
:
... copy completed ...
:
cd /QOpenSys/ranger/QOpenSys/usr/bin
ln -sf ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash c_rehash
:
ln -sf ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/lib/libcrypto.a libcrypto.a
cd /QOpenSys/ranger/QOpenSys/usr/lib
ln -sf ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/lib/libssl.a libssl.a
cd /QOpenSys/ranger/QOpenSys/usr/lib
ln -sf /QOpenSys/QIBM/ProdData/SC1/zlib/zlib-1.2.3/lib/libz.so libz.so
  13. Former user Account Deleted

    My only concern is if /QOpenSys/usr/bin|lib has been altered with other symlinks (i.e. PowerRuby ships their own openssl now).

    Well, i am thinking with our 'golden rule', where each xxx.lst takes care of itself, chroot_OPS_SC1.lst did the right thing. Therefore, i get PowerRuby has it's own openssl (SC1), but it must end up in a different location other than SC1 symbolic links, yes???? ... somewhere in /PowerRuby, yes??? ... therefore no 'interference' with PASE ("the law"), yes???

  14. Former user Account Deleted

    Ok, fixed python chroot links (not fully tested). 

    # Command:
#   > ./chroot_setup.sh chroot_OPS_PYTHON.lst /QOpenSys/root_path

#
# copy dirs list
# (
#  mkdir -p $CHROOT_DIR/path
#  cp -R  /path/* $CHROOT_DIR/path/.
# )
#
:cp_dir
/QOpenSys/QIBM/ProdData/OPS/Python3.4
/QOpenSys/QIBM/ProdData/OPS/Python-pkgs

#
# relative links list
# (
#   ls -l /QOpenSys/usr/bin/c_rehash
#   /QOpenSys/usr/bin/c_rehash -> ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash
#   cd $CHROOT_DIR/QOpenSys/usr/bin/
#   ln ../../QIBM/ProdData/SC1/OpenSSL/openssl-0.9.8j/bin/c_rehash c_rehash
# )
#
:ln_rel
/QOpenSys/usr/bin/2to3
/QOpenSys/usr/bin/2to3-3.4
/QOpenSys/usr/bin/easy_install-3.4
/QOpenSys/usr/bin/easy_install3
/QOpenSys/usr/bin/pip3
/QOpenSys/usr/bin/pip3.4
/QOpenSys/usr/bin/pydoc3
/QOpenSys/usr/bin/pydoc3.4
/QOpenSys/usr/bin/python3
/QOpenSys/usr/bin/python3.4
/QOpenSys/usr/bin/pyvenv
/QOpenSys/usr/bin/pyvenv-3.4
/QOpenSys/usr/lib/libpython3.4m.so
  15. Former user Account Deleted

    Added a new directive :ln_fix_rel for fixing relative links in node js from IBM (you also requested).

    # Command:
#   > ./chroot_setup.sh chroot_OPS_NODE.lst /QOpenSys/root_path
# Note:
#   This file assume prior chroot_minimal.lst (or uncomment below)
#     > ./chroot_setup.sh chroot_minimal.lst /QOpenSys/root_path
#

#
# copy dirs list
# (
#  mkdir -p $CHROOT_DIR/path
#  cp -R  /path/* $CHROOT_DIR/path/.
# )
#
:cp_dir
/QOpenSys/QIBM/ProdData/Node


#
# fix relative links list
# (
#   cd $CHROOT_DIR/QOpenSys/usr/bin/
#   ln node ../../QIBM/ProdData/Node/bin/node
# )
#
:ln_fix_rel
/QOpenSys/usr/bin/node ../../QIBM/ProdData/Node/bin/node
/QOpenSys/usr/bin/npm  ../../QIBM/ProdData/Node/bin/npm
/QOpenSys/usr/lib/libgcc_s.a  ../../QIBM/ProdData/Node/lib/libgcc_s.a
/QOpenSys/usr/lib/libstdc++.a  ../../QIBM/ProdData/Node/lib/libstdc++.a
  16. Aaron Bartell reporter

    Therefore, i get PowerRuby has it's own openssl (SC1), but it must end up in a different location other than SC1 symbolic links, yes???? ... somewhere in /PowerRuby, yes???

    Here's an example concerning the node binary included with PowerRuby:

    $ ls -l /QOpenSys/usr/bin/node
lrwxrwxrwx    1 aaron    0                68 May 22 21:18 /QOpenSys/usr/bin/node -> ../../../PowerRuby/prV2R1/bin/node
$ dump -H /QOpenSys/usr/bin/node

/QOpenSys/usr/bin/node:

                        ***Loader Section***
                      Loader Header Information
VERSION#         #SYMtableENT     #RELOCent        LENidSTR
0x00000001       0x0000010c       0x0000af08       0x00000093

#IMPfilID        OFFidSTR         LENstrTBL        OFFstrTBL
0x00000006       0x00084da0       0x00000772       0x00084e33


                        ***Import File Strings***
INDEX  PATH                          BASE                MEMBER
0      /PowerRuby/oss/lib:/usr/lib:/lib             <-------------the important bit
1                                    libc.a              shr.o
2                                    libpthreads.a       shr_xpg5.o
3                                    libstdc++.a         libstdc++.so.6
4                                    libpthreads.a       shr_comm.o
5                                    libgcc_s.a          shr.o

    Further, IBM's latest update of 5733OPS includes symbolic links for the node binary, but they are stored in /usr/bin/node and not /QOpenSys/usr/bin/node.

    $ ls -all /usr/bin/node
lrwxrwxrwx    1 qsys     0                74 Jul 21 01:15 /usr/bin/node -> /QOpenSys/QIBM/ProdData/Node/bin/node

    This is why I was concerned about doing a "blind" resolution of binaries(i.e. ls -l). SSL/OpenSSL is safe for now because PowerRuby seemingly only includes libraries (i.e. /PowerRuby/oss/lib) and not callable programs (i.e. ssh-keygen). I am not sure if this is cause for concern or not.

  17. Former user Account Deleted

    1) As far as 'blind' (ls -l), if we do anything wlese we will have a version of chroot_OPS_SC1_versionx.x.lst for everything ... not at all desirabble. 2) PowerRuby will have to be treated special, and, we must rely on thier good judgement for taking over symbolic links for PASE related tools (node, opensssl, etc.).

  18. Former user Account Deleted

    We have :ln (match pase root-2-root), :ln_rel (ls -1 relative), and :ln_fix_rel (DYI), can we close this issue now?

  20. Log in to comment
Assignee
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
0
Jira: the preferred issue tracker for Bitbucket. Join the team!