pkg_setup.sh doesn't work right if you already have a /opt directory

[Brandon Peterson]

When initially setting up RPM, it fails if you already have a /opt directory.

In pkg_setup.sh on line 203 it should check if /opt already exists as a directory, if so it should do something like this:

if [ -d /opt ]; then ln -s /QOpenSys/opt/freeware /opt/freeware; else ln -s /QOpenSys/opt /opt; fi

instead of this:

ln -s /QOpenSys/opt /opt

[Aaron Bartell]

Thanks for logging this @nemo3xm. Could you run the following so we can learn what has already started using /opt?

ls -al /

I ask because there are other areas that rely on opt being the starting point vs. opt/freeware, as show below.

function package_setup_rpm {
  cdhere=$(pwd)
  echo "setup $RPM_RTE ..."
  restore -xvqf $RPM_RTE
  cd $cdhere
  mkdir /QOpenSys/opt
  cp -R usr/opt/* /QOpenSys/opt/.
  rm -R usr
  ln -s /QOpenSys/opt /QOpenSys/var/opt    <---- here
  ln -s /QOpenSys/opt /opt
  mkdir /var
  ln -s /QOpenSys/var/opt /var/opt     <---- here
  ln -s /opt/freeware/bin/rpm /usr/bin/rpm
  cd /opt/freeware/lib
  ln -s libpopt.so.0.0.0 libpopt.so
  ln -s librpm.so.0.0.0 librpm.so
  ln -s librpmbuild.so.0.0.0 librpmbuild.so
  cd $cdhere
  echo "setup $RPM_WGET ..."
  rpm --ignoreos --ignorearch --nodeps --replacepkgs -hUv $RPM_WGET
  if [ -f /opt/freeware/bin/wget400 ]
  then
    wget400 --version
  else
    cp /opt/freeware/bin/wget /opt/freeware/bin/wget400
    wget400 --version
  fi
}

[Brandon Peterson]

Hey Aaron,

In our case we have an Apache Tomcat server that we installed under /opt. It's been a few years and at this point don't remember what the reasoning was for there vs /usr/local for example. We're probably going to upgrade it soon and we'll move it to /QOpenSys/opt at that point.

There wasn't anything else on our system that conflicted, but I agree with you that some of the other things might be vulnerable to things on other people's servers.

Brandon

[Former user]

Well, i don't know if this actually fails or simply reports and error and continues. However, either way any /opt that exists may be simply unworkable with these add on components (rpms). Aka, best best outcome may indeed be fail. Why? Because much of c language objects of Linux and AIX require a case sensitive file system. That is objects like libC.a and libc.a can be involved in any given load of of these applications. Now, in truth, i don't know if there are any shared objects that would occupy the same IFS space, aka, same name due to case insensitive /opt (bad news), but we are definitely rolling the gambling dice to encourage this bad /opt behaviour.

What would i do???

I would move the existing /opt to /QOpenSys/opt ...

> cd /
> mv opt /QOpenSys/.

... cross my fingers nothing goes bad.

[Brandon Peterson]

I am planning on moving the opt folder as you suggested when we do the Tomcat upgrade, but I think at minimum the pkg_setup.sh script should give an error or something if /opt is a directory (not a symlink).

[Former user]

I thought you said it failed???? Where, we hope for outcome fail (most likely). What???

[Brandon Peterson]

I guess I should have added some more info - it does actually fail. For example the symlinks on the libraries in /opt/freeware/lib don't happen. If you changed them to not depend on the /opt symlink then that would be better.

For example if you already have a /opt directory, here's what happens on the link:

bash-4.3$ cd /

bash-4.3$ ls -la | grep opt
drwxrwsrwx    4 bsp      0              8192 Nov  4 16:53 opt

bash-4.3$ ln -s /QOpenSys/opt /opt

bash-4.3$ ls -la /opt
total 1136
drwxrwsrwx    4 bsp      0              8192 Nov  8 15:00 .
drwxrwsrwx   64 qsys     0            548864 Nov  8 12:44 ..
drwxrwsrwx   10 bsp      0              8192 Aug 25 2014  apache-tomcat-7.0.50
lrwxrwxrwx    1 bsp      0                26 Nov  8 15:00 opt -> /QOpenSys/opt

So then from that point on everything else that depends on /opt being linked to /QOpenSys/opt fails. I had to read through the pkg_setup.sh script and figure out what it wanted for everything to fix it. It would be better if it would just give an error from the get-go saying it won't work if you already have /opt as a directory, or even do the alternate link command as I originally suggested.

[Former user]

If you changed them to not depend on the /opt symlink then that would be better.

Short answer: No. We are using binaries from other platforms (AIX), therefore we are playing ball with thier idea of correct file system geometry for 'add on' functions of 'questionable' lineage, aka, Open Source (not developed Austin Texas home of IBM AIX).

Longer answer: Maybe. IF we decided to PTF everything, then we will likely change the destination to be /QOpenSys/QIBM/ProdData/OPS/tools or something like that.

I guess I should have added some more info - it does actually fail

Mmm ... now we are in a very grey area. Where may be best to have the install limp to victory, spitting errors all the way.

Ok, i am going to ask you to do a bit less lobbying here and stay with facts. Did actual utilities of the RPMs actually work even though your machine used /opt, not a /QOpenSys/opt???

BTW -- There is a better than average chance these things did work. That is to say, did utilities like bash, tar, gcc actually run ok???

[Brandon Peterson]

Tony, the only thing I'm trying to lobby for here is it not to break halfway through installing and setting up rpm. I apologize if I'm not coming across in the easiest way to understand. I totally get that you guys need to decide what you want to do with this, I was just suggesting one possible option.

Many things like tar and bash I already had working from installing 5733OPS and all the options. So I was just trying to get some extras going here and I didn't test anything other than RPM & wget before fixing all the sym-links and installing wget.

RPM didn't work for two reasons: 1 the library sym-links (libpopt.so etc.) weren't in the right place (/opt/freeware/lib), instead they were in the /QOpenSys/QIBM/ProdData/OPS/GCC directory with no real libraries to point to in that directory. Also the LIBPATH was non-functional since /opt/freeware/lib wasn't a valid path at that point.

WGET wasn't installed yet since RPM didn't work, so I had to do that manually.

Nothing else happend in pkg_setup - no other packages got installed or anything. It bailed out because rpm and wget weren't working.

[Former user]

instead they were in the /QOpenSys/QIBM/ProdData/OPS/GCC directory ...

Actually ... copy worked (exhibit a), but symbolic link failed (exhibit b), because your machine already occupied /opt (exhibit c). By luck or mercy, we did not use ln -sf to force taking over your existing /opt. BTW -- I believe we will call this good luck due to great programming (cough).

function package_setup_rpm {
: ... stuff
:
  mkdir /QOpenSys/opt
  cp -R usr/opt/* /QOpenSys/opt/.  <-- exhibit a
  rm -R usr
  ln -s /QOpenSys/opt /QOpenSys/var/opt
  ln -s /QOpenSys/opt /opt < -- exhibit b
:

your machine:
/opt/tomcat-stuff <-- exhibit c

Anyway, you, human, needed to do something ... mmm ... thinking ... mmm ... we may be able to detect this condition and simply sym-link 'freeware' and subdir(s) of '/QOpenSys/opt(/freeware)' into your existing /opt and everything would work. Mmm ... mmm ... i dunno ... to hack or not to hack ... when this is already a hack to begin. Do two hacks make a right (stuff would work)??? Or is one hack enough (aka, /QOpenSys/opt because IBM i is not default root case sensitive)??? Mmmm ... mmm ... mmm ... thinking.

Aaron, assuming you understand my thinking, what say you???

[Former user]

... suggesting one possible option ...

Suggest away. This the nature of Open Source. Good news, you get to talk directly to the geeks (me). Bad news, you get to talk directly to the geeks (me).

FYI -- we know that this set of scripts is not the final answer in Open Source technology. As you can see in this project we have begun to play with yum/rpm to be more like Linux. Where, yum is like 'mother', she always takes care to install the correct rpms in the correct locations. Thanks Mom!

FYI (2) -- However, even if/when yum/mother works, we will also have to deal with people that only want PTFs. People have feelings. Feelings are always valid. (Even when Open source geeks think mother yum is much prettier).

[Aaron Bartell]

Aaron, assuming you understand my thinking, what say you???

I am thinking we need a "health check" part to the pkg_setup.sh that will let us know of issues out of the gates and if the "health check" fails then the only way to force the install is to add a --force | -f option to the pkg_seupt.sh call. This gives user fair warning and also gives them a way to continue if they want to risk it.

If the -f option is used on the pkg_setup.sh call then we will still need to massage things, like symlink to /opt/freeware instead of /opt.

Note I am not recommending the -f flag on the ln command. If somebody needs that then they can read the short amount of pkg_setup.sh code and do setup manually. If we had that in place today it would have wiped out the Tomcat install and Tony/Aaron get hate mail.

And finally, @nemo3xm, I highly recommend you learn about chroot (aka "IFS Containers"). It. Will. Save. Your. Bacon.

• Article: IFS Containers Part 1
• Article: IFS Containers Part 2 (published this Friday in MCPressOnline)
• Article: IFS Containers Part 3 (published next month in MCPressOnline)

None of these problems exist in IFS Containers because it is a "pristine" environment. You can screw up an IFS Container and simply delete it. Screw up the base of PASe and you're screwed for many more hours.

[Brandon Peterson]

Definitely ln -sf is bad. If there is any uneasiness with my original suggestion then the health check would be a great option, probably better even. I have some of what's needed already and can do more on it if you guys want me to do a pull request.

[Former user]

uneasiness with my original suggestion

Not sure which suggestion.

1) If you changed them to not depend on the /opt symlink then that would be better.

Confusion exists on AIX RPM file system geometry situation. Here is the short version. AIX toolbox and perzl RPMs are compiled with LIBPATH search for -L /opt/freeware/lib. Compile -L essentially 'hard codes' search location for dependent shared libraries/objects to /opt/freeware/lib/libthing.a. IBM i PASE/AIX loader follows this compile libpath suggestion in absence of LIBPATH env var. That is, user may set environment variable LIBPATH to override internal binary libpath search (similar fashion to "LIBL"). However, unfortunately, AIX RPMs also contain 'hard coded' locations to /opt/freeware/bin scripts, etc. Last, AIX RPMs target /opt/freeware geometry on installation as a part of AIX RPM meta-data.

Ignoring environment variable LIBPATH discussion (above), only real solution is to recompile everything manually setting the binary LIBPATH to IBM i PTF convention of using a compile option -L /QOpenSys/QIBM/ProdData/OPS/tool, as is present in IBM i supplied binaries (PTF).

Bottom line on AIX binary RPM /opt/freeware ...

I recommend you use AIX Toolbox and Perzl RPMs in a chroot location (see Aaron previous post). BTW -- If I may, please direct your attention to the name of this project is ibmichroot. Personally, my philosophy is freedom, aka, I allow options to do anything, wherein, two scripts chroot_setup.sh and pkg_setup.sh can be used independently (chroot or not).

2) In pkg_setup.sh on line 203 it should check if /opt already exists as a directory, if so it should do something like this:

if [ -d /opt ]; then ln -s /QOpenSys/opt/freeware /opt/freeware; else ln -s /QOpenSys/opt /opt; fi

Ok.

[Former user]

Ok, change made. Please see pkg/pkg_setup.sh. Thanks for your help Brandon.