Copy IBM-provided Python 2.7, Node 4, and Git into chroot environment
Issue #40
closed
Is there any plan to provide scripts for this? I created some here:
https://bitbucket.org/DavidRusso/ibmichroot/overview
I've tested on 7.3 with the latest PTFs and they seem to do the trick. I have yet to test on 7.2 and 7.1.
The only problem I've found so far is that the /QOpenSys/QIBM/ProdData/OPS/Node4/bin/nodever.sh script doesn't work out of the box in the minimal chroot environment, due to the hashbang. The script in the Node 0.12.x install directory has the same issue.
If there is any interest in including these, I can open a pull request. I'm also happy to make adjustments if there is a different plan on how to include these.
Comments (17)
-
-
Account Deleted
Thinking out loud guys ...
We maybe could do better OPS CHROOT items. That is to say, maybe we should allow a new directive 'manifest' in the .lst files. Seems to me that tool like chroot_setup.sh would be able to read the PTF manifest and copy all the right stuff into a chroot 'hole'.
> cd /QOpenSys/QIBM/ProdData/OPS $ > head .qptfinf.QNODE4.dat /QOpenSys/QIBM/ProdData/OPS/Node4/bin/npm /QOpenSys/QIBM/ProdData/OPS/Node4/notices.txt /QOpenSys/QIBM/ProdData/OPS/Node4/bin/node /QOpenSys/QIBM/ProdData/OPS/Node4/bin/nodever.sh /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/.npmignore /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/.mailmap /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/.travis.yml /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/AUTHORS /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/CHANGELOG.md /QOpenSys/QIBM/ProdData/OPS/Node4/lib/node_modules/npm/CONTRIBUTING.md $Of course the file is in EBCDIC for PTF tools, but, still, has all the files and symbolic links listed. After solving that issue, seem like a easier design for the chroot_OPS_x_.lst items (not much work).
What do you think???
-
I like that a lot. I didn't realize those files existed. Do they already remain there, or are there some
GO PTFcommands an admin might take that would clean them out? -
Account Deleted
I believe they manifest comes/goes with PTF install or remove. That is, if you have the PTF, then you have the manifest, because used in both the PTF install and remove process. The file has a dot prefix to 'hide' file from wandering admin minstrels (cough), so it should stay put with the PTF. So, summary, if you have PTF, you have the 'manifest'.
Next question (unasked) ... chroot friendly ... PTFs on PTFs for each option/product in OPS?
Nope. That is, last PTF wins, and, overlays manifest. This is nature of installing into root of the machine. Frankly, personally, we-chroot-people will better off if we can get yum/rpm working to allow you to keep 'versions' of RPMs around (*).
(*) These view are mine alone, not IBM. I am not a legal representative of IBM. We can make the technology work, but i do not understand the legal aspects of RPM distribution/hosting (TBD).
-
I agree the last PTF wins (makes sense).
I also agree with the hosted rpms being the way we can get the multiple versions working. We'll need to think through that because Node.js v4.4.3 might be released and then there is a PTF fix that doesn't bump the Node.js version. So we most likely need to append the PTF id to the end of the yum package so people can operate on the PTF-id-versions independently. (hopefully I explained that well).
-
Account Deleted
We'll need to think through ... PTF fix that doesn't bump the Node.js version.
This is known as a flagrant foul in RPM land. Aka, technical foul, where player must be removed from the game. Boss must dock pay, season ending, no millions for you. Wait .. i may have wondered off topic.
Anyway, i am not sure delta RPMs can save us here ... so think through is indeed something to ponder. Maybe these are stuff added with shell script (oh yuck).
-
Account Deleted
BTW -- i am not giving up. We may find a rpm way here.
-
It may be a flagrant foul in rpm land, but it's how some languages do their updates...
$ ruby -v ruby 2.1.5p273 (2014-11-13 revision 48405) [powerpc-aix6.1]Note the
p273in the above. It allows developers to know of minuscule changes. Node.js from IBM has separate incremental updates because there are Power specific fixes (SI59729). So even though I can do the below on IBM i, Mac, Linux, and Windows, it means something potentially entirely different on IBM i because of PTFs. If we don't store separate versions of Node.js based on the aforementioned PTF then we wouldn't know whether the performance issue was fixed without going through manual effort. I vote the effort is done once by us and then everyone benefits.$ node -v v4.4.3 -
Account Deleted
I think i agree ...
I don't want to write an entire yum/rpm meets IBM i PTF design document in this issue. Anyway, noted, we need to handle this while we test out our yum part of the project (*).
Back to topic.
Again, I think we both agree we could do a better job of chroot_OPS_things.lst using a 'manifest' keyword. This fits chroot_setup.sh anyway, that is, set-up a new chroot. Although, i also find, i am adding to a existing chroot using chroot_setup.sh. Anyway, I believe many of the tasks beyond chroot_minimal.lst, are probably better served with yum/rpms (when/if we get that running).
(*) -- OTHER PEOPLE reading -- yum part of project is not updated yet, so do not bother downloading. To date, we only put a small thing from aix to see if we could get it to work at all. To really get it working, you need the other patches and tools we are working on.
-
Account Deleted
Update --
Try 1 ... appears we will have to use a slightly different method. I added keyword 'manifest', but 'pure shell' fork/exec in chroot_setup.sh was too slow (way too slow).
Try 2 ... we will use program to generate new chroot_OPS_xxxx.lst files based on the manifest. If this works we can automate generating new files to be pushed into this git project. Also, when we fall behind latest PTFs, customer can gen the new chroot_OPS_xxxx.lst files based on the manifest, aka, less work for David Russo. (Theory anyway).
-
Account Deleted
Ok i added a new utility to generate chroot_gen_OPS_xxx.lst. I also included generated files from my machine (PTFs on).
$ ./gen_chroot_OPS_lst ---Directory ... /QOpenSys/QIBM/ProdData/OPS/.--- Found ... .qptfinf.QPYTHON.dat Reading ... .qptfinf.QPYTHON.dat Processing ... .qptfinf.QPYTHON.dat Output file for .qptfinf.QPYTHON.dat is chroot_gen_OPS_Python3.4.lst Writing ... ./chroot_gen_OPS_Python3.4.lst ... and so on ... -
I was able to successfully run your utility. #humanunittesting
I noticed the
/yumdirectory was removed from the repo. Did you mean to do that?% gen_chroot_OPS_lst ---Directory ... /QOpenSys/QIBM/ProdData/OPS/.--- Found ... .qptfinf.QPY2.dat Reading ... .qptfinf.QPY2.dat Processing ... .qptfinf.QPY2.dat Output file for .qptfinf.QPY2.dat is chroot_gen_OPS_Python2.7.lst Writing ... ./chroot_gen_OPS_Python2.7.lst Found ... .qptfinf.QPYTHON.dat Reading ... .qptfinf.QPYTHON.dat Processing ... .qptfinf.QPYTHON.dat Output file for .qptfinf.QPYTHON.dat is chroot_gen_OPS_Python3.4.lst Writing ... ./chroot_gen_OPS_Python3.4.lst Found ... .qptfinf.QCI.dat Reading ... .qptfinf.QCI.dat Processing ... .qptfinf.QCI.dat Output file for .qptfinf.QCI.dat is chroot_gen_OPS_cloudinit.lst Writing ... ./chroot_gen_OPS_cloudinit.lst Found ... .qptfinf.QPYDB.dat Reading ... .qptfinf.QPYDB.dat Processing ... .qptfinf.QPYDB.dat Output file for .qptfinf.QPYDB.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QPYTLKT.dat Reading ... .qptfinf.QPYTLKT.dat Processing ... .qptfinf.QPYTLKT.dat Output file for .qptfinf.QPYTLKT.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QPYFLIP.dat Reading ... .qptfinf.QPYFLIP.dat Processing ... .qptfinf.QPYFLIP.dat Output file for .qptfinf.QPYFLIP.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QPYBTL.dat Reading ... .qptfinf.QPYBTL.dat Processing ... .qptfinf.QPYBTL.dat Output file for .qptfinf.QPYBTL.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QPYDB2.dat Reading ... .qptfinf.QPYDB2.dat Processing ... .qptfinf.QPYDB2.dat Output file for .qptfinf.QPYDB2.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QPYTLKT2.dat Reading ... .qptfinf.QPYTLKT2.dat Processing ... .qptfinf.QPYTLKT2.dat Output file for .qptfinf.QPYTLKT2.dat is chroot_gen_OPS_Python-pkgs.lst Writing ... ./chroot_gen_OPS_Python-pkgs.lst Found ... .qptfinf.QNODE4.dat Reading ... .qptfinf.QNODE4.dat Processing ... .qptfinf.QNODE4.dat Output file for .qptfinf.QNODE4.dat is chroot_gen_OPS_Node4.lst Writing ... ./chroot_gen_OPS_Node4.lst Found ... .qptfinf.QLIBGCC482.dat Reading ... .qptfinf.QLIBGCC482.dat Processing ... .qptfinf.QLIBGCC482.dat Output file for .qptfinf.QLIBGCC482.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QGIT.dat Reading ... .qptfinf.QGIT.dat Processing ... .qptfinf.QGIT.dat Output file for .qptfinf.QGIT.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QICONV.dat Reading ... .qptfinf.QICONV.dat Processing ... .qptfinf.QICONV.dat Output file for .qptfinf.QICONV.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QBASH.dat Reading ... .qptfinf.QBASH.dat Processing ... .qptfinf.QBASH.dat Output file for .qptfinf.QBASH.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QUNZIP.dat Reading ... .qptfinf.QUNZIP.dat Processing ... .qptfinf.QUNZIP.dat Output file for .qptfinf.QUNZIP.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QZIP.dat Reading ... .qptfinf.QZIP.dat Processing ... .qptfinf.QZIP.dat Output file for .qptfinf.QZIP.dat is chroot_gen_OPS_tools.lst Writing ... ./chroot_gen_OPS_tools.lst Found ... .qptfinf.QGCC.dat Reading ... .qptfinf.QGCC.dat Processing ... .qptfinf.QGCC.dat Output file for .qptfinf.QGCC.dat is chroot_gen_OPS_GCC.lst Writing ... ./chroot_gen_OPS_GCC.lst Found ... .qptfinf.QORION.dat Reading ... .qptfinf.QORION.dat Processing ... .qptfinf.QORION.dat Output file for .qptfinf.QORION.dat is chroot_gen_OPS_Orion.lst Writing ... ./chroot_gen_OPS_Orion.lst -
Account Deleted
gen_chroot_OPS_lst
Cool. We will have to decide if we like the outcome, but it appears to work. However not well tested (warning).
yum dir
Oops, sorry. We are joint development ibmichroot scripts. I did not mean to keep you in dark. However, we/lab are discussing if we will actually 'release' in PTF, so yum is all experimental at this point. Perhaps you can try it out and see what you think.
-
Account Deleted
Ops. The chroot_gen_OPS_xxx.lst files are not working correctly yet with symbolic links. Sorry i will look at the issue.
-
Account Deleted
Ok, fixed generator ... fix links gen_chroot_OPS_lst chroot_gen_OPS_xxxx.lst
-
Account Deleted
Also updated all chroot_gen_OPS_xxxx.lst in the project (from my machine anyway).
-
Account Deleted
- edited description
- changed status to closed
I think we are done with this issue.
- Log in to comment
@DavidRusso ! Glad to hear from you.
Yes, please issue a pull request. The only changed I'd ask you to make is to rename the existing
chroot_OPS_PYTHON.lstfile to bechroot_OPS_PYTHON3.4.lst. Concerning how planning works with this project... we've added features mostly based on real-world scenarios (i.e. if we found a need we made changes and put it in the repo).If you have the time it would be great to submit an RFE(Request For Enhancement) for this so the fix is made upstream.