Add -r to read for escape sequence purposes

[Aaron Bartell]

I came across a scenario where I needed line feed (\n) to be retained, example below:

:sh
printf "line1\nline2\nline3\n\n" > file.txt

The above won't include new lines in file.txt and instead will remove the backslash and leave the n and resulting file will have single line. To make it honor the \n I added the -r option to the read, as shown below:

  while read -r name <&3; do

This has worked for all section types that I've tried (i.e. :sh, :system, :cp, etc) but wanted to run it past you( @rangercairns) to make sure it won't have negative affects on other things I might not be thinking about.

[Former user]

No, i don't this is a good idea.

We are very dependent on end of line marker in files, and, i have no idea what will happen.

Also, we do not what ':sh: to become a language, it is just to run a few small things "one liners".

[Aaron Bartell]

Also, we do not what ':sh: to become a language, it is just to run a few small things "one liners".

Fully automated creation of even the littlest of details will be important for chroot environments to gain acceptance. If people have to remember to run five subsequent commands after a chroot environment is created then people will move on from ibmichroot. There needs to be some level of ability to automate the customization of a chroot environment after/during creation.

Maybe we could make :sh section not dependent on end-of-line markers?

SIDE NOTE: Docker, a subsequent chroot-like technology/methodology that the Linux world is adopting like wildfire, has this type of feature with their RUN command.

[Former user]

I am not completely sure what you are doing that actually needs newlines. You should probably post the action so we can talk about the issue without using abstraction. In fact, if the operation you are doing is 'common', then we should add a custom label for the activity ':task1'. Also, i think that ':sh' could call any other custom script, so the road is not blocked, but chroot_setup.sh does not have to become a language parser.

Becoming Docker ... no, we have no chance of keeping up with Docker, at best we can make it easier to do this sort of 'development chroot' activity, possibly even run some production servers in a chroot. Let's be frank, I just do not have the time to do that much work. I mean RUN command syntax can be infinite and we are not building another language parser.

[Aaron Bartell]

My specific need is as follows. This is necessary to get around the IBM i HOMEDIR dot (.) hackery so a chroot environment can communicate with remote git repos (GitHub, BitBucket).

:sh
printf "Host *\nIdentityFile /home/myuser/.ssh/id_rsa\nUserKnownHostsFile /home/myuser/.ssh/known_hosts\n\n" > mydir/QOpenSys/QIBM/ProdData/SC1/OpenSSH/etc/ssh_config

Also, i think that ':sh' could call any other custom script, so the road is not blocked, but chroot_setup.sh does not have to become a language parser.

I thought about this also, but then each time I needed to something outside of a simple sh command I would need to create a separate custom script. My .lst files are getting more complex (a good thing because I am doing some needed customization of chroot spaces) so I have some sections (:sh, :system) repeated a number of times based on when they need to happen during the chroot creation process.

I mean RUN command syntax can be infinite and we are not building another language parser.

And that's why I am suggesting we pass :sh commands through completely unprocessed and rely entirely on the current shell interpreter to process it. It is up to the .lst author to know what they are doing in the :sh section.

[Former user]

Opp, sorry, looks like 'read -r' seems to work ok. I will add to current chroot_setup.sh and test with my new chroot_minimal.lst.

BTW -- new chroot_minimal.lst, which, we needed a better minimum PASE chroot, there was just too many things missing for many operations. However, for performance, i am still stopping short of full nls copy, as that takes 'forever' on my smaller system. I intend to leave that role in chroot_nls.lst.

[Former user]

Ok, i commit/push 'read -r' and chroot_minimal.sh. I had to rebase ... but ... looks like all worked.

I am losing track in issue, 'read -r' good enough? or do you want something else?

[adc@oc7083008330 ibmichroot]$ git add .
[adc@oc7083008330 ibmichroot]$ git commit -m 'read -r and new chroot_minimal.lst better match PASE symbolic links"
> '
[master bfc49fb] read -r and new chroot_minimal.lst better match PASE symbolic links"
 4 files changed, 575 insertions(+), 71 deletions(-)
 create mode 100644 how_to_php_ZS6.txt
[adc@oc7083008330 ibmichroot]$ git push
To https://rangercairns:[e-mail]/litmis/ibmichroot.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'https://rangercairns:[e-mail]/litmis/ibmichroot.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'Note about
fast-forwards' section of 'git push --help' for details.
[adc@oc7083008330 ibmichroot]$ git pull --rebase
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 7 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (7/7), done.
From https://bitbucket.org/litmis/ibmichroot
   b751f4c..c143238  master     -> origin/master
First, rewinding head to replay your work on top of it...
Applying: read -r and new chroot_minimal.lst better match PASE symbolic links"
[adc@oc7083008330 ibmichroot]$ git push
Counting objects: 10, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 5.45 KiB, done.
Total 6 (delta 3), reused 0 (delta 0)
To https://rangercairns:[e-mail]/litmis/ibmichroot.git
   c143238..f6bbf0a  master -> master

[Former user]

i am wondering if some of your 'custom script work' is 'standard', aka, add new arron_task22_custom.lst to this project?

[Aaron Bartell]

Opp, sorry, looks like 'read -r' seems to work ok. I will add to current chroot_setup.sh and test with my new chroot_minimal.lst.

then...

I am losing track in issue, 'read -r' good enough? or do you want something else?

This is good for now. The only other outstanding thing I am trying to figure out is how to do conditional logic. More on that later once I've worked through some things. Probably will end up as a separate script outside of .lst.

However, for performance, i am still stopping short of full nls copy, as that takes 'forever' on my smaller system.

One of the primary performance hogs is a change I put in place for the global variables feature. Scanning each line for all of the global variables is making that process a lot longer. Instead we need to find a way to do a sed on the inbound .lst content. I just haven't had time to do it.

i am wondering if some of your 'custom script work' is 'standard', aka, add new arron_task22_custom.lst to this project?

All the things I am requesting for features are what others will need, though the entirety of my various .lst files wouldn't help others as it is very specific to my context. Hope that makes sense. For example, the files like how_to_php_ZS6.txt are what I am automating from start to finish. Some of that works in a .lst file and some of it is still in my Ruby scripts, and others are in my custom shell scripts (I have a lot of clean-up to do is what I am getting at).

[Former user]

Ok, are you closing this issue then???

[Aaron Bartell]

Feature implemented. Closing request.