gmake: virtual memory exhausted
I am attempting to compile db2util and am getting the following error when run within chroot:
% git clone git@bitbucket.org:litmis/db2util.git
% cd db2util
% /opt/freeware/bin/make
make: *** virtual memory exhausted. Stop.
I do not get this error outside of chroot. So that's why I am posting the issue here in the ibmichroot project vs. the db2util project.
Some AIX forums report this error with gmake v3.80 and an upgrade to v3.81 fixes it. I am running gmake v4.1 from perzl both inside and outside chroot.
The Linux solution is to increase swap file size, and many point at this tutorial on how to do that. I am not sold it is a swap file issue because it works outside of chroot and the ulimit
command delivers the same values both inside and outside chroot.
Others in the IBM i community have had this error when running gmake from QP2TERM
and it resolved itself by running from an SSH session. I am already in an SSH session.
My next plan is go deeper with dbx
. Here's my first attempt...
% dbx /opt/freeware/bin/make
Type 'help' for help.
reading symbolic information ...warning: no source compiled with -g
(dbx) run
make: *** virtual memory exhausted. Stop.
execution completed (exit code 2)
I am following the line of thinking from our original deep journey into dbx. In that scenario we were blind debugging and knew the offending function; mmap64. In this case we don't know the offending function.
I am armed with the dbxme.py
program you provided but am lost as to what I should be looking for. That's where I am hoping you can guide me concerning what to look for. Please.
Comments (19)
-
Account Deleted -
reporter I am probably missing a dependency and the current error is a red herring. What chroot config (aka
.lst
) file did you use? Or do you have a custom one? -
Account Deleted Here is what i am using in my chroot ...
[adc@oc7083008330 ~]$ ssh -X monoroot@ut28p63 monoroot@ut28p63's password: $ bash bash-4.3$ cd db2util/ bash-4.3$ ls / QOpenSys bin dev etc home lib opt sbin tmp usr var bash-4.3$ export PATH=/opt/freeware/bin:$PATH bash-4.3$ export LIBPATH=/opt/freeware/lib:/usr/lib bash-4.3$ which make /opt/freeware/bin/make bash-4.3$ ls -l /opt/freeware/bin/make -rwxr-xr-x 1 monoroot 0 211998 Oct 6 2014 /opt/freeware/bin/make bash-4.3$ make --version GNU Make 4.1 Built for powerpc-ibm-aix5.3.0.0 Copyright (C) 1988-2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. bash-4.3$ make gcc -g -I. -I/usr/include -I.. -c db2util.c gcc -g db2util.o -L. -lpthreads -liconv -ldl -lpthread -ldb400 -o db2util bash-4.3$ db2util "select * from QIWS/QCUSTCDT where LSTNAM=? or LSTNAM=?" -p Jones Vine "839283","Jones ","B D","21B NW 135 St","Clay ","NY","13041","400","1","100.00",".00" "392859","Vine ","S S","PO Box 79 ","Broton","VT","5046","700","1","439.00",".00" bash-4.3$
-
Account Deleted What chroot config
I am using ... pkg_perzl_gcc-4.8.3.lst (bitbucket ibmichroot)
-
Account Deleted BTW -- i can run xmlservice with 1.0.3 beta ... binary found at ... YIPS download. Also, note that i can run system command again, even in the chroot, because we are using server mode in the db2util command line (QSQSRVR jobs used).
bash-4.3$ db2util "call xmlservice.iplugr512k(?,?,?)" -p "*na" "*here" "<?xml version='1.0'?><xmlservice><sh>system -i 'dsplibl'</sh></xmlservice>" "<?xml version='1.0'?><xmlservice><sh> 5770SS1 V7R1M0 100423 Library List 7/25/16 15:30:52 Page 1 ASP Library Type Device Text Description QSYS SYS System Library QSYS2 SYS System Library for CPI's QHLPSYS SYS QUSRSYS SYS System Library for Users QGPL USR General Purpose Library QTEMP USR QDEVELOP USR QBLDSYS USR QBLDSYSR USR * * * * * E N D O F L I S T I N G * * * * * </sh> </xmlservice>" bash-4.3$
-
Account Deleted Hey i am noticing ...
% git clone git@bitbucket.org:litmis/db2util.git % cd db2util % /opt/freeware/bin/make
You have a percent sign for the input marker ... just checking ... you are NOT trying to compile in a green screen??? We already know this does not work (virtual memory exhausted).
Also, i tend to run with X enabled, but i have no problem without (below) ...
[adc@oc7083008330 ~]$ ssh monoroot@ut28p63 monoroot@ut28p63's password: $ bash bash-4.3$ make --version GNU Make 4.1 Built for powerpc-ibm-aix5.3.0.0 Copyright (C) 1988-2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. bash-4.3$ gcc --version gcc (GCC) 4.8.3 Copyright (C) 2013 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. bash-4.3$ cd db2util/ bash-4.3$ ls LICENSE contributors.txt db2util.c Makefile cpysqlincludes.sh db2util.o README.md db2util sqlcli1.h bash-4.3$ touch db2util.c bash-4.3$ export PATH=/opt/freeware/bin:$PATH bash-4.3$ export LIBPATH=/opt/freeware/lib:/usr/lib bash-4.3$ make gcc -g -I. -I/usr/include -I.. -c db2util.c gcc -g db2util.o -L. -lpthreads -liconv -ldl -lpthread -ldb400 -o db2util bash-4.3$
-
reporter Here is what i am using in my chroot ...
That's what I have also. I am going to incrementally build the chroot and find the issue. I have an article due (<cough>it's late</cough>) so I suppose building a chroot environment to compile db2util is the article topic :-)
i can run system command again, even in the chroot, because we are using server mode in the db2util command line (QSQSRVR jobs used).
Very cool! But I am not sure I follow how you accomplished that. Where is
system
being called in the db2util code base or examples? -
reporter You have a percent sign for the input marker ... just checking ... you are NOT trying to compile in a green screen???
I am compiling from a shell session. The
%
is the default prompt forzsh
. -
Account Deleted Try bash ...
-
Account Deleted Very cool! But I am not sure I follow how you accomplished that. Where is system being called in the db2util code base or examples?
I am inside my chroot using new db2util calling xmlservice with <sh>system </sh> ... no tricks. Again, db2util is using server mode (SQL_ATTR_SERVER_MODE), therefore my db2util inside my chroot is attaching to pre-started QSQ jobs (started outside chroot), thereby we walk outside the chroot via Db2 stored procedure call ... bingo ... we can run qsh stuff again.
db2util (in chroot) -> call xmlservice.iplugr(?,?,?) -> |(attach)| -> QSQSRVR (run xmlservice outside chroot, therefore 'system' works)
rc = SQLSetEnvAttr((SQLHENV)henv, SQL_ATTR_SERVER_MODE, (SQLPOINTER)&attr, SQL_IS_INTEGER);
-
reporter Try bash ...
Yep, tried that before creating this issue.
Btw, my simple chroot environment is working with
gmake
, so there's probably something hosed in my other more complicated one. For others visiting this thread some day, here's the steps to create a fully functioning chroot environment to compiledb2util
.$ cd /QOpenSys $ git clone git@bitbucket.org:litmis/ibmichroot.git $ cd ibmichroot $ chroot_setup.sh chroot_minimal.lst /QOpenSys/ibmichroot_spaces/db2util $ cp -R /QOpenSys/ibmichroot /QOpenSys/ibmichroot_spaces/db2util/QOpenSys/. $ chroot /QOpenSys/ibmichroot_spaces/db2util /usr/bin/sh $ cd /QOpenSys/ibmichroot $ ./pkg_setup.sh pkg_perzl_gcc-4.8.3.lst $ gmake -v GNU Make 4.1 Built for powerpc-ibm-aix5.3.0.0 Copyright (C) 1988-2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
-
Account Deleted Btw, my simple chroot environment is working with gmake
So, would appear we have partial answer, aka, something in your 'more complicated' set-up is leading to an error in the tooling. Well, i assume i don't have access to said 'more complicated' (proprietary litmis i suspect), so i cannot even hazard a guess.
-
reporter Well, i assume i don't have access to said 'more complicated' (proprietary litmis i suspect), so i cannot even hazard a guess.
It actually is just using a vanilla Litmis Space environment that I manually installed
pkg_perzl_gcc-4.8.3.lst
into. I will figure out the differences between the ibmichroot .lst file and mine.More of what I was hoping you could comment on is what steps would you take to debug the error given we have no idea what C method it is bombing on. Doing it in the open, like we did last time, will allow many other people to visit this in the future.
Do I just start doing
dbx
stepi
andregister
commands and blinding look for things that seem out of place? -
reporter I am inside my chroot using new db2util calling xmlservice with <sh>system </sh> ... no tricks.
I follow now.
Last week I needed a short term solution so I did something similar by putting a "Poor Mans Compile RPG from PASE" project together that uses the Node.js iToolKit. I will change out my poor man's version for the new db2util version once I get everything working as expected.
-
Account Deleted The science answer ... peek gnumake source code outputs message "virtual memory exhausted" misc.c . A few places misc.c related malloc (multiple places, example below). You could set a object code break in the code and watch them all. YIKES!!!
The speculation answer ... Any event, not likely malloc subsystem is in error, as 5 billion malloc hamburgers served every day (AIX and PASE). So, if malloc really ran out memory, implies something was looping like crazy or size was really, really big request. Anyway ... if loop ... which ... you may see in wrkactjob stack (depending on how fast all running).
void * xmalloc (unsigned int size) { /* Make sure we don't allocate 0, for pre-ISO implementations. */ void *result = malloc (size ? size : 1); if (result == 0) fatal (NILF, _("virtual memory exhausted")); return result; }
-
Account Deleted Well, if the above did not put you off the debug in public idea ... here is some more ...
There was also a place in read.c. We can see read.c parse_file_seq is loaded with ifdefs (compile in/out), so not sure acutally compiled into gnumake module, but anything that says 'list of' filenames, etc, environment plays a roll.
else switch (glob (name, GLOB_NOSORT|GLOB_ALTDIRFUNC, NULL, &gl)) { case GLOB_NOSPACE: fatal (NILF, _("virtual memory exhausted")); case 0:
in this function ...
/* Parse a string into a sequence of filenames represented as a chain of struct nameseq's and return that chain. Optionally expand the strings via glob(). The string is passed as STRINGP, the address of a string pointer. The string pointer is updated to point at the first character not parsed, which either is a null char or equals STOPCHAR. SIZE is how big to construct chain elements. This is useful if we want them actually to be other structures that have room for additional info. PREFIX, if non-null, is added to the beginning of each filename. FLAGS allows one or more of the following bitflags to be set: PARSEFS_NOSTRIP - Do no strip './'s off the beginning PARSEFS_NOAR - Do not check filenames for archive references PARSEFS_NOGLOB - Do not expand globbing characters PARSEFS_EXISTS - Only return globbed files that actually exist (cannot also set NOGLOB) PARSEFS_NOCACHE - Do not add filenames to the strcache (caller frees) */ void * parse_file_seq (char **stringp, unsigned int size, int stopchar, const char *prefix, int flags)
-
Account Deleted I don not have time to look into this now (vacation), so here is a debug trick for you.
You can script dbx like this ...
#!/opt/freeware/bin/python # syntax: pdbxwtach gmake import sys, os, time, fcntl import subprocess proc = subprocess.Popen(['dbx','-d 100','-I .','-I ..',sys.argv[1]], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) proc.stdin.write('stop in malloc\n') proc.stdin.write('cont\n') while True: # get output from process "Something to print" nextline = proc.stdout.readline() if 'internal error' in nextline: continue sys.stdout.write(nextline) sys.stdout.flush() if 'execution completed' in nextline: exit(0) # write 'a line\n' to the process if 'virtual' in nextline: print "****" proc.stdin.write('quit\n') if 'stopped' in nextline: print "****" proc.stdin.write('print $r3\n') proc.stdin.write('cont\n')
However when i tried this on the green screen, gmake worked ok, aka, does not put up the virtual memory message.
/QOpenSys/usr/bin/-sh > cd /tmp $ > pdbxwatch /opt/freeware/bin/gmake Type 'help' for help. reading symbolic information ... [1] stop in malloc [1] stopped in malloc at 0xd011c7e0 **** 0xd011c7e0 (malloc) 9421ffb0 stwu r1,-80(r1) 0x000002a0 [1] stopped in malloc at 0xd011c7e0 ($t1) **** 0xd011c7e0 (malloc) 9421ffb0 stwu r1,-80(r1) 0x00001400 [1] stopped in malloc at 0xd011c7e0 ($t1) **** 0xd011c7e0 (malloc) 9421ffb0 stwu r1,-80(r1) 0x0000000c [1] stopped in malloc at 0xd011c7e0 ($t1) **** 0xd011c7e0 (malloc) 9421ffb0 stwu r1,-80(r1) 0x00000188 [1] stopped in malloc at 0xd011c7e0 ($t1) ****
-
Account Deleted - changed status to closed
I am not going to chase. To wit, I am using gmake for many projects in a chroot and works fine. Aka, 1 billion hamburgers served by gmake, so i am not motivated to look for edge effects most likely user environment or usage errors.
Also not sure this belongs in ibmichroot. This project loads stuff, where we check stuff works at all (my chroots work fine), then user is on their own really.
I am closing the issue.
-
reporter For the archives...
Turns out
gmake -version
works fine inQP2TERM
withgmake
v4.0 and v4.2.1 but ~not~ withgmake
v4.1. Not taking the time to learn why, we just know v4.1 doesn't work in all circumstance.I am going to update gmake in our
xxxx.lst
configs to use 4.2.1 instead of 4.1. - Log in to comment
I am compiling inside a chroot and do not have any error.