Nginx and PHP-FPM error child exited on signal 11 (SIGSEGV)
I have spent a few days scouring the internet and still am not able to get this issue resolved. Hoping the IBMi experts can chime in with some direction. I am on IBMi V7R3.
I can get a basic PHP page with phpinfo()
to display successfully with Nginx and PHP-FPM, however when I try a database connection using the newly installed ODBC driver I get a “502 Bad Gateway” response with the error logs below:
Nginx error:
upstream prematurely closed connection while reading response header from upstream
PHP-FPM error:
WARNING: [pool www] child #### exited on signal 11 (SIGSEGV) after 24.209474 seconds from start
One suggestion online was to enable a Core dump, but I am not sure what other configurations on the IBMi are needed to get this enabled. In the php-fpm.conf file I set rlimit_core = unlimited
but the PHP-FPM logs are still not showing the Core dump info. The log should update to “… child #### exited on signal 11 (SIGSEGV - core dumped)“, but I cannot seem to get the Core dump enabled. Any suggestions? Is this possible on the IBMi?
The other online suggestions for this issue/error were the session save_path permissions, or certain PHP extensions causing issues, but I have eliminated these suggestions as possible solutions thru troubleshooting and enabling/disabling, etc…
Let me know what other information that would be helpful in troubleshooting.
Comments (9)
-
reporter -
What version of the ODBC driver are you running?
rpm -qi ibm-iaccess
-
reporter I am using ibm-iaccess 1.1.0.14-0, from a recently downloaded and reinstalled file ibm-iaccess-1.1.0.14-0.ibmi7.2.ppc64
-
Can you ensure that you have the latest libstdcplusplus6 package installed? There was a bug a while ago because we shipped the non-threaded versions and you can’t mix and match.
If that is up to date, can you also try the 1.1.0.13 driver version and see if it has the same issue?
-
reporter I am currently on libstdcplusplus6 version 6.3.0-15. There is a newer version available at version 6.3.0-25. I tried the yum update command and got the error messages below.
Downloading Packages:
ftp://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo/ppc64/libstdcplusplus6-6.3.0-25.ibmi7.2.ppc64.rpm: [Errno 14] FTP Error 451 - server did not report OK, got 451
Trying other mirror.
Error Downloading Packages:
libstdcplusplus6-6.3.0-25.ppc64: failure: ppc64/libstdcplusplus6-6.3.0-25.ibmi7.2.ppc64.rpm from ibm: [Errno 256] No more mirrors to try.
Would this version of libstdcplusplus6 (6.3.0-15) have that bug you mentioned, with non-threaded versions?
Can you let me know where to download the older version for the ibm-iaccess ODBC driver (1.1.0.13)? I can give that a try next.
Thanks for all your help, I appreciate it!
-
reporter The full backtrace shows the information on frame 28, with a couple messages about “Cannot access memory…”.
#28 0x0000000100001480 in main (argc=-2144116105, argv=0x180336277) at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/sapi/fpm/fpm/fpm_main.c:1950 primary_script = 0x700000000003380 <error: Cannot access memory at address 0x700000000003380> __bailout = {-1, -1, -1, -1, 4294971288, 1152921504606841472, 6443195896, 6443388832, -1, 0, 0, 0, 0, 0, 0, 0, -1, 0, 1, 0, 6442454672, 6442454960, 1, 1152921504606845184, 1152921504606844768, 6, 0 <repeats 19 times>, 2594077940715686000, 8, 1, 6443356696, 648535941211936184, 648535941213278296, 24, 1152921504606843552, 648535941213295936, 1152921504606843520, 648535941213828360, 1152921504606843536, 648535941212342384, 24, 0, 648535941212270576, 0, 1152921504606843584, 272756745, 1152921504606843600, 648535941213295936, 648535941213278284, 648535941213828360, 0, 648535941211803072, 1152921504606843696, 648535941211888872, 648518346344007296, 648535941213296888, 1152921504606843696, 128, 648535941212271392, 648535941211803064, 1152921504606843696, 648535941211888864, 648518346344638976, 648535941212342384, 648518346345256128, 648535941213299784, 8} exit_status = 0 use_extended_info = -2144115867 file_handle = {handle = {fd = 117440512, fp = 0x700000000077000, stream = {handle = 0x700000000077000, isatty = 0, mmap = {len = 2584, pos = 0, map = 0x0, buf = 0x700000000233000 <error: Cannot access memory at address 0x700000000233000>, old_handle = 0x0, old_closer = 0x0}, reader = @0x1800a0798: 0x100163bf0 <_php_stream_read>, fsizer = @0x1800a0780: 0x10014de90 <php_zend_stream_fsizer>, closer = @0x1800a07c8: 0x10014de10 <php_zend_stream_mmap_closer>}}, filename = 0x700000000003000 <error: Cannot access memory at address 0x700000000003000>, opened_path = 0x0, type = ZEND_HANDLE_MAPPED, free_filename = 0 '\000'} orig_optind = 0 orig_optarg = 0x0 max_requests = 0 requests = 0 request = 0x18033b670 fpm_config = 0x0 fpm_prefix = 0x30 <error: Cannot access memory at address 0x30> fpm_pid = 0x0 test_conf = -2144115994 force_daemon = -2144116010 force_stderr = 0 php_information = 12288 php_allow_to_run_as_root = 0 __func__ = 0x180000850 <_fpmmain.rw_> "" len = 134225921
-
To fix the yum issue, you’ll need to follow the steps here: https://ibmi-oss-docs.readthedocs.io/en/latest/yum/README.html?highlight=http#switching-from-ftp-to-http-s
That should definitely fix your problem. I believe 6.3.0-17 fixed the issue.
-
reporter It’s working now!!!
I successfully updated to the new ibm.repo, which in turn allowed me to successfully update libstdcplusplus6 to version 6.3.0-25.
I restarted Nginx and PHP-FPM (to be safe), and I am no longer getting “child #### exited on signal 11 (SIGSEGV)” errors.
I can now continue with the ODBC connection setup and testing.
Thank you for pointing me to the fix!
-
reporter - changed status to resolved
Update was needed to package libstdcplusplus6
- Log in to comment
I did find a file named “core” in the same directory with my php script. So even though the log did not indicate “core dumped” I do see the core file.
I installed ‘gdb’ package via yum, and got the backtrace below:
Core was generated by `php-fpm'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0900000006867524 in std::locale::operator=(std::locale const&) () from /QOpenSys/pkgs/lib/libstdc++.so.6(shr_64.o)
(gdb) bt
#0 0x0900000006867524 in std::locale::operator=(std::locale const&) () from /QOpenSys/pkgs/lib/libstdc++.so.6(shr_64.o)
#10x090000000686d134 in std::ios_base::_M_init() () from /QOpenSys/pkgs/lib/libstdc++.so.6(shr_64.o)#20x09000000067d3128 in std::basic_ios<char, std::char_traits<char> >::init () from /QOpenSys/pkgs/lib/libstdc++.so.6(shr_64.o)#30x0900000007936d30 in global constructors keyed to 65535_0__QOpenSys_jenkins_workspace_odbc_core_source_picococo.cpp_AE91A18C_0xe63b5225263dd1() from /QOpenSys/pkgs/lib/libcwbcore.so
#40x0900000007987724 in _GLOBAL__FI_libcwbcore_so () from /QOpenSys/pkgs/lib/libcwbcore.so#50x0900000007987864 in _GLOBAL__AIXI_libcwbcore_so () from /QOpenSys/pkgs/lib/libcwbcore.so#60x0900000007812234 in _GLOBAL__FI_libcwbodbc_so () from /QOpenSys/pkgs/lib/libcwbodbc.so#70x09fffffff0002668 in mod_init1 () from /QOpenSys/sbin/usla#80x09fffffff0003848 in usl_init_mods () from /QOpenSys/sbin/usla#90x09fffffff0002204 in uload () from /QOpenSys/sbin/usla#100x09000000002bc198 in load1 () from /QOpenSys/usr/lib/libc.a(shr_64.o)#110x09000000002bd514 in load () from /QOpenSys/usr/lib/libc.a(shr_64.o)#120x0900000000315a70 in loadAndInit () from /QOpenSys/usr/lib/libc.a(shr_64.o)#130x090000000035260c in dlopen () from /QOpenSys/usr/lib/libc.a(shr_64.o)#140x090000000622089c in vm_open () from /QOpenSys/pkgs/lib/libltdl.so.7(shr_64.o)#150x090000000621a324 in tryall_dlopen () from /QOpenSys/pkgs/lib/libltdl.so.7(shr_64.o)#160x090000000621cbfc in try_dlopen () from /QOpenSys/pkgs/lib/libltdl.so.7(shr_64.o)#170x090000000621d368 in lt_dlopenadvise () from /QOpenSys/pkgs/lib/libltdl.so.7(shr_64.o)#180x090000000621d530 in lt_dlopen () from /QOpenSys/pkgs/lib/libltdl.so.7(shr_64.o)#190x09000000061bf968 in odbc_dlopen () from /QOpenSys/pkgs/lib/libodbc.so.2(shr_64.o)#200x09000000061c0278 in __connect_part_one () from /QOpenSys/pkgs/lib/libodbc.so.2(shr_64.o)#210x09000000061c7350 in SQLDriverConnect () from /QOpenSys/pkgs/lib/libodbc.so.2(shr_64.o)#220x0900000006420e50 in pdo_odbc_handle_factory () from /QOpenSys/pkgs/lib/php-7.3/extensions/pdo_odbc.so#230x090000000625697c in zim_PDO_dbh_constructor () from /QOpenSys/pkgs/lib/php-7.3/extensions/pdo.so#240x00000001000d12b8 in execute_ex (ex=0x9001000a0ac8188 <_ZN17PiSvRuntimeConfig4cfg_E+456>)at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/Zend/zend_execute.c:52397
#250x00000001000d3fc8 in zend_execute (op_array=0x9001000a0ac8188 <_ZN17PiSvRuntimeConfig4cfg_E+456>, return_value=0xfffffffffff48f0)at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/Zend/zend_execute.c:60939
#260x00000001000370b0 in zend_execute_scripts (type=-1599307384, retval=0xfffffffffff48f0, file_count=0)at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/Zend/zend.c:1568
#270x0000000100150c7c in php_execute_script (primary_file=0xffffffffffff650) at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/main/main.c:2639#280x0000000100001480 in main (argc=-2144116105, argv=0x180336277) at /HOME/QSECOFR/rpmbuild/BUILD/php-7.3.14/sapi/fpm/fpm/fpm_main.c:1950
I then entered the commands below, but didn’t get and details:
(gdb) print (char *)(executor_globals.function_state_ptr->function)->common.function_name
There is no member named function_state_ptr.
(gdb) print (char *)executor_globals.active_op_array->function_name
There is no member named active_op_array.
(gdb) print (char *)executor_globals.active_op_array->filename
There is no member named active_op_array.
I am exploring around at the frames in the backtrace currently. Let me know if there are any suggestions or indicates of what the problem may be.