check-tag test fails with modules

Issue #10 resolved
Richard PALO created an issue

I'm encountering a difficult issue seemingly involving "modules". The test suite fails as follows:

/opt/local/bin/bmake  check-TESTS
PASS: check-extlang
PASS: check-grandfathered
PASS: check-lang
PASS: check-list
PASS: check-region
PASS: check-script
FAIL: check-tag
PASS: check-trie
PASS: check-variant
============================================================================
Testsuite summary for liblangtag 0.5.4
============================================================================
# TOTAL: 9
# PASS:  8
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to http://github.com/tagoh/liblangtag/issues

and

============================================
   liblangtag 0.5.4: tests/test-suite.log
============================================

# TOTAL: 9
# PASS:  8
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: check-tag
===============

Running suite(s): lt_tag_t
0%: Checks: 7, Failures: 0, Errors: 7
(null):-1:S:Basic functionality:test_lt_tag_parse:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_parse_with_extra_token:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_canonicalize:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_canonicalize_in_extlang_form:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_match:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_transform:0: (after this point) Received signal 11 (Segmentation Fault)
(null):-1:S:Basic functionality:test_lt_tag_convert_from_locale_string:0: (after this point) Received signal 11 (Segmentation Fault)

adding CFLAGS+= -g and configuring with '--enable-debug' I'm able to reproduce the issue with a trivial test program

#include <stdlib.h>
#include <unistd.h>
#include <liblangtag/langtag.h>
int
main(int argc, char **argv)
{
    lt_db_initialize();
    lt_db_finalize();
}

which, built with debug gives interestingly:

#!
richard@omnis:/home/richard/src/tlt$ ./check-tag
I: /tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/data/common/supplemental/likelySubtags.xml
W: Duplicate extension module: ldml-u
W: Duplicate extension module: ldml-t
Segmentation Fault (core dumped)

The segfault is more or less always involving:

open("/opt/local/lib/liblangtag", O_RDONLY|O_NDELAY|O_LARGEFILE) = 3
fcntl(3, F_SETFD, 0x00000001)           = 0
fstat64(3, 0x080479C0)              = 0
getdents64(3, 0xFE954000, 8192)         = 336
getuid()                    = 101 [101]
getuid()                    = 101 [101]
getgid()                    = 10 [10]
getgid()                    = 10 [10]
stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/liblangtag-ext-ldml-u.so", 0x08047190) Err#2 ENOENT
write(2, " W :  ", 3)               = 3
write(2, " D u p l i c a t e   e x".., 34)  = 34
write(2, "\n", 1)               = 1
getuid()                    = 101 [101]
getuid()                    = 101 [101]
getgid()                    = 10 [10]
getgid()                    = 10 [10]
stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/liblangtag-ext-ldml-t.so", 0x08047190) Err#2 ENOENT
write(2, " W :  ", 3)               = 3
write(2, " D u p l i c a t e   e x".., 34)  = 34
write(2, "\n", 1)               = 1
getdents64(3, 0xFE954000, 8192)         = 0
close(3)                    = 0
    Incurred fault #6, FLTBOUNDS  %pc = 0x646C0076
      siginfo: SIGSEGV SEGV_MAPERR addr=0x646C0076
    Received signal #11, SIGSEGV [default]
      siginfo: SIGSEGV SEGV_MAPERR addr=0x646C0076

the address strangely resembles the ASCII string "dl\0v"

(BTW, it's probably a bug that the build directory is maintained in liblangtag.so, as verified with strings)

If I configure with '--disable-modules' the testsuite passes and my program (and libreoffice) seem to work okay.

This is i386 pkgsrc running on SunOS 5.11 (illumos).

Is there anything particularly special about these "modules", if so, any hints on how to get over this?

Comments (9)

  1. Richard PALO reporter

    Just to be clear, the two 'Duplicate extension' warnings seem related to the sticky build directory path if it is still around, because if I clobber it only the

    I: /tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/data/common/supplemental/likelySubtags.xml
    

    and the truss snippet:

    open("/opt/local/lib/liblangtag", O_RDONLY|O_NDELAY|O_LARGEFILE) = 3
    fcntl(3, F_SETFD, 0x00000001)           = 0
    fstat64(3, 0x080479C0)              = 0
    getdents64(3, 0xFE954000, 8192)         = 336
    sysconfig(_CONFIG_PAGESIZE)         = 4096
    getuid()                    = 101 [101]
    getuid()                    = 101 [101]
    getgid()                    = 10 [10]
    getgid()                    = 10 [10]
    stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/liblangtag-ext-ldml-u.so", 0x08047190) Err#2 ENOENT
    open("/usr/lib/locale/fr_FR.UTF-8/LC_MESSAGES/SUNW_OST_OSLIB.mo", O_RDONLY) Err#2 ENOENT
    open("/usr/lib/locale/fr_FR.UTF-8/LC_MESSAGES/SUNW_OST_SGS.mo", O_RDONLY) Err#2 ENOENT
    stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/.libs/liblangtag-ext-ldml-u.so", 0x08047190) Err#2 ENOENT
    stat64("/opt/local/lib/liblangtag/liblangtag-ext-ldml-u.so", 0x08047190) = 0
    resolvepath("/opt/local/lib/liblangtag/liblangtag-ext-ldml-u.so", "/opt/local/lib/liblangtag/liblangtag-ext-ldml-u.so", 1023) = 50
    open("/opt/local/lib/liblangtag/liblangtag-ext-ldml-u.so", O_RDONLY) = 4
    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFE940000
    mmapobj(4, MMOBJ_INTERPRET, 0xFE940018, 0x080471FC, 0x00000000) = 0
    close(4)                    = 0
    memcntl(0xFE920000, 5352, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
    stat64("/opt/local/gcc49/i486-sun-solaris2.11/lib/./liblangtag.so.1", 0x080470A0) Err#2 ENOENT
    getuid()                    = 101 [101]
    getuid()                    = 101 [101]
    getgid()                    = 10 [10]
    getgid()                    = 10 [10]
    stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/liblangtag-ext-ldml-t.so", 0x08047190) Err#2 ENOENT
    stat64("/tmp/pkgsrc/devel/liblangtag/work/liblangtag-0.5.4/extensions/.libs/liblangtag-ext-ldml-t.so", 0x08047190) Err#2 ENOENT
    stat64("/opt/local/lib/liblangtag/liblangtag-ext-ldml-t.so", 0x08047190) = 0
    resolvepath("/opt/local/lib/liblangtag/liblangtag-ext-ldml-t.so", "/opt/local/lib/liblangtag/liblangtag-ext-ldml-t.so", 1023) = 50
    open("/opt/local/lib/liblangtag/liblangtag-ext-ldml-t.so", O_RDONLY) = 4
    mmapobj(4, MMOBJ_INTERPRET, 0xFE940F10, 0x080471FC, 0x00000000) = 0
    close(4)                    = 0
    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFE8F0000
    memcntl(0xFE900000, 5692, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
    getdents64(3, 0xFE954000, 8192)         = 0
    close(3)                    = 0
    munmap(0xFE900000, 10618)           = 0
    munmap(0xFE912000, 2720)            = 0
    munmap(0xFE920000, 10036)           = 0
    munmap(0xFE932000, 2112)            = 0
    _exit(0)
    
  2. Akira TAGOH repo owner

    Duplicate loading won't makes any crashes. if it's reproducible with the trivial code, attaching the backtrace would helps me for debugging.

  3. Richard PALO reporter

    Well, not that I had the time, but I finally got fed up and debugged this out (on 0.5.7).

    pstack shows on my test programs core:

    feea2b8c lmutex_lock (67617477, fe8f4030, 30, fef6ee84, 98728a0, feffb0a8) + 2a
     fee44ba8 readdir64_r (67617467, 8047878, 8047a8c, fef41f24, 80478d0, feffb0a8) + 28
     fee44d00 __posix_readdir_r (67617467, 8047ac4, 8047ac0, fef4a285) + 3c
     fef4a42e lt_ext_modules_load (8047b18) + 1b5
     fef48a13 lt_db_initialize (feffb0a8, 8047b30, 8047b4c, 8050bc3, feeb0a37, fef296e8) + 3a
     08050dbf main     (feeb0a37, fef296e8, 8047b4c, 8050bc3, 1, 8047b58) + 16
     08050bc3 _start   (1, 8047c50, 0, 8047c58, 8047c7f, 8047c86) + 83
    

    ahh, noticed in the manpage for readdir_r:

    The caller must allocate storage pointed to by entry to be large enough
    for a dirent structure with an array of char d_name member containing
    at least NAME_MAX (that is, pathconf(directory, _PC_NAME_MAX)) plus one
    elements. (_PC_NAME_MAX is defined in <unistd.h>.)
    

    even found this tidbit : http://womble.decadent.org.uk/readdir_r-advisory.html

    so I used that for a model to come up with the patches that I attach that now let me run the liblangtag testsuite as well as my test program.

    I'll let the maintainers decide how to handle any errors (note the case returning -1 where neither _PC_NAME_MAX nor NAME_MAX are useful...

    although I do still get the following at runtime?:

    W: Unknown node under /registry/extlang: deprecated
    W: Unknown node under /registry/grandfathered: comments
    W: Unknown node under /registry/grandfathered: comments
    
  4. Richard PALO reporter

    seems okay.

    ===> Testing for liblangtag-0.5.7
    Making check in liblangtag
      GEN      stamp-lt-config
    config.status: executing liblangtag/lt-config.h commands
    config.status: liblangtag/lt-config.h is unchanged
      GEN      stamp-lt-stdint
    config.status: executing liblangtag/lt-stdint.h commands
    config.status: creating liblangtag/lt-stdint.h : _LIBLANGTAG_LIBLANGTAG_LT_STDINT_H
    config.status: liblangtag/lt-stdint.h is unchanged
    /opt/local/bin/bmake  check-recursive
    Making check in extensions
    Making check in data
    Making check in docs
    Making check in tests
    /opt/local/bin/bmake  check-TESTS
    PASS: check-extlang
    PASS: check-grandfathered
    PASS: check-lang
    PASS: check-list
    PASS: check-region
    PASS: check-script
    PASS: check-tag
    PASS: check-trie
    PASS: check-variant
    ============================================================================
    Testsuite summary for liblangtag 0.5.7
    ============================================================================
    # TOTAL: 9
    # PASS:  9
    # SKIP:  0
    # XFAIL: 0
    # FAIL:  0
    # XPASS: 0
    # ERROR: 0
    ============================================================================
    
  5. Log in to comment