1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

buffer overflow detected: pilerimport

Issue #20 resolved
János Csárdi-Braunstein created an issue

piler 0.1.19 Configure command: ./configure --enable-memcached --localstatedir=/var

{{{ buffer overflow detected : pilerimport terminated ======= Backtrace: ========= /lib64/libc.so.6(__fortify_fail+0x37)[0x7feb8eb6af37] /lib64/libc.so.6(+0xf0d50)[0x7feb8eb68d50] /lib64/libc.so.6(+0xefbd7)[0x7feb8eb67bd7] pilerimport(parse_line+0x35c)[0x40a0dc] pilerimport(parse_message+0x174)[0x40b1b4] pilerimport(import_message+0xad)[0x407fed] pilerimport(import_from_maildir+0x195)[0x405245] pilerimport(import_from_maildir+0x1b5)[0x405265] pilerimport(import_from_maildir+0x1b5)[0x405265] pilerimport(main+0x2ac)[0x40569c] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7feb8ea9a2ad] pilerimport[0x404e29] ======= Memory map: ======== 00400000-00416000 r-xp 00000000 ca:02 3033123 /usr/local/bin/pilerimport 00615000-00616000 r--p 00015000 ca:02 3033123 /usr/local/bin/pilerimport 00616000-00618000 rw-p 00016000 ca:02 3033123 /usr/local/bin/pilerimport 024c8000-02521000 rw-p 00000000 00:00 0 [heap] 7feb8d2dc000-7feb8dbfa000 r--p 00000000 ca:02 3147823 /usr/lib64/locale/locale-archive 7feb8dbfa000-7feb8dc06000 r-xp 00000000 ca:02 1118380 /lib64/libnss_files-2.14.1.so 7feb8dc06000-7feb8de05000 ---p 0000c000 ca:02 1118380 /lib64/libnss_files-2.14.1.so 7feb8de05000-7feb8de06000 r--p 0000b000 ca:02 1118380 /lib64/libnss_files-2.14.1.so 7feb8de06000-7feb8de07000 rw-p 0000c000 ca:02 1118380 /lib64/libnss_files-2.14.1.so 7feb8de07000-7feb8de1c000 r-xp 00000000 ca:02 1186918 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/libgcc_s.so.1 7feb8de1c000-7feb8e01b000 ---p 00015000 ca:02 1186918 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/libgcc_s.so.1 7feb8e01b000-7feb8e01c000 r--p 00014000 ca:02 1186918 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/libgcc_s.so.1 7feb8e01c000-7feb8e01d000 rw-p 00015000 ca:02 1186918 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.3/libgcc_s.so.1 7feb8e01d000-7feb8e01e000 ---p 00000000 00:00 0 7feb8e01e000-7feb8e81e000 rw-p 00000000 00:00 0 7feb8e81e000-7feb8e870000 r-xp 00000000 ca:02 1130287 /usr/lib64/libssl.so.1.0.0 7feb8e870000-7feb8ea70000 ---p 00052000 ca:02 1130287 /usr/lib64/libssl.so.1.0.0 7feb8ea70000-7feb8ea73000 r--p 00052000 ca:02 1130287 /usr/lib64/libssl.so.1.0.0 7feb8ea73000-7feb8ea78000 rw-p 00055000 ca:02 1130287 /usr/lib64/libssl.so.1.0.0 7feb8ea78000-7feb8ebfd000 r-xp 00000000 ca:02 1118367 /lib64/libc-2.14.1.so 7feb8ebfd000-7feb8edfd000 ---p 00185000 ca:02 1118367 /lib64/libc-2.14.1.so 7feb8edfd000-7feb8ee01000 r--p 00185000 ca:02 1118367 /lib64/libc-2.14.1.so 7feb8ee01000-7feb8ee02000 rw-p 00189000 ca:02 1118367 /lib64/libc-2.14.1.so 7feb8ee02000-7feb8ee07000 rw-p 00000000 00:00 0 7feb8ee07000-7feb8ee1f000 r-xp 00000000 ca:02 1118324 /lib64/libpthread-2.14.1.so 7feb8ee1f000-7feb8f01e000 ---p 00018000 ca:02 1118324 /lib64/libpthread-2.14.1.so 7feb8f01e000-7feb8f01f000 r--p 00017000 ca:02 1118324 /lib64/libpthread-2.14.1.so 7feb8f01f000-7feb8f020000 rw-p 00018000 ca:02 1118324 /lib64/libpthread-2.14.1.so 7feb8f020000-7feb8f024000 rw-p 00000000 00:00 0 7feb8f024000-7feb8f15e000 r-xp 00000000 ca:02 1196133 /usr/lib64/mysql/libmysqlclient_r.so.16.0.0 7feb8f15e000-7feb8f35d000 ---p 0013a000 ca:02 1196133 /usr/lib64/mysql/libmysqlclient_r.so.16.0.0 7feb8f35d000-7feb8f360000 r--p 00139000 ca:02 1196133 /usr/lib64/mysql/libmysqlclient_r.so.16.0.0 7feb8f360000-7feb8f3ab000 rw-p 0013c000 ca:02 1196133 /usr/lib64/mysql/libmysqlclient_r.so.16.0.0 7feb8f3ab000-7feb8f3ad000 rw-p 00000000 00:00 0 7feb8f3ad000-7feb8f3bc000 r-xp 00000000 ca:02 1191174 /usr/lib64/libtre.so.5.0.0 7feb8f3bc000-7feb8f5bb000 ---p 0000f000 ca:02 1191174 /usr/lib64/libtre.so.5.0.0 7feb8f5bb000-7feb8f5bc000 r--p 0000e000 ca:02 1191174 /usr/lib64/libtre.so.5.0.0 7feb8f5bc000-7feb8f5bd000 rw-p 0000f000 ca:02 1191174 /usr/lib64/libtre.so.5.0.0 7feb8f5bd000-7feb8f724000 r-xp 00000000 ca:02 1130285 /usr/lib64/libcrypto.so.1.0.0 7feb8f724000-7feb8f923000 ---p 00167000 ca:02 1130285 /usr/lib64/libcrypto.so.1.0.0 7feb8f923000-7feb8f93b000 r--p 00166000 ca:02 1130285 /usr/lib64/libcrypto.so.1.0.0 7feb8f93b000-7feb8f944000 rw-p 0017e000 ca:02 1130285 /usr/lib64/libcrypto.so.1.0.0 7feb8f944000-7feb8f948000 rw-p 00000000 00:00 0 7feb8f948000-7feb8f94a000 r-xp 00000000 ca:02 1118400 /lib64/libdl-2.14.1.so 7feb8f94a000-7feb8fb4a000 ---p 00002000 ca:02 1118400 /lib64/libdl-2.14.1.so 7feb8fb4a000-7feb8fb4b000 r--p 00002000 ca:02 1118400 /lib64/libdl-2.14.1.so 7feb8fb4b000-7feb8fb4c000 rw-p 00003000 ca:02 1118400 /lib64/libdl-2.14.1.so 7feb8fb4c000-7feb8fbce000 r-xp 00000000 ca:02 1118356 /lib64/libm-2.14.1.so 7feb8fbce000-7feb8fdcd000 ---p 00082000 ca:02 1118356 /lib64/libm-2.14.1.so 7feb8fdcd000-7feb8fdce000 r--p 00081000 ca:02 1118356 /lib64/libm-2.14.1.so 7feb8fdce000-7feb8fdcf000 rw-p 00082000 ca:02 1118356 /lib64/libm-2.14.1.so 7feb8fdcf000-7feb8fde3000 r-xp 00000000 ca:02 1584256 /lib64/libz.so.1.2.5.1 7feb8fde3000-7feb8ffe3000 ---p 00014000 ca:02 1584256 /lib64/libz.so.1.2.5.1 7feb8ffe3000-7feb8ffe4000 r--p 00014000 ca:02 1584256 /lib64/libz.so.1.2.5.1 7feb8ffe4000-7feb8ffe5000 rw-p 00015000 ca:02 1584256 /lib64/libz.so.1.2.5.1 7feb8ffe5000-7feb8fffe000 r-xp 00000000 ca:02 263122 /lib64/libpiler.so.0.1.1 7feb8fffe000-7feb901fd000 ---p 00019000 ca:02 263122 /lib64/libpiler.so.0.1.1 7feb901fd000-7feb901fe000 r--p 00018000 ca:02 263122 /lib64/libpiler.so.0.1.1 7feb901fe000-7feb90200000 rw-p 00019000 ca:02 263122 /lib64/libpiler.so.0.1.1 7feb90200000-7feb90221000 r-xp 00000000 ca:02 1118312 /lib64/ld-2.14.1.so 7feb9040e000-7feb90414000 rw-p 00000000 00:00 0 7feb9041c000-7feb90420000 rw-p 00000000 00:00 0 7feb90420000-7feb90421000 r--p 00020000 ca:02 1118312 /lib64/ld-2.14.1.so 7feb90421000-7feb90422000 rw-p 00021000 ca:02 1118312 /lib64/ld-2.14.1.so 7feb90422000-7feb90423000 rw-p 00000000 00:00 0 7fffc13e9000-7fffc144c000 rw-p 00000000 00:00 0 [stack] 7fffc1473000-7fffc1474000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Aborted }}}

Comments (30)

  1. Janos SUTO repo owner

    this must be a bug in the parser code. Please download the latest master branch, and compile it, but don't install it yet. It contains a debug tool (test/debug), please run on the problematic email (do the same for issue #19, too):

    ./test/debug /path/to/email.eml

    it will print each line preceeded with the string 'line:', and after parsing the whole message, it will print some statistics about the email, eg. from, to, subject, and the parsed body as well.

    Please show me the last 2 'line:' lines to see where the parser_line function fails.

    Or if they are not confidential messages, please send them to me (in a zip or gz file) for investigation.

  6. János Csárdi-Braunstein reporter
    • changed status to open

    Megint jelentkezett.

    Egy ilyen sornál hal el (ez egy spam ahogy nézem):

    IURPQ1RZUEUgSFRNTCBQVUJMSUMgIi0vL1czQy8vRFREIEhUTUwgNC4wIFRyYW5zaXRpb25hbC8vRU4iPg0KPEhUTUw+PEhFQUQ+DQo8TUVUQSBodHRwLWVxdWl2PUNvbnRlbnQtVHlwZSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9aXNvLTg4NTktMSI+DQo8L0hFQUQ+DQo8Qk9EWT48dHI+PHRkPg0KPGRpdiBhbGlnbj1jZW50ZXI+IDxhIGhyZWY9Imh0dHA6Ly90cmFkZW90aGVyLmNvbS8iIHRhcmdldD0iX2JsYW5rIj4NCjxpbWcgc3JjPSJodHRwOi8vdHJhZGVvdGhlci5jb20vYWR2MS5qcGciIGJvcmRlcj0wIGFsdD0iQ2xpY2sgSGVyZSEiPjwvYT48L2Rpdj4NCjwvdGQ+PC90cj48dHI+PHRkIGNsYXNzPUVDX2xlZ2FsPjxzdHJvbmc+QWJvdXQgdGhpcyBtYWlsaW5nOiA8L3N0cm9uZz48YnI+DQpZb3UgYXJlIHJlY2VpdmluZyB0aGlzIGUtbWFpbCBiZWNhdXNlIHlvdSBzdWJzY3JpYmVkIHRvIE1TTiBGZWF0dXJlZCBPZmZlcnMuIA0KTWljcm9zb2Z0IHJlc3BlY3RzIHlvdXIgcHJpdmFjeS4gSWYgeW91IGRvIG5vdCB3aXNoIHRvIHJlY2VpdmUgdGhpcyBNU04gRmVhdHVyZWQgDQpPZmZlcnMgZS1tYWlsLCBwbGVhc2UgY2xpY2sgdGhlICJVbnN1YnNjcmliZSIgbGluayBiZWxvdy4gVGhpcyB3aWxsIG5vdCB1bnN1YnNjcmliZSANCnlvdSBmcm9tIGUtbWFpbCBjb21tdW5pY2F0aW9ucyBmcm9tIHRoaXJkLXBhcnR5IGFkdmVydGlzZXJzIHRoYXQgbWF5IGFwcGVhciBpbiBNU04gDQpGZWF0dXJlIE9mZmVycy4gVGhpcyBzaGFsbCBub3QgY29uc3RpdHV0ZSBhbiBvZmZlciBieSBNU04uIE1TTiBzaGFsbCBub3QgYmUgDQpyZXNwb25zaWJsZSBvciBsaWFibGUgZm9yIHRoZSBhZHZlcnRpc2VycycgY29udGVudCBub3IgYW55IG9mIHRoZSBnb29kcyBvciBzZXJ2aWNlDQogYWR2ZXJ0aXNlZC4gUHJpY2VzIGFuZCBpdGVtIGF2YWlsYWJpbGl0eSBzdWJqZWN0IHRvIGNoYW5nZSB3aXRob3V0IG5vdGljZS48YnI+PGJyPg0KQzIwMDggTWljcm9zb2Z0IHwgPGEgaHJlZj0iaHR0cDovL3RyYWRlb3RoZXIuY29tLyIgdGFyZ2V0PSJfYmxhbmsiPlVuc3Vic2NyaWJlPC9hPiB8IA0KPGEgaHJlZj0iaHR0cDovL3RyYWRlb3RoZXIuY29tLyIgdGFyZ2V0PSJfYmxhbmsiPk1vcmUgTmV3c2xldHRlcnM8L2E+IHwgDQo8YSBocmVmPSJodHRwOi8vdHJhZGVvdGhlci5jb20vIiB0YXJnZXQ9Il9ibGFuayI+UHJpdmFjeTwvYT48YnI+PGJyPg0KICAgICAgICAgIE1pY3Jvc29mdCBDb3Jwb3JhdGlvbiwgT25lIE1pY3Jvc29mdCBXYXksIFJlZG1vbmQsIFdBIDk4MDUyDQo8L3RkPjwvdHI+PC90YWJsZT48L3RkPjwvdHI+PC90YWJsZT48L2Rpdj48L2Rpdj48L2Rpdj48L0JPRFk+PC9IVE1MPnsvQkFTRTY0X0VOQ09ERUR9DQoNCgAAAAAAAAAAAAAAAA==

    *** buffer overflow detected ***: ./debug terminated                                                                                                                                                                                         
======= Backtrace: =========                                                                                                                                                                                                                 
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f802e1467a7]                                                                                                                                                                                        
/lib64/libc.so.6(+0xff5a0)[0x7f802e1445a0]                                                                                                                                                                                                   
/lib64/libc.so.6(+0xfe427)[0x7f802e143427]                                                                                                                                                                                                   
/lib64/libpiler.so(parse_line+0x38d)[0x7f802f68c07d]                                                                                                                                                                                         
/lib64/libpiler.so(parse_message+0x18c)[0x7f802f68d97c]                                                                                                                                                                                      
./debug(main+0x110)[0x400e00]                                                                                                                                                                                                                
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f802e0674bd]                                                                                                                                                                                     
./debug[0x400c29]                                                                                                                                                                                                                            
======= Memory map: ========                                                                                                                                                                                                                 
00400000-00402000 r-xp 00000000 ca:02 3284788                            /usr/src/piler/test/debug                                                                                                                                           
00601000-00602000 r--p 00001000 ca:02 3284788                            /usr/src/piler/test/debug                                                                                                                                           
00602000-00603000 rw-p 00002000 ca:02 3284788                            /usr/src/piler/test/debug                                                                                                                                           
00dca000-00deb000 rw-p 00000000 00:00 0                                  [heap]                                                                                                                                                              
7f802dc12000-7f802dc27000 r-xp 00000000 ca:02 2572909                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.4/libgcc_s.so.1                                                                                                              
7f802dc27000-7f802de26000 ---p 00015000 ca:02 2572909                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.4/libgcc_s.so.1
7f802de26000-7f802de27000 r--p 00014000 ca:02 2572909                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.4/libgcc_s.so.1
7f802de27000-7f802de28000 rw-p 00015000 ca:02 2572909                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.5.4/libgcc_s.so.1
7f802de28000-7f802de40000 r-xp 00000000 ca:02 3260692                    /lib64/libpthread-2.15.so
7f802de40000-7f802e03f000 ---p 00018000 ca:02 3260692                    /lib64/libpthread-2.15.so
7f802e03f000-7f802e040000 r--p 00017000 ca:02 3260692                    /lib64/libpthread-2.15.so
7f802e040000-7f802e041000 rw-p 00018000 ca:02 3260692                    /lib64/libpthread-2.15.so
7f802e041000-7f802e045000 rw-p 00000000 00:00 0 
7f802e045000-7f802e1e6000 r-xp 00000000 ca:02 3260709                    /lib64/libc-2.15.so
7f802e1e6000-7f802e3e6000 ---p 001a1000 ca:02 3260709                    /lib64/libc-2.15.so
7f802e3e6000-7f802e3ea000 r--p 001a1000 ca:02 3260709                    /lib64/libc-2.15.so
7f802e3ea000-7f802e3ec000 rw-p 001a5000 ca:02 3260709                    /lib64/libc-2.15.so
7f802e3ec000-7f802e3f0000 rw-p 00000000 00:00 0 
7f802e3f0000-7f802e52a000 r-xp 00000000 ca:02 2962128                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7f802e52a000-7f802e729000 ---p 0013a000 ca:02 2962128                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7f802e729000-7f802e72c000 r--p 00139000 ca:02 2962128                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7f802e72c000-7f802e777000 rw-p 0013c000 ca:02 2962128                    /usr/lib64/mysql/libmysqlclient_r.so.16.0.0
7f802e777000-7f802e779000 rw-p 00000000 00:00 0 
7f802e779000-7f802e788000 r-xp 00000000 ca:02 3115641                    /usr/lib64/libtre.so.5.0.0
7f802e788000-7f802e987000 ---p 0000f000 ca:02 3115641                    /usr/lib64/libtre.so.5.0.0
7f802e987000-7f802e988000 r--p 0000e000 ca:02 3115641                    /usr/lib64/libtre.so.5.0.0
7f802e988000-7f802e989000 rw-p 0000f000 ca:02 3115641                    /usr/lib64/libtre.so.5.0.0
7f802e989000-7f802e9db000 r-xp 00000000 ca:02 1911714                    /usr/lib64/libssl.so.1.0.0
7f802e9db000-7f802ebdb000 ---p 00052000 ca:02 1911714                    /usr/lib64/libssl.so.1.0.0
7f802ebdb000-7f802ebde000 r--p 00052000 ca:02 1911714                    /usr/lib64/libssl.so.1.0.0
7f802ebde000-7f802ebe3000 rw-p 00055000 ca:02 1911714                    /usr/lib64/libssl.so.1.0.0
7f802ebe3000-7f802ed4a000 r-xp 00000000 ca:02 1911240                    /usr/lib64/libcrypto.so.1.0.0
7f802ed4a000-7f802ef49000 ---p 00167000 ca:02 1911240                    /usr/lib64/libcrypto.so.1.0.0
7f802ef49000-7f802ef61000 r--p 00166000 ca:02 1911240                    /usr/lib64/libcrypto.so.1.0.0
7f802ef61000-7f802ef6a000 rw-p 0017e000 ca:02 1911240                    /usr/lib64/libcrypto.so.1.0.0
7f802ef6a000-7f802ef6e000 rw-p 00000000 00:00 0 
7f802ef6e000-7f802ef70000 r-xp 00000000 ca:02 3260681                    /lib64/libdl-2.15.so
7f802ef70000-7f802f170000 ---p 00002000 ca:02 3260681                    /lib64/libdl-2.15.so
7f802f170000-7f802f171000 r--p 00002000 ca:02 3260681                    /lib64/libdl-2.15.so
7f802f171000-7f802f172000 rw-p 00003000 ca:02 3260681                    /lib64/libdl-2.15.so
7f802f172000-7f802f268000 r-xp 00000000 ca:02 3260703                    /lib64/libm-2.15.so
7f802f268000-7f802f467000 ---p 000f6000 ca:02 3260703                    /lib64/libm-2.15.so
7f802f467000-7f802f468000 r--p 000f5000 ca:02 3260703                    /lib64/libm-2.15.so
7f802f468000-7f802f469000 rw-p 000f6000 ca:02 3260703                    /lib64/libm-2.15.so
7f802f469000-7f802f47d000 r-xp 00000000 ca:02 1652899                    /lib64/libz.so.1.2.7
7f802f47d000-7f802f67d000 ---p 00014000 ca:02 1652899                    /lib64/libz.so.1.2.7
7f802f67d000-7f802f67e000 r--p 00014000 ca:02 1652899                    /lib64/libz.so.1.2.7
7f802f67e000-7f802f67f000 rw-p 00015000 ca:02 1652899                    /lib64/libz.so.1.2.7
7f802f67f000-7f802f69e000 r-xp 00000000 ca:02 3033116                    /usr/local/lib/libpiler.so.0.1.1
7f802f69e000-7f802f89d000 ---p 0001f000 ca:02 3033116                    /usr/local/lib/libpiler.so.0.1.1
7f802f89d000-7f802f89e000 r--p 0001e000 ca:02 3033116                    /usr/local/lib/libpiler.so.0.1.1
7f802f89e000-7f802f8a1000 rw-p 0001f000 ca:02 3033116                    /usr/local/lib/libpiler.so.0.1.1
7f802f8a1000-7f802f8c3000 r-xp 00000000 ca:02 3260685                    /lib64/ld-2.15.so
7f802fab0000-7f802fab6000 rw-p 00000000 00:00 0 
7f802fabe000-7f802fac2000 rw-p 00000000 00:00 0 
7f802fac2000-7f802fac3000 r--p 00021000 ca:02 3260685                    /lib64/ld-2.15.so
7f802fac3000-7f802fac4000 rw-p 00022000 ca:02 3260685                    /lib64/ld-2.15.so
7f802fac4000-7f802fac5000 rw-p 00000000 00:00 0 
7fffb8680000-7fffb871f000 rw-p 00000000 00:00 0                          [stack]
7fffb87ff000-7fffb8800000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted

    piler -v:

    piler 0.1.22, build 735, Janos SUTO <sj@acts.hu>

Build Date: Mon Dec 3 14:35:37 CET 2012
Configure command: ./configure --enable-memcached --localstatedir=/var
  7. Janos SUTO repo owner

    Ha spam, akkor lehetseges, hogy gzip-pelve mellekletkent elkuldod a cimemre? Jo lenne az egeszet egyben latni. A Received:/To: sorokat nyugodtan ird at, de jo lenne, ha a level minel inkabb hasonlitana az eredetihez, amit te is kaptal.

  10. Janos SUTO repo owner

    Hmm, erdekes level. Eleve az rossz benne, hogy a Content-type: utan nem azt szoktak irni, hogy base64, hanem mondjuk text/plain, stb. A Content-Transfer-Encoding: utan szoktak irni, hogy pl. base64.

    A levelet az eredeti formajaban sem akasztja ki nalam a pilertest programot, bar igy csak szemet jelenik meg a body dekodolasa utan. De ha atirom a header-ben a Content-Type mezot Content-Transfer-Encodingra, akkor rendben megjelenik a dekodolt bse64 tartalom.

    Milyen glibc es gcc verziot hasznalsz?

  11. János Csárdi-Braunstein reporter 
    Portage 2.1.11.31 (default/linux/amd64/10.0, gcc-4.5.4, glibc-2.15-r3, 3.3.8-gentoo-domU-4.0 x86_64)
=================================================================
System uname: Linux-3.3.8-gentoo-domU-4.0-x86_64-Intel-R-_Xeon-R-_CPU_X5470_@_3.33GHz-with-gentoo-2.1
Timestamp of tree: Mon, 03 Dec 2012 13:00:02 +0000
ld GNU ld (GNU Binutils) 2.22
app-shells/bash:          4.2_p37
dev-lang/python:          2.6.6-r2, 2.7.3-r2, 3.2.3
dev-util/pkgconfig:       0.23
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.11.5
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.68
sys-devel/automake:       1.11.1
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.5.4
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.6 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo x-overlay
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="-GK"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/overlay"
SYNC="rsync://rsync/gentoo-portage"
USE="acl amd64 berkdb bindist bzip2 cli cracklib crypt cxx dri fortran gdbm gif gpm iconv idn iproute2 ipv6 jpeg logrotate mmx modules mudflap multilib ncurses nls nptl openmp pam pcre png pppd qt3support readline session sse sse2 ssl svg syslog tcpd unicode zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_default authn_file authz_default authz_groupfile authz_host authz_owner authz_user autoindex cgi cgid dir env expires headers info log_config mime mime_magic negotiation rewrite setenvif status vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGINX_MODULES_HTTP="fastcgi gzip gzip_static limit_req limit_zone map proxy realip rewrite stub_status upstream_ip_hash" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
  12. János Csárdi-Braunstein reporter

    Van egy csomo ilyen levelem amire ezt dobja a pilerimport. Sajna egy 40e levelet tartalmazo Maildirben van.

    Nem tudnál belerakni valamit, hogy írja ki, melyik fájlnál van hiba, vagy ilyesmi?

  13. János Csárdi-Braunstein reporter

    Ahogy nézem, attól halódik el, hogy túl hosszú egy sor. Ez lehet?

    Aminél elhal sor, az 990 karakter hosszú, ha azt kitörlöm, rendben megy.

  14. Janos SUTO repo owner

    ezt kapom az eredeti levelre: ./pilertest aaa1

    locale: hu_HU build: 739 message-id: E1LH75b-0004vZ-00@ms1.domainem.hu from: cimzett@domainem.hu cimzett domainem hu (domainem.hu) to: cimzett@domainem.hu cimzett domainem hu (domainem.hu ) reference: subject: * Please confirm your message body: PEhFQUQ DQo8L0hFQUQ DQo8Qk9EWT48dHI PC90cj48dHI UHJpdmFjeTwvYT48YnI PC90YWJsZT48L3RkPjwvdHI * sent: 1230514720, delivered-date: 0 hdr len: 830 body digest: 460b0bb86bf85d0f5e16da75defcdbfe58d52aeee6b2ae4497c06affb190cee8 rules check: (null) retention period: 1575574752 attachments: direction: 0 spam: 0

    Ezt pedig akkor, ha beirom a kodolast a header-be: ./pilertest aaa2

    locale: hu_HU build: 739 message-id: E1LH75b-0004vZ-00@ms1.domainem.hu from: cimzett@domainem.hu cimzett domainem hu (domainem.hu) to: cimzett@domainem.hu cimzett domainem hu (domainem.hu ) reference: subject: * Please confirm your message body: !DOCTYPE HTML PUBLIC - W3C DTD HTML 4.0 Transitional EN HTML HEAD META http-equiv Content-Type content text html charset iso-8859-1 HEAD BODY tr td div align center a href __URL__tradeotherXcom target blank img src __URL__tradeotherXcom border 0 alt Click Here a div td tr tr td class EC legal strong About this mailing strong br You are receiving this e-mail because you subscribed to MSN Featured Offers Microsoft respects your privacy If you do not wish to receive this MSN Featured Offers e-mail please click the Unsubscribe link below This will not unsubscribe you from e-mail communications from third-party advertisers that may appear in MSN Feature Offers This shall not constitute an offer by MSN MSN shall not be responsible or liable for the advertisers content nor any of the goods or service advertised Prices and item availability subject to change without notice br br C2008 Microsoft a href __URL__tradeotherXcom target blank Unsubscribe a a href __URL__tradeotherXcom target blank More Newsletters a a href __URL__tradeotherXcom target blank Privacy a br br Microsoft Corporation One Microsoft Way Redmond WA 98052 td tr table td tr table div div div BODY HTML BASE64 ENCODED * sent: 1230514720, delivered-date: 0 hdr len: 843 body digest: 460b0bb86bf85d0f5e16da75defcdbfe58d52aeee6b2ae4497c06affb190cee8 rules check: (null) retention period: 1575574989 attachments: direction: 0 spam: 0

    legfrissebb piler verzio, 2.15-os glibc es 4.7.1-es gcc 32-bites x86-on.

  15. Janos SUTO repo owner

    ja, es az nem gond neki, ha egy levelben hosszu sorok vannak. Egy orosz felhasznalo mutatott korabban 4-5kB (!) hosszu sorokat tartalmazo levelet, amivel valoban elhasalt egy korabbi verzio, de ez mar (nyar korul?) javitva van.

    Tudsz a problemas levelre egy strace-t futtatni?

  16. János Csárdi-Braunstein reporter

    Az itt lévő rossz.eml fájlt letöltöttem a /tmp/mail mappába, majd ráfuttattam a következőt:

    strace -s 999999 pilerimport -d /tmp/mail/ 2>strace.log
  17. Janos SUTO repo owner

    Ok, kossz. Erdekes, hogy 2 db libpiler.so-t is latok benne:

    /lib64/libpiler.so es /usr/local/lib/libpiler.so.0.1.1

    Ha a prefix a default /usr/local, akkor hogy kerul a kepbe a /lib64 alatti libpiler.so? Melyik a frissebb?

  18. János Csárdi-Braunstein reporter

    /lib64 alatti csak symlink

    lrwxrwxrwx 1 root root 17 Apr 26  2012 /lib64/libpiler.so -> libpiler.so.0.1.1
lrwxrwxrwx 1 root root 17 Apr 26  2012 /lib64/libpiler.so.0 -> libpiler.so.0.1.1
lrwxrwxrwx 1 root root 32 Nov  7 13:47 /lib64/libpiler.so.0.1.1 -> /usr/local/lib/libpiler.so.0.1.1
  19. Janos SUTO repo owner

    Ok, ertem. Es a /usr/local/lib/libpiler.so.0.1.1 meg tuti friss. Hmm... megprobalok keriteni egy x64-es gepet, aztan kiprobalom x64-en is. x86-on nem tudom reprodukalni a hibat.

    Btw. ha van meg x86-os geped valahol, esetleg kiprobalhatnad, hogy azon is elhasal-e a pilertest ezzel a levellel.

  20. János Csárdi-Braunstein reporter

    Nincs x86-om sehol.

    Megint lett egy ilyen levél. Itt is a 9413 karakter hosszú sornál hal el, a sornak a 2594. karaktere után.

  21. Janos SUTO repo owner

    Nem gond. Este osszekalapalok egy x64-es virtualis gepet, es megnezem abban. Lehet, hogy ez lesz eletem 1. gentoo installja :-)

  22. Janos SUTO repo owner

    sikerult ubuntu-n beleszaladni ebbe a problemaba 2.15-os glibc es 4.6.1-es gcc mellett.

    A master branch-bol a src/misc.c file-t kell felulirni, majd ujraforditani a cuccot.

  23. János Csárdi-Braunstein reporter

    Bemásoltam a 0.1.22 forrásába ezt a fájlt, leforgattam, telepítettem, az import faszán megy. viszont most, hogy beimportáltam ezeket a leveleket, ezt kapom:

    indexer --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate

    Sphinx 2.0.6-release (r3473)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinx/sphinx.conf'...
merging index 'delta1' into index 'main1'...
read 79.7 of 79.7 MB, 100.0% done
*** Oops, indexer crashed! Please send the following report to developers.
Sphinx 2.0.6-release (r3473)
-------------- report begins here ---------------
Current document: docid=0, hits=0
Current batch: minid=0, maxid=0
Hit pool start: docid=0, hit=0
-------------- backtrace begins here ---------------
Program compiled with x86_64-pc-linux-gnu-gcc 4.5.4
Host OS is Linux stagingbinhost 3.3.8-gentoo-domU-4.0 #10 SMP Tue Sep 4 21:50:06 CEST 2012 x86_64 Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz GenuineIntel GNU/Linux
Stack bottom = 0x0, thread stack size = 0x10000
FATAL: Cannot get stack frame pointer on this architecture
begin of system backtrace:
begin of system symbols:
indexer(_Z12sphBacktraceib+0x7e)[0x5419ee]
indexer(_Z7sigsegvi+0xa0)[0x45ef20]
/lib64/libpthread.so.0(+0x10460)[0x7f6694445460]
indexer(_ZN13CSphIndex_VLN7PrereadEv+0x387)[0x49e257]
indexer(_ZN13CSphIndex_VLN5MergeEP9CSphIndexR10CSphVectorI18CSphFilterSettings16CSphVectorPolicyIS3_EEb+0xdc)[0x4abaac]
indexer(_Z7DoMergeRK17CSphConfigSectionPKcS1_S3_R10CSphVectorI18CSphFilterSettings16CSphVectorPolicyIS5_EEbb+0x55a)[0x46190a]
indexer(main+0x995)[0x469035]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f6693b944bd]
indexer[0x45ecf9]
Backtrace looks OK. Now you have to do following steps:
  1. Run the command over the crashed binary (for example, 'indexer'):
     nm -n indexer > indexer.sym
  2. Attach the binary, generated .sym and the text of backtrace (see above) to the bug report.
Also you can read the section about resolving backtraces in the documentation.
-------------- backtrace ends here ---------------

    Nem tudom, hogy van e összefüggés a kettő között.

    z az üzenet egy gentoo-s (?) patchbe van benne amúgy:

    http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/app-misc/sphinx/files/sphinx-2.0.1_beta-darwin8.patch?view=markup

  24. Janos SUTO repo owner

    Ez egy sphinx problema lesz. Sajnos nem tudok annal en sem jobbat mondani, minthogy kuldd el a fejlesztoknek, amit fentebb kernek, es remelhetoleg kijavitjak, amit kell.

  25. János Csárdi-Braunstein reporter

    Ahogy nézem már a searchd is segfaulttol.

    using config file '/etc/sphinx/sphinx.conf'...
listening on 127.0.0.1:9312
listening on 127.0.0.1:9306
precaching index 'main1'
DEBUG: CSphIndex_VLN::Preread invoked
DEBUG: Prereading .spa
DEBUG: Prereading .spm51.4% done
DEBUG: Prereading .sps
DEBUG: Prereading .spk51.4% done
DEBUG: Prereading .spi
Segmentation fault

    csinalok nekik bugreportot.

  27. Janos SUTO repo owner

    hmm. Kiprobaltam a demo gepen is, es beimportaltam a rossz.eml levelet, es siman megette. Ra is kereshetsz pl. az alabbit beirva:

    DQo8Qk9EWT48dHI PC90cj48dHI UHJpdmFjeTwvYT48YnI

    Szoval nekem az az osszeeskuves elmeletem, hogy a glibc 2.15 es a 4.5.x (vagy 4.6.x) gcc valahogy nem nyero parositas. Ha mondanak a sphinx-esek valami okosat, akkor erdekel a fejlemeny.

  28. János Csárdi-Braunstein reporter

    hmm,

    kipróbáltam egy újraindexelést, és ez jött szembe:

    WARNING: sql_query_post_index: The total number of locks exceeds the lock table size (DSN=mysql://piler:***@localhost:3306/piler)

    Bár, kicsi az esélye, hogy ez okoz valamit, de azért megemelem ennek méretét a mysql-be

  29. János Csárdi-Braunstein reporter

    Hmm, egy hete átírtam ilyenre a cront:

    indexer --quiet delta1 --rotate && sleep 2 && indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate && sleep 2 && indexer --quiet tag1 --rotate && sleep 2 && indexer --quiet note1 --rotate

    És azóta nem jelentkezett ez a hiba. Lehet a párhuzamos futtatást nem birja?

  30. Janos SUTO repo owner

    Orulok, hogy ujra mukodik. Mondjuk rejtely, hogy miert jott elo, amikor egyszerre kezd dolgozni a delta+main ill. a tag es note source-okkal, mert azok kulon file-ok a sphinx konyvtarban. Mindegy, ha igy atirva a cron job-ot jo lett nalad, az nekem eleg jo.

  31. Log in to comment
Assignee
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
2
Jira: the preferred issue tracker for Bitbucket. Join the team!