Upgrade SO Ubuntu Server from 18.04 LTS to 22.04 LTS and upgrade Piler from 1.3.5 to 1.4.2 (.deb) - libcrypto.so.3

[zappulec]

Hi,
I have performed an upgrade of my Piler Mail server.
First of all, I did an operating system upgrade from Ubuntu Server 18.04 LTS to 20.04 LTS. Piler is still working fine.
Then I upgraded from 20.04 LTS to 22.04 LTS and upgraded the Piler from 1.3.5 to 1.4.2 (.deb package available on the project website).
While the web panel shows messages, I can't see the contents.
I checked manually:
piler@piler:/usr/lib/x86_64-linux-gnu$ pilerget 400000005d64f0b729fb42d400da4244010f
Segmentation fault (core dumped)

dmesg displays the information:
[ 5828.876971] pilerget[5222]: segfault at 4 ip 00007f9dac143044 sp 00007ffdca3c86d8 error 4 in libcrypto.so.3[7f9dac068000+25d000].
[5828.876986] Code: 84 00 00 00 00 00 00 f3 0f 1e fa 80 3d 35 19 2b 00 00 00 ba 00 00 00 00 00 48 8d 05 29 19 2b 00 48 0f 44 c2 c3 0f 1f 40 00 f3 0f 1e fa <8b> 47 04 c3 0f 1f 84 00 00 00 00 00 00 00 f3 0f 1e fa 48 8b 3f e9 e4 ff

I have been sitting on solving this problem since yesterday. I can't seem to figure it out.
Is it possible to ask for help on this issue ?

Thank you

‌

[Janos SUTO]

Does it happen consistently for all emails both archived before and after upgrading to ubuntu 22.04?

[zappulec]

In my environment, messages are imported from the hosting server (imap) once a week on the weekend. So in the database I currently have only old messages. I tried now to download the messages of one of the users but I get an authentication error. Other accounts are also not authenticated.
So I am not sure how pilerget behaves when it handles new messages.

root@piler:/var/piler/tmp# pilerimport -i imap.zenbox.pl -P 993 -t 10 -u "[e-mail]" - p "XXXXXX"
login failed, server reponse: A1 NO [AUTHENTICATIONFAILED] Authentication failed.

‌

‌

[zappulec]

I noticed that in the current version there is an option to import from the gui piler. I created two test instances. Unfortunately, I have a status of "PENDING."

‌

[Janos SUTO]

Please keep one issue per ticket. I'll create an ubuntu 20.04 vm, import some emails, verify that pilerget works properly, then do a dist-upgrade, import some more emails, and see what happens with the archived emails readability before and after upgrade.

[Janos SUTO]

I had the “other” edition at hand, did an upgrade from 20.04 to 22.04, and the previously archived messages (I had only 10 of them) have been retrieved and displayed in the gui just fine.

[Janos SUTO]

Can you try compiling piler from source? Then run make install in the “src” dir only.

[zappulec]

Thank you for your guidance. I'll try it with compilation in a while and let you know what the result is.

I found still in the log /var/log/syslog such information , I do not know what cause the problem with mariadb.

Jan 18 19:11:59 piler kernel: [32562.298818] pilerget[10199]: segfault at 4 ip 00007f9bba181044 sp 00007ffd2c87f988 error 4 in libcrypto.so.3[7f9bba0a6000+25d000]
Jan 18 19:11:59 piler kernel: [32562.298830] Code: 84 00 00 00 00 00 f3 0f 1e fa 80 3d 35 19 2b 00 00 ba 00 00 00 00 48 8d 05 29 19 2b 00 48 0f 44 c2 c3 0f 1f 40 00 f3 0f 1e fa <8b> 47 04 c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 8b 3f e9 e4 ff
Jan 18 19:11:59 piler mariadbd[9551]: 2023-01-18 19:11:59 3984 [Warning] Aborted connection 3984 to db: 'piler' user: 'piler' host: 'localhost' (Got an error reading communication packets)

‌

[zappulec]

root@piler:~/YYY/piler-1.4.1# cd src/
root@piler:~/YYY/piler-1.4.1/src# make install
/usr/bin/install -c -m 0644 libpiler.a /usr/local/lib
/usr/bin/install -c -m 0755 libpiler.so.0.1.1 /usr/local/lib
(cd /usr/local/lib && ln -sf libpiler.so.0.1.1 libpiler.so)
(cd /usr/local/lib && ln -sf libpiler.so.0.1.1 libpiler.so.0)
/usr/bin/install -c -m 0755 piler /usr/local/sbin
/usr/bin/install -c -m 0755 piler-smtp /usr/local/sbin
/usr/bin/install -c -m 0755 pilerconf /usr/local/sbin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pilerget /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pileraget /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pilerimport /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pilerexport /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler reindex /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pilertest /usr/local/bin
/usr/bin/install -c -m 6755 -o piler -g id -gn piler pilerstats /usr/local/bin
root@piler:~/YYY/piler-1.4.1/src# su piler
piler@piler:/root/YYY/piler-1.4.1/src$ /usr/local/bin/pilerget 400000005d64f0b729fb42d400da4244010f
Segmentation fault (core dumped)

I have no idea :-/

[Janos SUTO]

I can see you have trouble with an older format encrypted file. I’ll take a deeper look tomorrow, thank you for your input.

[zappulec]

Thank you very much.

[zappulec]

Hi,
Today I downloaded new messages to my piler server from the mail server using pilerimport (IMAP). And here everything is fine, in webgui the message content opens correctly.
It seems that after the upgrade actually the problem is only with in messages that were in piler server before the upgrade to version 1.4.X

Best regards,
Pawel

[Janos SUTO]

I did a little digging, and the culprit is in src/archive.c in the retrieve_file_from_archive() function. The problem is that EVP_DecryptInit_ex() from openssl 3.0.x provides an error (while openssl 1.x is just fine) when specifying the older cipher (Blowfish) algorithm for the legacy 400000…. encrypted files:

EVP_DecryptInit_ex(ctx, EVP_bf_cbc(), NULL, cfg->key, cfg->iv);

I was lazy to check it’s return value, so when it proceeds and tries to determine the cipher’s block size, it segfaults. I’ll ask on the openssl mailing list, wtf? But for now either you should build piler with openssl 1.x or export all your emails on ubuntu 20.04 and import them to ubuntu 22.04.

‌

[Janos SUTO]

Good news! I’ve managed to find and fix the issue. Long story short: openssl 3 moved Blowfish to the legacy stuff that needs to be loaded explicitely. Get the src/archive.c from the master branch, recompile, and you should be fine.

[zappulec]

Thank you for your help. Actually the recompilation solved the problem with pilerget and allowed to open older messages at which pilerget previously caused a core dump.

[Janos SUTO]

Great! One issue is out of the way.