No GUI after upgrading to 1.4.4

[Ricardo Bessa]

Hi!

‌

I’ve upgraded my docker installation from 1.3.12 to 1.4.4, manually creating manticore.conf (follow directions from here as well https://www.mailpiler.org/wiki/current:manticore) and piler seems to be starting correctly, but i can’t access the GUI anymore (error 404).

I was hoping you would have some insights that could help, if you have some time.

‌

The container logs don’t show anything unusual I think:

DEBUG: mysql is ready
DEBUG: metadata table exists
 * Starting periodic command scheduler cron
   ...done.
 * Starting nginx nginx
   ...done.
starting searchd . . .
[Sun Jul  2 19:02:13.307 2023] [62] using config file '/etc/piler/manticore.conf' (3574 chars)...
Manticore 6.0.4 1a3a4ea82@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)
starting daemon version '6.0.4 1a3a4ea82@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)' ...
listening on 127.0.0.1:9312 for sphinx and http(s)
listening on 127.0.0.1:9306 for mysql
Manticore 6.0.4 1a3a4ea82@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)
precaching table 'piler1'
precaching table 'tag1'
precaching table 'note1'
Manticore 6.0.4 1a3a4ea82@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)
starting piler-smtp . . . 
starting piler . . .

‌

This is my docker-compose file (with ammended passwords and domain):

version: "3"
services:
  mysql:
    image: mariadb:10.5
    restart: always
    container_name: mailpiler_db
    environment:
      - MYSQL_DATABASE=piler
      - MYSQL_USER=piler
      - MYSQL_PASSWORD=password
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    volumes:
      - /docker_volumes/mailpiler/db_data:/var/lib/mysql

  memcached:
    image: memcached:latest
    restart: always
    container_name: mailpiler_memcached
    command: -m 64

  piler:
    image: sutoj/piler:1.4.4
    container_name: mailpiler_app
    restart: always
    init: true
    environment:
      - MYSQL_HOSTNAME=mysql
      - MYSQL_DATABASE=piler
      - MYSQL_USER=piler
      - MYSQL_PASSWORD=password
      - PILER_HOSTNAME=mailarchive.domain.net
      - MULTITENANCY=0
      - MEMCACHED_HOSTNAME=memcached
      #- NODE_TYPE=WORKER
    ports:
      - "16741:80"
    volumes:
      - mailpiler_etc:/etc/piler
      - mailpiler_var_store:/var/piler/store
      - mailpiler_var_manticore:/var/piler/manticore
    depends_on:
      - "memcached"
      - "mysql"

volumes:
  mailpiler_etc:
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/docker_volumes/mailpiler/etc'
  mailpiler_var_store:
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/tank/backup/email_archive/store'
  mailpiler_var_manticore:
    driver: local
    driver_opts:
      type: 'none'
      o: 'bind'
      device: '/tank/backup/email_archive/manticore'

‌

[Janos SUTO]

Check out the nginx error log as well as the mail.log for clues what might have gone wrong. Also Try the stock docker compose file with a fresh environment, and see if all services come up fine.

[Ricardo Bessa]

The nginx error log seems to contain fastcgi errors:

2023/07/02 19:02:22 [crit] 45#45: *1 connect() to unix:/var/run/php/php7.4-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.1.85, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "192.168.1.25:16741"
2023/07/02 19:02:22 [error] 45#45: *1 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.1.85, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock", host: "192.168.1.25:16741"
2023/07/02 19:02:23 [error] 45#45: *1 open() "/var/piler/www/favicon.ico" failed (2: No such file or directory), client: 192.168.1.85, server: mailarchive.domain.net, request: "GET /favicon.ico HTTP/1.1", host: "192.168.1.25:16741", referrer: "http://192.168.1.25:16741/"
2023/07/02 19:02:25 [crit] 45#45: *3 connect() to unix:/var/run/php/php7.4-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "mailarchive.domain.net"
2023/07/02 19:02:25 [error] 45#45: *3 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock", host: "mailarchive.domain.net"
2023/07/02 19:13:47 [crit] 45#45: *5 connect() to unix:/var/run/php/php7.4-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.1.85, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "192.168.1.25:16741"
2023/07/02 19:13:47 [error] 45#45: *5 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 192.168.1.85, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock", host: "192.168.1.25:16741"
2023/07/02 19:13:50 [crit] 45#45: *7 connect() to unix:/var/run/php/php7.4-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "mailarchive.domain.net"
2023/07/02 19:13:50 [error] 45#45: *7 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock", host: "mailarchive.domain.net"
2023/07/02 19:24:52 [crit] 45#45: *9 connect() to unix:/var/run/php/php7.4-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "mailarchive.domain.net"
2023/07/02 19:24:52 [error] 45#45: *9 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 172.80.81.1, server: mailarchive.domain.net, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock", host: "mailarchive.domain.net"

I wasn’t able to find the mail.log though.

‌

Recreating it on a fresh environment does fix the issue, as mailpiler starts normally, so it seems it is an issue with my current config/files

[Ricardo Bessa]

Alright, I don’t know what the problem was exactly, but I seem to have fixed it by replacing the old files in etc with the newly generated ones from the fresh environment. Thanks for the diagnostic help!
(I still have the old etc files, if you wanna look at them for your own debugging)

[Janos SUTO]

I think the php version has changed, so it must be changed in /etc/piler/piler-nginx.conf as well.

[Ricardo Bessa]

Now manticore doesn’t seem to want to work properly, as it only shows 2 of the nearly 30k emails

I’ve tried both the reindex -a command and the indexer.*.sh commands and no dice. Manticore.log does not show any errors. reindex utility can see the correct number of files (even the stats show it correctly).

‌

Edit: it seems that only newly imported emails are showing in the search. Additionally, i activated RT indexing and executing indexer.main.sh shows the fatal error “FATAL: no tables found in config file '/etc/piler/manticore.conf'“

[Janos SUTO]

Check if manticore is configured to use RT index, and if piler.conf rtindex value matches it.

[Ricardo Bessa]

Yeah, it seems so, define('RT', 1); in manticore.conf and rtindex=1 in piler.conf (along with sphxdb=piler1)