1. Janos SUTO Avatar Janos SUTO
  2. Piler
  3. piler
  4. Issues

Several Problems

Issue #506 resolved
Former user created an issue

Dear Jsuto,

The archive-data should be on a second disk, mount point is /piler-data. Web-Interface is running with some problems (found similar problems, but solution didnt fit for me):

SMTP-Status : ERROR $config['SITE_NAME'] = 'Piler Mailarchiv'; $config['SITE_URL'] = 'http://172.16.173.204/'; $config['DIR_BASE'] = '/var/www/piler/';

$config['ENABLE_AUDIT'] = 1; $config['MEMCACHED_ENABLED'] = 1; $config['ENABLE_ON_THE_FLY_VERIFICATION'] = 1; $config['ENABLE_SYSLOG'] = 1; $config['PILER_HOST'] = '172.16.173.204'; $config['PILER_PORT'] = 10026; $config['SMARTHOST'] = '172.16.173.254'; $config['SMARTHOST_PORT'] = 25; $config['SMTP_DOMAIN'] = 'xxxxxx.com'; $config['SMTP_FROMADDR'] = 'no-reply@xxxxxx.com'; $config['ADMIN_EMAIL'] = 'admin@xxxxxx.com';

$config['DEFAULT_LANG'] = 'de'; $config['TIMEZONE'] = 'Europe/Berlin';

$config['INDEXER_BEACON'] = '/piler-data/piler/stat/indexer'; $config['PURGE_BEACON'] = '/piler-data/piler/stat/purge'; $config['PILER_HEADER_FIELD'] = 'X-piler-id: ';

$config['DIR_SPHINX'] = '/piler-data/piler/sphinx/'; $config['DIR_STAT'] = '/piler-data/piler/stat'; $config['DIR_IMAP'] = '/piler-data/piler/imap'; $config['DIR_TMP'] = '/piler-data/piler/tmp';

$config['DB_DRIVER'] = 'mysql'; $config['DB_PREFIX'] = ''; $config['DB_HOSTNAME'] = 'localhost'; $config['DB_USERNAME'] = 'piler';

After installation of Piler it shows in Health monitor- CPU usage 100%. Shows the SMTP status: piler: ERROR

piler -V piler 1.1.0, build 884, Janos SUTO sj@acts.hu

Build Date: Sat Feb 14 16:31:18 CET 2015 ldd version: ldd (Ubuntu EGLIBC 2.19-0ubuntu6.5) 2.19 gcc version: gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) Configure command: ./configure --localstatedir=/piler-data --sysconfdir=/etc --enable-clamd --enable-memcached --with-database=mysql --enable-starttls

cat /piler-data/piler/stat/cpu.stat 0.00

netstat -antl tcp 0 0 127.0.0.1:3306 0.0.0.0: LISTEN tcp 0 0 127.0.0.1:11211 0.0.0.0: LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN tcp 0 0 0.0.0.0:25 0.0.0.0: LISTEN tcp 0 0 127.0.0.1:9306 0.0.0.0: LISTEN tcp 0 0 127.0.0.1:9312 0.0.0.0: LISTEN

There is no indexer in /usr/local/bin. I found indexer in /usr/bin crontab -l -u piler | grep mpstat #/5 * * LC_ALL=C mpstat | tail -1 | awk '{print $11}' > /piler-data/piler/stat/cpu.stat #/15 * * /usr/local/bin/indexer --quiet delta1 --rotate && sleep 2 && /usr/local/bin/indexer --quiet --mer$ #/15 * * /usr/local/bin/indexer --quiet tag1 --rotate #/15 * * /usr/local/bin/indexer --quiet note1 --rotate #30 7 * * /usr/bin/php /usr/local/libexec/piler/daily-report.php /var/www/piler # /5 * * LC_ALL=C mpstat | tail -1 | awk '{print $11}' > /piler-data/piler/stat/cpu.stat /15 * * /usr/bin/indexer --quiet delta1 --rotate && sleep 2 && /usr/local/bin/indexer --quiet --merge main$ /15 * * /usr/bin/indexer --quiet tag1 --rotate /15 * * /usr/bin/indexer --quiet note1 --rotate 30 7 * * /usr/bin/php /usr/local/libexec/piler/daily-report.php /var/www/piler

Thanks for your help

Comments (31)

  1. Janos SUTO repo owner

    I'd really appreciate if you could format the copy-pasted stuff. Anyway if this is really a major bug for you, then try setting this in config-site.php:

    $config['CPU_USAGE_COMMAND'] = "LC_ALL=C mpstat | tail -1 | awk '{print $10}'";
  2. H. Hintze

    Dear Jsuto, sorry for the bad format. I realised after sending. And the red bar of course is not really a major bug, but the SMTP-Status pilier:ERROR seems to be a Show-Stopper for me

    So I put in your suggested config-site.php line. Nothing changed!

    Is it possible that putting the data on a second disk causes the error? Do I have to change more settings to point to that partition?

    Thanks for yout help

  3. Janos SUTO repo owner

    Can you telnet from piler to 172.16.173.204 tcp/25? The smtp check basically does the same. Also check out the command the gui uses to determine the current load, so show me the output of LC_ALL=C mpstat | tail -1 | awk '{print $11}' (or $10 in case of a centos/redhat distro).

  4. H. Hintze

    Thank you for your quick reply,

    172.16.173.204 = piler host (ubuntu 14.04 64bit) 172.16.173.254 = SMTP Server Do you want a telnet tcp/25 from piler to piler?

     mpstat | tail -1 | awk '{print $11}'
0,00
  5. H. Hintze

    Just to mak sure: config-site.php

    $config['SITE_NAME'] = 'Piler Mailarchiv';
$config['SITE_URL'] = 'http://172.16.173.204/';
$config['DIR_BASE'] = '/var/www/piler/';

$config['ENABLE_AUDIT'] = 1;
$config['MEMCACHED_ENABLED'] = 1;
$config['ENABLE_ON_THE_FLY_VERIFICATION'] = 1;
$config['ENABLE_SYSLOG'] = 1;
$config['PILER_HOST'] = '172.16.173.204';
$config['PILER_PORT'] = 10026;
$config['SMARTHOST'] = '172.16.173.254';
$config['SMARTHOST_PORT'] = 25;
$config['SMTP_DOMAIN'] = 'xxxxx.com';
$config['SMTP_FROMADDR'] = 'no-reply@xxxxx.com';
$config['ADMIN_EMAIL'] = 'admin@xxxxx.com';
$config['CPU_USAGE_COMMAND'] = "LC_ALL=C mpstat | tail -1 | awk '{print $11}'";

$config['DEFAULT_LANG'] = 'de';
$config['TIMEZONE'] = 'Europe/Berlin';

$config['INDEXER_BEACON'] = '/piler-data/piler/stat/indexer';
$config['PURGE_BEACON'] = '/piler-data/piler/stat/purge';
$config['PILER_HEADER_FIELD'] = 'X-piler-id: ';

$config['DIR_SPHINX'] = '/piler-data/piler/sphinx/';
$config['DIR_STAT'] = '/piler-data/piler/stat';
$config['DIR_IMAP'] = '/piler-data/piler/imap';
$config['DIR_TMP'] = '/piler-data/piler/tmp';

$config['DB_DRIVER'] = 'mysql';
$config['DB_PREFIX'] = '';
$config['DB_HOSTNAME'] = 'localhost';
$config['DB_USERNAME'] = 'piler';
  6. H. Hintze

    God morning. The telnet dialog:

    test@piler:/usr/local/bin# telnet 172.16.173.204 25
Trying 172.16.173.204...
Connected to 172.16.173.204.
Escape character is '^]'.
220 mailarchiver.localhost ESMTP
EHLO test.example.com
250-mailarchiver.localhost
250-PIPELINING
250-SIZE
250 8BITMIME
MAIL FROM:test@gmail.com
250 Ok
RCPT TO:test.account@piler.local
250 Ok
DATA
354 Send mail data; end it with <CRLF>.<CRLF>
Subject: Test Subject

Test Content.

.
250 Ok 4000000054e2ef463ab58c74007ee96889cf <>
  8. Janos SUTO repo owner

    Run ngrep -x -e port 25 when you reload the health page, ans show me the result regarding the pilerhost. Btw. what Linux distribution os this?

  10. Janos SUTO repo owner

    By default the gui runs the following command to determine the system load:

    LC_ALL=C mpstat | tail -1 | awk '{print $11}'

    and $11 should match the %idle column. Can you show me "LC_ALL=C mpstat" output?

  11. H. Hintze 
    test@piler:~# ngrep -x -e port 25
interface: eth0 (172.16.173.0/255.255.255.0)
filter: (ip or ip6) and ( port 25 )
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [S]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [AS]
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [A]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [AP]
  32 32 30 20 75 74 6d 6f    76 68 2d 72 62 78 32 2e    220 xxx.yyy.com
  67 6b 72 6d 63 2e 63 6f    6d 20 45 53 4d 54 50 20    ESMTP
  72 65 61 64 79 2e 0d 0a                               ready...
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [A]
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [AP]
  51 55 49 54 0d 0a                                     QUIT..
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [AF]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [A]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [AP]
  32 32 31 20 75 74 6d 6f    76 68 2d 72 62 78 32 2e    221 xxx.yyy.com
  67 6b 72 6d 63 2e 63 6f    6d 20 63 6c 6f 73 69 6e    closin
  67 20 63 6f 6e 6e 65 63    74 69 6f 6e 0d 0a          g connection..
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [R]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [AF]
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [R]
#
T 172.16.173.254:25 -> 172.16.173.204:37648 [A]
#
T 172.16.173.204:37648 -> 172.16.173.254:25 [R]
^Cexit
15 received, 0 dropped
  12. H. Hintze 
    test@piler:~# LC_ALL=C mpstat
Linux 3.13.0-32-generic (piler)         02/17/15        _x86_64_        (4 CPU)

13:58:28     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
13:58:28     all    0.08    0.00    0.15    0.19    0.02    0.00    0.00    0.00    0.00   99.56
  13. Janos SUTO repo owner

    Fix the following, and it should resolve the cpu usage issue:

    $config['CPU_USAGE_COMMAND'] = "LC_ALL=C mpstat | tail -1 | awk '{print $12}'";
  15. Janos SUTO repo owner

    OK. The 220 banner stuff looks ok, however for some reason the gui can't recognise it. Can you locate view/theme/default/templates/health/worker.tpl, and

    print $status

    (in the ~73th line), and change it to 

    print "*" . $h[1] . "*" . $status

    and show me what it prints to the gui.

  18. Janos SUTO repo owner

    It's odd. I can't see "Error" in the ngrep output, though it's only for the smarthost. Try the following: ngrep -d lo -x -e port 25

  21. H. Hintze

    Reloading the page doesnt give any output.

    ngrep -x -e port 25

    is still showing the same result (see above)

  22. Janos SUTO repo owner

    We can't progress further until we find the packets for piler. If they are not on lo or eth0, then where are those packets?

  24. Janos SUTO repo owner

    The smtp check does 2 things: a) telnets to the smarthost, and b) telnets to PILER_HOST. In both cases it expects a standard 220 ... smtp banner. It gets from the smarthost, but for some reason it can't from the piler daemon.

  25. H. Hintze

    So it cant be firewall issue? The problem is within the server, right? No need to check the infrastructure?

  32. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
2
Jira: the preferred issue tracker for Bitbucket. Join the team!