Docker enhancements

[Alexander Noack]

I have been running Piler for 7+ years on Debian and have been a big fan ever since.

Recently I was tasked with deploying Piler to Docker and I would like to get some feedback from other users, using Piler in Docker.

This is my Dockerfile (package lists from the original Dockerfile were a great help!)

FROM debian:buster-slim as builder

ENV DEBIAN_FRONTEND="noninteractive" 

RUN apt-get update && \
    apt-get -y --no-install-recommends install \
       build-essential make libmariadb-dev-compat libmemcached-dev libssl-dev \
       libpoppler-dev libtre-dev libzip-dev mariadb-client sphinxsearch \
       php-cli catdoc poppler-utils unrtf tnef wget ca-certificates git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src

RUN wget https://bitbucket.org/jsuto/piler/downloads/xlhtml-0.5.1-sj-mod.tar.gz && tar xfz xlhtml-0.5.1-sj-mod.tar.gz && cd xlhtml-0.5.1-sj-mod && \
     ./configure && make && make install && ldconfig && cd ..

RUN git clone https://bitbucket.org/jsuto/piler.git && cd piler && \
     useradd -ms /bin/bash piler && ./configure --sysconfdir=/etc --localstatedir=/var --with-database=mysql --with-piler-user=piler --enable-memcached && \
     make && make install && ldconfig && \
     sed -i "s/define('SPHINX_VERSION', 331);/define('SPHINX_VERSION', 222);/" etc/sphinx.conf.dist && php etc/sphinx.conf.dist > /etc/piler/sphinx.conf.dist

FROM debian:buster-slim

LABEL description="piler debian buster-slim image"

ENV DEBIAN_FRONTEND="noninteractive"

ARG SMARTHOST
ARG SMTPMASQ

RUN apt-get update && \
    apt-get -y --no-install-recommends install \
       openssl ca-certificates sphinxsearch mariadb-client libtre5 catdoc unrtf poppler-utils tnef libzip4 \
       python3 python3-mysqldb curl rsyslog cron dma apt-transport-https gnupg sysstat locales sudo && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN echo "SMARTHOST ${SMARTHOST}\nSECURETRANSFER\nSTARTTLS\nMASQUERADE ${SMTPMASQ}" > /etc/dma/dma.conf && \
        chmod a+r /etc/dma/dma.conf && \
        sed -i 's/# de_DE.UTF-8 UTF-8/de_DE.UTF-8 UTF-8/' /etc/locale.gen && \
        sed -i 's/# en_GB.UTF-8 UTF-8/en_GB.UTF-8 UTF-8/' /etc/locale.gen && \
        locale-gen && \
        curl -s https://packages.sury.org/php/apt.gpg | apt-key add - && echo 'deb https://packages.sury.org/php/ buster main' > /etc/apt/sources.list.d/deb.sury.org.list

RUN apt-get update && \
        apt-get install -y --no-install-recommends \
                apache2 \
                libapache2-mod-php7.4 \
                libapache2-mod-auth-gssapi \
                php7.4 \
                php7.4-curl \
                php7.4-gd \
                php7.4-ldap \
                php7.4-memcached \
                php7.4-mysql \
                php7.4-zip && \
        apt-get clean && \
        rm -f /etc/alternatives/php && ln -s /usr/bin/php7.4 /etc/alternatives/php && \
        rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* /var/cache/apt/archive/*.deb && \
        a2enmod rewrite && a2enmod ssl && a2enmod auth_gssapi

COPY --from=builder /usr/local /usr/local
COPY --from=builder /var/piler /var/piler
COPY --from=builder /etc/piler /etc/piler
COPY --from=builder /etc/init.d/rc.* /etc/init.d/

RUN useradd -ms /bin/bash piler && ldconfig && echo "module(load=\"imuxsock\")\n*.* /var/piler/syslog" > /etc/rsyslog.conf && \
        echo 'www-data ALL=NOPASSWD: /etc/init.d/rc.piler reload' > /etc/sudoers

VOLUME ["/etc/piler"]
VOLUME ["/var/piler"]

EXPOSE 25 80 443

COPY start.sh /start.sh

ENTRYPOINT ["/start.sh"]

A valid Apache config file needs to be mounted to /etc/apache2/sites-enabled/000-default.conf!

‌

1. I added a mail forwarder, that sends mail from within the container to a smarthost. This seems to be missing from the original Dockerfile (I overlooked that Piler sends mail directly via Smarthost using PHP )
2. Syslog was driving me crazy - ideally I would like to have everything on Stdout to leverage Docker log shipping via ELK/EFK. Piler binaries and PHP files have a hard dependency on syslog - I opted for a syslog file on my /var/piler bind mount.
3. Running multiple processes in a single container is not the true docker-way… I tried separating piler/piler-smtp/searchd/apache but gave up due to the syslog dependency for proper logging.

‌

Any hints on those topics are greatly appreciated!

[Janos SUTO]

You are right, docker prefers a single app in a container, however, it’s not a rigid rule you are obliged to follow. With that being said I’ve already separated the mysql db to run in its own container. You may create an image for sphinx as well. It’s also possible to run piler-smtp and piler daemons in separated containers, though they share both /etc/piler and /var/piler volumes. I may reason that it’s worth to keep them together.

I’ve been thinking about adding support logging to stdout, but I’m still reluctant to give up syslog, as it’s an industry standard thing and a convenient way to have all piler related log at a single place. Also, you may configure rsyslog.conf to ship the log entries to an external source, eg. Graylog, EFK, etc. This is what I do in my dev infrastructure when running my tests in Docker.

I’ve just did a quick research on sending php logs to stdout, and it seems that it doesn’t work with php-fpm (at least what I use with nginx). However, you have apache with mod_php, and it might work differently. Even though fixing rsyslog config is much easier to ship the logs where you want them to be.