Pragmatism in the real world

Prevent the Docker container from taking 10 seconds to stop

For one project that I’m working on the PHP-FPM-based Docker container is built from a Ubuntu container with PHP is installed into it.

A little like this:

FROM ubuntu:22.04

RUN apt-get update && apt-get upgrade -y && apt-get install -y gnupg curl

# Register the Ondrej package repo for PHP
RUN mkdir -p /etc/apt/keyrings \
    curl -sS 'https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x14aa40ec0831756756d7f66c4f4ea0aae5267a6c' | gpg --dearmor | tee /etc/apt/keyrings/ppa_ondrej_php.gpg > /dev/null \
    && echo "deb [signed-by=/etc/apt/keyrings/ppa_ondrej_php.gpg] https://ppa.launchpadcontent.net/ondrej/php/ubuntu jammy main" > /etc/apt/sources.list.d/ppa_ondrej_php.list \
    && apt-get update

# Install PHP
RUN apt-get install -y php8.2-cli php8.2-fpm php8.2-dev \
       php8.2-sqlite3 php8.2-gd php8.2-intl php8.2-imagick \
       php8.2-mbstring php8.2-xml php8.2-zip php8.2-bcmath \
       php8.2-curl php8.2-pdo php8.2-opcache php8.2-gettext \
       php-pear


# other stuff, such as copying over settings files


# Run php-fpm
WORKDIR /var/www/html
EXPOSE 9000
CMD /usr/sbin/php-fpm8.2 -F -R

This works fine, however, when stopping the containers with docker compose down, I noticed that it takes 10 seconds for the PHP container to stop:

$ docker compose down
[+] Running 5/5
 ✔ Container portal-web-1   Removed                            0.1s 
 ✔ Container portal-db-1    Removed                            0.3s 
 ✔ Container portal-mail-1  Removed                            0.1s 
 ✔ Container portal-php-1   Removed                           10.1s 
 ✔ Network portal_default   Removed                            0.0s 

Googling around, it seems that 10 seconds is a Docker timeout, so it seems that the PHP container isn’t shutting down itself, but is being killed by Docker.

Further googling and many tests later, I found that the solution is to use exec form for the CMD:

CMD ["/usr/sbin/php-fpm8.2", "-F", "-R"]

Once I had made this change, stopping the PHP container takes 0.1s as I would hope:

$ docker compose down
[+] Running 5/5
 ✔ Container portal-web-1   Removed                            0.2s 
 ✔ Container portal-mail-1  Removed                            0.1s 
 ✔ Container portal-db-1    Removed                            0.3s 
 ✔ Container portal-php-1   Removed                            0.1s 
 ✔ Network portal_default   Removed                            0.0s 

Much better!

Aside: why does this work?

The underlying reason is the way unix-like operations systems handle termination of processes. When a process needs to terminate via say a SIGTERM signal, it will terminate and become one of those “zombie” processes that you sometimes see when you type ps. It’s parent process then “waits on” (or “reaps”) it for the exit code and then it is really gone. If that process that’s been terminated has children, then they now no longer have a parent and so the init process (PID 1) takes them over (“adopts” them) and all is well with the system.

With Docker however, we usually run our process as PID 1 and we need this process to correctly reap child processes and adopt orphans (i.e. act like init). Specifically, CMD /usr/sbin/php-fpm8.2 -F -R will start Bash as PID 1 and then php-fpm as a child. This seems to be fine as Bash can reap and adopt processes, except that Bash doesn’t handle signals properly!

On shutdown, Docker sends SIGTERM to Bash, which terminates. However, Bash, does not send SIGTERM to its child processes (php-fpm in this case) and so it sits there waiting php-fpm to terminate. As php-fpm doesn’t know that it needs to terminate, we have to wait 10 seconds until Docker gets bored and kills the container via SIGKILL.

Knowing this, the solution becomes obvious: we want php-fpm to be PID 1 as it knows how to handle it’s children. This happens when you specify the CMD using exec form as a shell is not created which leaves php-fpm as PID 1 and the SIGTERM is handled correctly.

This leads naturally on to another way to solve it: use the shell’s exec command:

CMD exec /usr/sbin/php-fpm8.2 -F -R

This has the same effect as bash exec will replace the current shell with the command being executed.

Another solution is to run a real init process as PID 1. There are many options available out there.

Thoughts? Leave a reply

Your email address will not be published. Required fields are marked *