Get best of the week

Secure secrets when building in Docker Compose

A translation of the article was prepared ahead of the start of the Python Web Developer course .



When you build a Docker image, you may need secrets, such as a password for a private package repository. You don’t want this secret to end up in the image, because then anyone who gets access to the image will get access to your private repository.
Note : If you think “Why not just use environment variables?” Which are used for secrets in runtime when creating an image. This article focuses on build secrets that are used when creating an image using a Docker file.
Newer versions of Docker maintain secrets using the experimental BuildKit service , and in Docker Compose 1.25 and later, you can already create images using BuildKit. Unfortunately, as of March 2020, the ability to safely work with secrets from Compose is still under development .

So what to do now?

In today's article, I’ll show how you can use the same Dockerfile to securely create secrets images without losing the benefits of rapid development using Docker Compose.

Two options for using your dockerfile


It is very convenient to use the same Dockerfile for production and for local development with Docker Compose. Usually you use the Dockerfile along with the build function from Compose:

version: "3.7"
services:
  yourapp:
    build:
      context: "."

Then you can do:

$ docker-compose up

With this command, you can (re) assemble the image and then run it.
For use on production, you collect the image and send it with push :

$ docker build -t myimage .
$ docker push myimage

And while everything is going well. But what if you need a secret build?

First try (unsafe)


Suppose you have a script that needs a secret build, for example, to download a Python package from a private DevPI repository . For simplicity, we will simply derive the secret with the help use-secret.shto show that we have it.

#!/bin/bash
set -euo pipefail

echo "Secret is: $THEPASSWORD"

You can simply pass on the secret using the Docker build arguments, as they are supported everywhere, including in Docker Compose.
Note : Going beyond the scope of our discussion, I want to say that the use of Docker files in this article is not the best practice, however, excessive complexity can interfere with conveying the main meaning of the article.
Therefore, if you want to run your Python application on production with Docker, here are two good ways to do this:


FROM python:3.8-slim-buster
# Using ARG for build secrets is INSECURE!
ARG THEPASSWORD
COPY use_secret.sh .
RUN ./use_secret.sh

We can write docker-compose.yml, which will be transmitted in secret:

version: "3.7"
services:
  yourapp:
    build:
      context: "."
      args:
        THEPASSWORD: "s3kr!t"

For local work, you can run or build an image using Compose:

$ docker-compose build | grep Secret
Secret is: s3kr!t

And everything's good.

And we can also assemble the image using Docker, in preparation for moving it to the image registry:

$ docker build -t myimage --build-arg THEPASSWORD=s3krit . | grep Secret
Secret is: s3krit

Doing so is unsafe: never do it . If we decide to look at the layers of the image, we will see the secret lying in it!

$ docker history myimage
IMAGE               CREATED              CREATED BY                                      SIZE
c224231ec30b        47 seconds ago       |1 THEPASSWORD=s3krit /bin/sh -c ./use_secre…   0B
6aef62acf0db        48 seconds ago       /bin/sh -c #(nop) COPY file:7aa28bbe6595e0d5…   62B
f88b19ca8e65        About a minute ago   /bin/sh -c #(nop)  ARG THEPASSWORD              0B
...

Anyone who gains access to this image will recognize your password. What can be done then?

BuildKit Secrets (Partial Solution)


BuildKit is a new (and still experimental) solution for creating Docker images, which, among other things, adds support for the safe use of secrets during assembly . Docker Compose has BuildKit support since v1.25.

But there is one problem: Docker Compose still does not support BuildKit secrets functionality. Now work is underway on this, but as of March 2020, there are no ready-made solutions, not to mention a stable release.

Therefore, we are going to combine these two approaches:

  • Docker Compose will continue to use build arguments to pass secrets;
  • For a production image built using docker build, we use BuildKit to pass secrets.

This way we can use the same Dockerfile to work locally and on production.
BuildKit works with secrets as follows: the file with secrets is mounted in a temporary directory while the RUN command is executed, for example, in /var/secrets/thepassword. Since it is mounted during the execution of the RUN command, it will not be added to the final image.

We will modify the file use_secret.shto check if such a temporary file exists. If exists, it uses its environment variable settings $THEPASSWORD. If the file does not exist, then we will return to the environment variable. That is, it $THEPASSWORDcan be installed using BuildKit or build arguments:

#!/bin/bash
set -euo pipefail
if [ -f /run/secrets/thepassword ]; then
   export THEPASSWORD=$(cat /run/secrets/thepassword)
fi

echo "Secret is: $THEPASSWORD"

Then we will modify the Dockerfile to add the BuildKit and mount the secret:

# syntax = docker/dockerfile:1.0-experimental
FROM python:3.8-slim-buster
# Only use the build arg for local development:
ARG THEPASSWORD
COPY use_secret.sh .
# Mount the secret to /run/secrets:
RUN --mount=type=secret,id=thepassword ./use_secret.sh

docker-compose.ymlWe do not change the file :

version: "3.7"
services:
  yourapp:
    build:
      context: "."
      args:
        THEPASSWORD: "s3kr!t"

Now you need to define two environment variables, one of which will tell Docker that you need to use BuildKit, the second that Compose needs to use the CLI version of Docker and, therefore, BuildKit. We will also write the secret to the file:

$ export DOCKER_BUILDKIT=1
$ export COMPOSE_DOCKER_CLI_BUILD=1
$ echo 's3krit' > /tmp/mypassword

With Compose, we use the build arguments:

$ docker-compose build --progress=plain \
    --no-cache 2>&1 | grep Secret
#12 0.347 Secret is: s3kr!t

Please note that it is --no-cachenecessary to understand that the image will really rebuild if you yourself run all of the above. In real life, this parameter can be omitted. 2>&1redirect stderrto stdoutfor correct operation grep.

When we are ready to build on production, we use docker build with the functionality of secrets from BuildKit:

$ docker build --no-cache -t myimage \
    --secret id=thepassword,src=/tmp/mypassword \
    --progress=plain . 2>&1 | grep Secret
#12 0.359 Secret is: s3krit

Is it safe?


Let's make sure the secret is not visible:

$ docker history myimage
IMAGE               CREATED             CREATED BY                                      SIZE
a77f3c32b723        25 seconds ago      RUN |1 THEPASSWORD= /bin/sh -c ./use_secret.…   0B
<missing>           25 seconds ago      COPY use_secret.sh . # buildkit                 160B
...

Hooray! We passed the secret to the same Dockerfile using Compose and docker build, and in the latter case did not reveal the secret from the assembly.


Learn more about the course.

More articles:


All Articles