The tale of how I configured the services in docker on the Raspberry PI and why this might not be the best idea.

Introduction (or how it all began)

It all started ~~a long time ago~~a couple of years ago. It just so happened that I ended up in China and I had to somehow get in touch with the outside world. I did not really trust third-party VPN and proxies, so I decided to raise DigitalOcean with my proxy. It just so happened that over time, a server with a proxy has grown with different differences: from file storage ( Syncthing) to CI ( Jenkins).

Upon returning to Russia, it was decided to leave DO for some kind of self-hosting. I didn’t want to buy a separate server for this - it’s expensive, and so far there is no need, for this reason I took Raspberry PI 4B. Naturally, I had to transfer all the basic services from DO to this machine, which will be the subject of this post.

Introductory

It was necessary to configure the following services:

Syncthing - file synchronization
Jenkins - CI
Telegraf - |
Influxdb - graphics of CPU, GPU and other such
Grafana - |
Gogs - git
Radicale - sync calendars / contacts

Besides:

It was supposed to store files and data on a pair of flash drives USB3in encrypted form ( LUKS)
All web interfaces were hidden behind Nginx reverse proxy

Problems and Nuances

I would immediately like to talk about the problems that arose during the assembly of this whole thing and / or are arising now:

Raspberry PI 1A, . , , —
RaspberryPI 4B ( ) USB . — / . USB3
, . ( ) ( " ")
— syncthing, , /
60

— 502 ssh.

, Micro SD 16GB ( ) Raspbian. .

Noobs
Micro SD Fat32
( uNetbootin)
Raspberry PI

. : GUI . GUI, ssh GUI .

()

Elementary OS GUI (GParted disks) . , :

, .

— ext4 LUKS.

GParted
- Device -> Create partition table...
- gpt
- Apply
- ext4
gnome-disks ( sudo apt install gnome-disk-utility) ( Disks):
- ()
- Format partition...
- Ext 4
- Password protect volume
- Next
- :
  - - UUID,
  - - Device,

. , RaspberryPI . :

: /dev/sdb1 — , Device

dd if=/dev/urandom bs=4M count=1 of=/tmp/usb_decrypt_file
sudo cryptsetup luksAddKey /dev/sdb1 /tmp/usb_decrypt_file

: .

, . ()- /

Raspberry PI

, :

(sudo apt update && sudo apt -y dist-upgrade)
:
- docker docker-compose (sudo apt -y install docker docker-compose)
- Nginx(sudo apt -y install nginx). reverse-proxy
- - (, /root/cryptfiles; )
- /etc/crypttab :
```
usb1_crypt UUID=___UUID /root/cryptfiles/_- luks
```
- /etc/fstab
```
/dev/mapper/usb1_crypt /media/pi/usb1 ext4 defaults,nofail 0 2
```
  :
  - /dev/mapper/usb1_crypt — /dev/mapper/ + ( ) /etc/crypttab
  - /media/pi/usb1 — . (mkdir /media/pi/usb1). — , /mnt /media/$USER
- ,

Nginx

bash, reverse-proxy .

, ,
, letsencrypt

reverse-proxy nginx

, — mydomain.com. nginx /etc/nginx, /etc/nginx/autocompile.

compile_apps_configs.sh

#!/bin/bash

# Use "-pl" key in subname to make it like https://hostname/subname.
# E.g.: for https://my.domain/example will be used "-pl example"

APPS=("syncthing" "grafana" "radicale" "git" "jenkins")
APPS_PROXIES=(http://localhost:8880 http://localhost:3000 http://localhost:8882 http://localhost:8883 http://localhost:8884)
HOSTNAMES=(my.domain)
# HOSTNAMES=()

conf_file="/etc/nginx/sites-available/autocompiled.conf"
ln_file="/etc/nginx/sites-enabled/autocompiled.conf"

echo "" > "$conf_file"

for app_index in ${!APPS[*]}
do
    app="${APPS[app_index]}"
    app_proxy="${APPS_PROXIES[app_index]}"

    for host in ${HOSTNAMES[*]}
    do
        echo "`./compile_config.sh "$host" "$app_proxy" $app`" >> "$conf_file"
        echo "" >> "$conf_file"
    done
done

ln -s "$conf_file" "$ln_file"

compile_config.sh

#!/bin/bash

# FIRST ARG IS DOMAIN BASE

DOMAIN_BASE="$1"
shift

# THIRD ARG IS PROXY_PASS

PROXY_PATH=$(echo "$1" | sed -e "s/\//\\\\\//g")
PROXY_LOCATION=""
shift

HOSTNAME="$DOMAIN_BASE"

while [ -n "$1" ]
do
    case "$1" in
        "-pl") shift; PROXY_LOCATION="$(echo "$1" | sed -e "s/\//\\\\\//g")" ;;
        *) HOSTNAME="$1.$HOSTNAME" ;;
    esac

    shift
done

cat template.conf | sed "s/HOSTNAME_BASE/$DOMAIN_BASE/g" | sed "s/HOSTNAME/$HOSTNAME/g" | sed "s/PROXYPATH/$PROXY_PATH/g" | sed "s/PROXYLOCATION/$PROXY_LOCATION/"

location_template.conf

    location /PROXYLOCATION {
        proxy_pass PROXYPATH;
    }

template.conf

server {
    server_name "HOSTNAME";

    ssl_certificate /etc/letsencrypt/live/HOSTNAME_BASE/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/HOSTNAME_BASE/privkey.pem;

    listen 443 ssl;

    keepalive_timeout 60;
    ssl on;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "HIGH:!RC4:!aNULL:!MD5:!kEDH";
    add_header Strict-Transport-Security 'max-age=604800';
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;

    # set max upload size
    client_max_body_size 4000M;

    sendfile on;
    send_timeout 600s;
    proxy_connect_timeout 600;

    location /PROXYLOCATION {
        proxy_pass_request_headers on;
        proxy_pass_request_body on;
        proxy_pass PROXYPATH;
    }
}

, ./compile_apps_configs.sh nginx: sudo systemd reload nginx.

Docker

, docker docker-compose:

sudo apt install docker docker-compose

yml . :

├── doForAll
├── gogs
│   ├── docker-compose.yml
│   ├── Dockerfile
│   └── .env
├── grafana
│   ├── configs
│   │   ├── influxdb.conf
│   │   └── telegraf.conf
│   ├── docker-compose.yml
│   └── .env
├── jenkins
│   ├── docker-compose.yml
│   ├── .env
├── makeFullUpdate
├── radicale
│   ├── docker-compose.yml
│   └── .env
└── syncthing
    ├── docker-compose.yml
    └── .env

, ( grafana — , — influxdb). .

, :

.env DATA_PATH
Grafana - :
- Telegraf. .env INFLUXDB_WRITE_USER_PASSWORD, configs/telegraf.conf — password
- Grafana. .env INFLUXDB_READ_USER_PASSWORD, Grafana

root. :

#    root
sudo -i
mkdir -p /root/scripts/
touch "/root/scripts/monitor_startup_docker_container"
chmod 700 "/root/scripts/monitor_startup_docker_container"
nano "/root/scripts/monitor_startup_docker_container"

/root/scripts/monitor_startup_docker_container :

#!/bin/bash

function log() {
    echo `date`: "$@"
}

container_name="$1"

true=1
false=0

function restartContainer() {
    docker container restart "$1"
}

function checkContanerExitStatus() {
    container_name="$1"
    status_line="`docker container ps -a --filter "name=$container_name" --filter "exited=255" | grep "$container_name"`"
    [[ -z "$status_line" ]] && echo $false || echo $true
}

function checkContanerStatusIsEqual() {
    container_name="$1"
    container_dest_status="$2"
    status_line="`docker container ps -a --filter "name=$container_name" --filter "status=$container_dest_status" | grep "$container_name"`"
    [[ -z "$status_line" ]] && echo $false || echo $true
}

function isRunning() {
    echo "`checkContanerStatusIsEqual "$container_name" "running"`"
}

while [[ "`isRunning`" != "$true" ]]; do
    log check cycle "$container_name"

    if [ "`checkContanerStatusIsEqual "$container_name" "exited"`" == "$true" -o "`checkContanerStatusIsEqual "$container_name" "dead"`" == "$true" ]; then
        log restart "$container_name"

        restartContainer "$container_name"
    fi

    if [[ "`isRunning`" -eq "$false" ]]; then
        sleep 5
    else
        sleep 120
    fi

done

log started "$container_name"

crontab . root:

`crontab -e`

@reboot rm /root/startup_docker_logs

1/10 * * * * /root/scripts/monitor_startup_docker_container telegraf >> /root/startup_docker_logs
2/10 * * * * /root/scripts/monitor_startup_docker_container influxdb >> /root/startup_docker_logs
3/10 * * * * /root/scripts/monitor_startup_docker_container grafana >> /root/startup_docker_logs
3/10 * * * * /root/scripts/monitor_startup_docker_container jenkins >> /root/startup_docker_logs
3/10 * * * * /root/scripts/monitor_startup_docker_container gogs >> /root/startup_docker_logs
1/10 * * * * /root/scripts/monitor_startup_docker_container radicale >> /root/startup_docker_logs
2/10 * * * * /root/scripts/monitor_startup_docker_container syncthing >> /root/startup_docker_logs

, , On average, almost no maintenance. What was not covered in this article:

Setting access to SSHthe raspberry: there are a large number of tutorials on this topic, here is an example with DigitalOcean
Configure the services themselves
Buying and Configuring DNS for Domains

I will be glad to comments and useful comments.

How to raise your server on docker based RaspberryPI