Helmfile - a wrapper for helm , which allows you to describe many helm releases in one place, parameterize their charts for several environments, and also set the order of their deployment.
You can read about helmfile itself and examples of its use in the readme and best practices guide .
We will get acquainted with non-obvious ways to describe releases in helmfile
Suppose we have a bunch of helm charts (for example, let there be postgres and some kind of backend application) and several environments (several kubernetes clusters, several namespace'ov or several of both). We take helmfile, read the documentation and begin to describe our environments and releases:
.
βββ envs
β βββ devel
β β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
β βββ production
β βββ values
β βββ backend.yaml
β βββ postgres.yaml
βββ helmfile.yaml
helmfile.yaml
environments:
devel:
production:
releases:
- name: postgres
labels:
app: postgres
wait: true
chart: stable/postgresql
version: 8.4.0
values:
- envs/{{ .Environment.Name }}/values/postgres.yaml
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: 1.0.5
needs:
- postgres
values:
- envs/{{ .Environment.Name }}/values/backend.yaml
We got 2 environments: devel , production - each has its own values ββfor helm release charts. We will deploy to them like this:
helmfile -n <namespace> -e <env> apply
Different versions of helm charts in different environments
What if we need to roll out different versions of the backend in different environments? How to parameterize the release version? Ambient values ββavailable through{{ .Values }}
helmfile.yaml
environments:
devel:
+ values:
+ - charts:
+ versions:
+ backend: 1.1.0
production:
+ values:
+ - charts:
+ versions:
+ backend: 1.0.5
...
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
- version: 1.0.5
+ version: {{ .Values.charts.versions.backend }}
...
A different set of applications in different environments
, production
postgres, , k8s postgres? (labels)
helmfile -n <namespace> -e devel apply
helmfile -n <namespace> -e production -l app=backend apply
, , , . ? , "" ,
.
βββ envs
β βββ devel
β β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
β βββ production
β βββ values
β βββ backend.yaml
β βββ postgres.yaml
+ βββ releases
+ β βββ backend.yaml
+ β βββ postgres.yaml
βββ helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
- releases:
- - name: postgres
- labels:
- app: postgres
- wait: true
- chart: stable/postgresql
- version: 8.4.0
- values:
- - envs/{{ .Environment.Name }}/values/postgres.yaml
- - name: backend
- labels:
- app: backend
- wait: true
- chart: private-helm-repo/backend
- version: {{ .Values.charts.versions.backend }}
- needs:
- - postgres
- values:
- - envs/{{ .Environment.Name }}/values/backend.yaml
+ ---
+ bases:
+ {{- range .Values.apps }}
+ - releases/{{ . }}.yaml
+ {{- end }}
releases/postgres.yaml
releases:
- name: postgres
labels:
app: postgres
wait: true
chart: stable/postgresql
version: 8.4.0
values:
- envs/{{ .Environment.Name }}/values/postgres.yaml
releases/backend.yaml
releases:
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: {{ .Values.charts.versions.backend }}
needs:
- postgres
values:
- envs/{{ .Environment.Name }}/values/backend.yaml
bases:
yaml ---
, releases ( , helmDefaults) environments
postgres production. !
, , helm , , , , affinity
, - , .
2 values: , , , .
.
βββ envs
+ β βββ default
+ β β βββ values
+ β β βββ backend.yaml
+ β β βββ postgres.yaml
β βββ devel
β β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
β βββ production
β βββ values
β βββ backend.yaml
β βββ postgres.yaml
βββ releases
β βββ backend.yaml
β βββ postgres.yaml
βββ helmfile.yaml
releases/backend.yaml
releases:
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: {{ .Values.charts.versions.backend }}
needs:
- postgres
values:
+ - envs/default/values/backend.yaml
- envs/{{ .Environment.Name }}/values/backend.yaml
envs/default/values/backend.yaml
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- backend
topologyKey: "kubernetes.io/hostname"
helm
, ingress β hosts:
, , ? values, , .gotmpl
, helmfile , .
.
βββ envs
β βββ default
β β βββ values
- β β βββ backend.yaml
- β β βββ postgres.yaml
+ β β βββ backend.yaml.gotmpl
+ β β βββ postgres.yaml.gotmpl
β βββ devel
β β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
β βββ production
β βββ values
β βββ backend.yaml
β βββ postgres.yaml
βββ releases
β βββ backend.yaml
β βββ postgres.yaml
βββ helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
+ - global:
+ ingressDomain: k8s.devel.domain
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
+ - global:
+ ingressDomain: production.domain
---
bases:
{{- range .Values.apps }}
- releases/{{ . }}.yaml
{{- end }}
envs/default/values/backend.yaml.gotmpl
ingress:
enabled: true
paths:
- /api
hosts:
- {{ .Values.global.ingressDomain }}
envs/default/values/postgres.yaml.gotmpl
ingress:
enabled: true
paths:
- /
hosts:
- postgres.{{ .Values.global.ingressDomain }}
, ingress postgres β , , - ingress
(secrets)
helm secrets . , secrets, , default.yaml.gotmpl , , . , , .
.
βββ envs
β βββ default
β β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
β βββ devel
β β βββ values
β β β βββ backend.yaml
β β β βββ postgres.yaml
+ β β βββ secrets.yaml
β βββ production
β βββ values
β β βββ backend.yaml
β β βββ postgres.yaml
+ β βββ secrets.yaml
βββ releases
β βββ backend.yaml
β βββ postgres.yaml
βββ helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
- global:
ingressDomain: k8s.devel.domain
+ secrets:
+ - envs/devel/secrets.yaml
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
- global:
ingressDomain: production.domain
+ secrets:
+ - envs/production/secrets.yaml
---
bases:
{{- range .Values.apps }}
- releases/{{ . }}.yaml
{{- end }}
envs/devel/secrets.yaml
secrets:
elastic:
password: ENC[AES256_GCM,data:hjCB,iv:Z1P6/6xBJgJoKLJ0UUVfqZ80o4L84jvZfM+uH9gBelc=,tag:dGqQlCZnLdRAGoJSj63rBQ==,type:int]
...
envs/production/secrets.yaml
secrets:
elastic:
password: ENC[AES256_GCM,data:ZB/VpTFk8f0=,iv:EA//oT1Cb5wNFigTDOz3nA80qD9UwTjK5cpUwLnEXjs=,tag:hMdIUaqLRA8zuFBd82bz6A==,type:str]
...
envs/default/values/backend.yaml.gotmpl
elasticsearch:
host: elasticsearch
port: 9200
password: {{ .Values | getOrNil "secrets.elastic.password" | default "password" }}
envs/devel/values/backend.yaml
elasticsearch:
host: elastic-0.devel.domain
envs/production/values/backend.yaml
elasticsearch:
host: elastic-0.production.domain
, getOrNil
β go helmfile, , .Values.secrets
, , default
-
The described things seem pretty obvious, but the information on a convenient deployment description in several environments using helmfile is very scarce, and I love IaC (Infrastructure-as-Code) and I want to have a clear description of the deployment state.
In conclusion, I want to add that the variables for the default environment can, in turn, be parameterized by the OS environment variables of a runner from which the deployment will be launched, and thus get dynamic environments
helmfile.yaml
environments:
default:
values:
- global:
clusterDomain: {{ env "CLUSTER_DOMAIN" | default "cluster.local" }}
ingressDomain: {{ env "INGRESS_DOMAIN" }}