Get best of the week

Apprendre à déployer des microservices. Partie 3. Heaume


Bonjour, Habr!


Il s'agit de la troisième partie d'une série d'articles «Apprendre à déployer des microservices » et aujourd'hui nous parlerons de Helm 3. Dans la dernière partie, nous avons créé la configuration Kubernetes pour un projet de formation de 2 microservices (backend et passerelle) et déployé tout cela dans Google Kubernetes Engine. Dans cet article, nous allons écrire un graphique Helm pour notre système, créer un référentiel pour celui-ci basé sur les pages GitHub et déployer un projet dans GKE à l'aide de Helm.


:


  1. Spring Boot, Docker


    : Java 11, Spring Boot, Docker, image optimization


  2. Kubernetes Google Kubernetes Engine


    : Kubernetes, GKE, resource management, autoscaling, secrets


  3. Helm 3


    : Helm 3, chart deployment


  4. Jenkins


    : Jenkins configuration, plugins, separate configs repository



Helm — , , Kubernetes. Helm — yaml- , . Helm-. .


Helm Kubernetes , . values.yaml, Kubernetes-. , .


GitHub .



:


helm create <chart-name>

Helm. msvc-chart :


.
└── msvc-chart
    ├── charts
    ├── Chart.yaml
    ├── templates
    │   ├── backend.yaml
    │   ├── gateway.yaml
    │   ├── _helpers.tpl
    │   ├── NOTES.txt
    │   ├── secrets.yaml
    │   ├── service-account.yaml
    │   ├── tests
    │   │   └── interaction-test.yaml
    │   └── urls-config.yaml
    ├── values.schema.json
    └── values.yaml

  • charts/ — . , .


  • Chart.yaml — , .


  • *templates/*.yaml* — Kuberenetes.


  • templates/NOTES.txt — , .


  • *templates/tests/*.yaml* — Helm.


  • templates/_helpers.tpl — , . tpl.


  • values.yaml — .


  • values.schema.json — JSON- values.yaml.



values.yaml


— values.yaml. . .


backend:
  deployment:
    name:
    replicas: 2
  container:
    name:
    resources: {}
#      limits:
#        memory: 1024Mi
#        cpu: 500m
#      requests:
#        memory: 512Mi
#        cpu: 100m
  service:
    name:
    port: 8080
  image:
    name: anshelen/microservices-backend
    tag: latest
    pullPolicy: IfNotPresent
  hpa:
    enabled: false
    name:
    minReplicas: 1
    maxReplicas: 3
    targetCPUUtilizationPercentage: 50

gateway:
  deployment:
    name:
    replicas: 2
  container:
    name:
    resources: {}
#      limits:
#        memory: 1024Mi
#        cpu: 500m
#      requests:
#        memory: 512Mi
#        cpu: 100m
  service:
    name:
    port: 80
#   Can be one of ClusterIP, NodePort or LoadBalancer
    type: LoadBalancer
  image:
    name: anshelen/microservices-gateway
    tag: latest
    pullPolicy: IfNotPresent
  hpa:
    enabled: false
    name:
    minReplicas: 1
    maxReplicas: 3
    targetCPUUtilizationPercentage: 50

secrets:
  secret: default-secret

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the release and
  # chart names
  name:

, . , (№ gateway.service.type). , Kubernetes- (№ gateway.container.resources).


values.schema.json JSON-, . , , .


values.schema.json
{
  "$schema": "https://json-schema.org/draft-07/schema#",
  "properties": {
    "backend": {
      "properties": {
        "deployment": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "replicas": {
              "minimum": 1,
              "type": "integer"
            }
          },
          "type": "object"
        },
        "container": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "resources": {
              "properties": {
                "limits": {
                  "properties": {
                    "memory": {
                      "type": ["string", "number"]
                    },
                    "cpu": {
                      "type": ["string", "number"]
                    }
                  },
                  "type": "object"
                },
                "requests": {
                  "properties": {
                    "memory": {
                      "type": ["string", "null"]
                    },
                    "cpu": {
                      "type": ["string", "null"]
                    }
                  },
                  "type": "object"
                }
              },
              "type": "object"
            }
          },
          "type": "object"
        },
        "service": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "port": {
              "minimum": 1,
              "type": "integer"
            }
          },
          "type": "object"
        },
        "image": {
          "properties": {
            "name": {
              "type": "string"
            },
            "tag": {
              "type": "string"
            },
            "pullPolicy": {
              "enum": ["IfNotPresent", "Always", "Never"]
            }
          },
          "type": "object"
        },
        "hpa": {
          "properties": {
            "enabled": {
              "type": "boolean"
            },
            "name": {
              "type": ["string", "null"]
            },
            "minReplicas": {
              "minimum": 1,
              "type": "integer"
            },
            "maxReplicas": {
              "minimum": 1,
              "type": "integer"
            },
            "targetCPUUtilizationPercentage": {
              "minimum": 1,
              "maximum": 99,
              "type": "integer"
            }
          },
          "type": "object"
        }
      },
      "type": "object"
    },
    "gateway": {
      "properties": {
        "deployment": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "replicas": {
              "minimum": 1,
              "type": "integer"
            }
          },
          "type": "object"
        },
        "container": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "resources": {
              "properties": {
                "limits": {
                  "properties": {
                    "memory": {
                      "type": ["string", "number"]
                    },
                    "cpu": {
                      "type": ["string", "number"]
                    }
                  },
                  "type": "object"
                },
                "requests": {
                  "properties": {
                    "memory": {
                      "type": ["string", "null"]
                    },
                    "cpu": {
                      "type": ["string", "null"]
                    }
                  },
                  "type": "object"
                }
              },
              "type": "object"
            }
          },
          "type": "object"
        },
        "service": {
          "properties": {
            "name": {
              "type": ["string", "null"]
            },
            "port": {
              "minimum": 1,
              "type": "integer"
            },
            "type": {
              "enum": ["ClusterIP", "NodePort", "LoadBalancer"]
            }
          },
          "type": "object"
        },
        "image": {
          "properties": {
            "name": {
              "type": "string"
            },
            "tag": {
              "type": "string"
            },
            "pullPolicy": {
              "enum": ["IfNotPresent", "Always", "Never"]
            }
          },
          "type": "object"
        },
        "hpa": {
          "properties": {
            "enabled": {
              "type": "boolean"
            },
            "name": {
              "type": ["string", "null"]
            },
            "minReplicas": {
              "minimum": 1,
              "type": "integer"
            },
            "maxReplicas": {
              "minimum": 1,
              "type": "integer"
            },
            "targetCPUUtilizationPercentage": {
              "minimum": 1,
              "maximum": 99,
              "type": "integer"
            }
          },
          "type": "object"
        }
      },
      "type": "object"
    },
    "secrets": {
      "properties": {
        "secret": {
          "type": ["number", "string"]
        }
      },
      "type": "object"
    },
    "createAccount": {
      "properties": {
        "create": {
          "type": "boolean"
        },
        "name": {
          "type": ["string", "null"]
        }
      },
      "type": "object"
    }
  },
  "title": "Values",
  "type": "object"
}


Helm , Kubernetes.


— . Helm , (Release), (Values) (Chart). , secrets.secret {{ .Values.secrets.secret }}.


, "" ("pipeline") — . , : {{ .Chart.AppVersion | quote }}. .


, . define:


{{- define "msvc-chart.someFragment" -}}
...
{{- end -}}

include:


{{ include "msvc-chart.someFragment" . }}

{{- {{ , . . , {{- include "msvc-chart.someFragment" . | nindent 4 }} 4 .


.


_helpers.tpl


Helm . Kubernetes , . , . , , . — , , . Release.Name.



:


{{- define "msvc-chart.selectorLabels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}

.Chart.Name ('msvc-chart' ). , Helm.


:


{{- define "msvc-chart.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

, , 63 ( Kubernetes) . Helm. Kubernetes- ( DNS ).


, -Kubernetes:


{{- define "msvc-chart.labels" -}}
helm.sh/chart: {{ include "msvc-chart.chart" . }}
{{ include "msvc-chart.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

, 'msvc-project', :


helm.sh/chart: msvc-chart-1.0.0
app.kubernetes.io/name: msvc-chart
app.kubernetes.io/instance: msvc-project
app.kubernetes.io/version: 1.0.0
app.kubernetes.io/managed-by: Helm

Kubernetes


Kubernetes ServiceAccount. . , , . ServiceAccount . ServiceAccount , Jenkins .


:


  • serviceAccount.create=true, serviceAccount.name. .
  • serviceAccount.create=false, serviceAccount.name 'default', .

{{- define "msvc-chart.serviceAccountName" -}}
{{- if .Values.serviceAccount.create -}}
    {{ default (printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-") .Values.serviceAccount.name }}
{{- else -}}
    {{ default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}

if default, , null.


Kubernetes , .


:


{{- define "msvc-chart.gateway.defaultName" -}}
{{- printf "gateway-%s" .Release.Name -}}
{{- end -}}

{{- define "msvc-chart.gateway.deployment.name" -}}
{{- default (include "msvc-chart.gateway.defaultName" .) .Values.gateway.deployment.name | trunc 63 | trimSuffix "-" -}}
{{- end -}}


, . , , Helm Kubernetes- . , . . Helm checksum/config, , . Helm , , .


:


{{- define "msvc-chart.propertiesHash" -}}
{{- $secrets := include (print $.Template.BasePath "/secrets.yaml") . | sha256sum -}}
{{- $urlConfig := include (print $.Template.BasePath "/urls-config.yaml") . | sha256sum -}}
{{ print $secrets $urlConfig | sha256sum }}
{{- end -}}

print , $.Template.BasePath (_helpers.tpl). include , - $secrets. ConfigMap, .


, include '. ', '\$' Template. Helm '. ' (scope), '\$' — , . , . . .



Secret (secrets.yaml)


apiVersion: v1
kind: Secret
metadata:
  name: {{ include "msvc-chart.secrets.defaultName" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
type: Opaque
stringData:
  secret: {{ .Values.secrets.secret }}

ConfigMap (urls-config.yaml)


apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "msvc-chart.urlConfig.defaultName" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
data:
  BACKEND_URL: "http://{{ include "msvc-chart.backend.service.name" . }}:{{ .Values.backend.service.port }}/"

ServiceAccount (service-account.yaml)


{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
  name: {{ include "msvc-chart.serviceAccountName" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: "{{ include "msvc-chart.serviceAccountName" . }}-binding"
  labels:  
    {{- include "msvc-chart.labels" . | nindent 4 }}
subjects:
- kind: ServiceAccount
  name: {{ include "msvc-chart.serviceAccountName" . }}
  namespace: {{ .Release.Namespace }}
roleRef:
  kind: ClusterRole
  name: admin
  apiGroup: rbac.authorization.k8s.io
{{- end -}}

serviceAccount.create=true, .


Deployments (backend.yaml gateway.yaml)


gateway.yaml , , backend.yaml — . , HorizontalPodAutoscaler', . , gateway.container.resources.requests.cpu — . Kubernetes .


gateway.yaml:


apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "msvc-chart.gateway.deployment.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: gateway
spec:
  replicas: {{ .Values.gateway.deployment.replicas }}
  selector:
    matchLabels:
      {{- include "msvc-chart.selectorLabels" . | nindent 6 }}
      tier: gateway
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        checksum/config: {{ include "msvc-chart.propertiesHash" . }}
      labels:
        {{- include "msvc-chart.selectorLabels" . | nindent 8 }}
        tier: gateway
    spec:
      serviceAccountName: {{ include "msvc-chart.serviceAccountName" . }}
      containers:
        - name: {{ include "msvc-chart.gateway.container.name" . }}
          image: "{{ .Values.gateway.image.name }}:{{ .Values.gateway.image.tag }}"
          imagePullPolicy: {{ .Values.gateway.image.pullPolicy }}
          envFrom:
            - configMapRef:
                name: {{ include "msvc-chart.urlConfig.defaultName" . }}
          env:
            - name: SECRET
              valueFrom:
                secretKeyRef:
                  name: {{ include "msvc-chart.secrets.defaultName" . }}
                  key: secret
          readinessProbe:
            httpGet:
              path: /actuator/health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 3
          ports:
            - containerPort: 8080
              protocol: TCP
          resources:
            {{- toYaml .Values.gateway.container.resources | nindent 12 }}

---

apiVersion: v1
kind: Service
metadata:
  name: {{ include "msvc-chart.gateway.service.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: gateway
spec:
  type: {{ .Values.gateway.service.type }}
  ports:
    - port: {{ .Values.gateway.service.port }}
      protocol: TCP
      targetPort: 8080
      name: http
  selector:
    {{- include "msvc-chart.selectorLabels" . | nindent 4 }}
    tier: gateway

---

{{- if .Values.gateway.hpa.enabled -}}
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "msvc-chart.gateway.hpa.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: gateway
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "msvc-chart.gateway.deployment.name" . }}
  minReplicas: {{ .Values.gateway.hpa.minReplicas }}
  maxReplicas: {{ .Values.gateway.hpa.maxReplicas }}
  targetCPUUtilizationPercentage: {{ .Values.gateway.hpa.targetCPUUtilizationPercentage }}
{{- end -}}

"checksum/config", , ServiceAccount (spec.template.spec.serviceAccountName). toYaml backend.container.resources values.yaml .


.


backend.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "msvc-chart.backend.deployment.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: backend
spec:
  replicas: {{ .Values.backend.deployment.replicas }}
  selector:
    matchLabels:
      {{- include "msvc-chart.selectorLabels" . | nindent 6 }}
      tier: backend
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        {{- include "msvc-chart.selectorLabels" . | nindent 8 }}
        tier: backend
    spec:
      serviceAccountName: {{ include "msvc-chart.serviceAccountName" . }}
      containers:
        - name: {{ include "msvc-chart.backend.container.name" . }}
          image: "{{ .Values.backend.image.name }}:{{ .Values.backend.image.tag }}"
          imagePullPolicy: {{ .Values.backend.image.pullPolicy }}
          ports:
            - containerPort: 8080
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /actuator/health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 3
          resources:
            {{- toYaml .Values.backend.container.resources | nindent 12 }}

---

apiVersion: v1
kind: Service
metadata:
  name: {{ include "msvc-chart.backend.service.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: backend
spec:
  ports:
    - port: {{ .Values.backend.service.port }}
      protocol: TCP
      targetPort: 8080
      name: http
  selector:
    {{- include "msvc-chart.selectorLabels" . | nindent 4 }}
    tier: backend

---

{{- if .Values.backend.hpa.enabled -}}
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "msvc-chart.backend.hpa.name" . }}
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
    tier: backend
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "msvc-chart.backend.deployment.name" . }}
  minReplicas: {{ .Values.backend.hpa.minReplicas }}
  maxReplicas: {{ .Values.backend.hpa.maxReplicas }}
  targetCPUUtilizationPercentage: {{ .Values.backend.hpa.targetCPUUtilizationPercentage }}
{{- end -}}

NOTES.txt


NOTES.txt , , (helm status <name>).


URL- . , , . , .


1. Get the application URL by running these commands:
{{- if contains "NodePort" .Values.gateway.service.type }}
  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "msvc-chart.gateway.service.name" . }})
  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.gateway.service.type }}
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "msvc-chart.gateway.service.name" . }}'
  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "msvc-chart.gateway.service.name" . }} -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
  echo http://$SERVICE_IP:{{ .Values.gateway.service.port }}
{{- else if contains "ClusterIP" .Values.gateway.service.type }}
  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name= {{ .Chart.Name }},app.kubernetes.io/instance={{ .Release.Name }},tier=gateway" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:8080
{{- end }}
2. Get service account token:
  export TOKEN=$(kubectl get serviceaccount {{ include "msvc-chart.serviceAccountName" . }} -n {{ .Release.Namespace }} -o go-template --template='{{`{{range .secrets}}{{.name}}{{"\n"}}{{end}}`}}')
  export TOKEN_DECODED=$(kubectl get secrets "$TOKEN" -n {{ .Release.Namespace }} -o go-template --template '{{`{{index .data "token"}}`}}' | base64 -d)
  echo $TOKEN_DECODED

Chart.yaml


. — version appVersion. version — , . appVersion — . type 'application' 'library' — .


apiVersion: v2
name: msvc-chart
version: 1.0.0
description: Microservices project
type: application
sources:
  - https://github.com/Anshelen/microservices-backend
  - https://github.com/Anshelen/microservices-gateway
  - https://github.com/Anshelen/microservices-deploy
maintainers:
  - name: Anton Shelenkov
    email: anshelen@yandex.ru
    url: https://shelenkov.herokuapp.com
appVersion: 1.0.0


Helm . helm test <name> (№ ). Kubernetes-, , http-. , .


, . Kubernetes- Job — , . , Job 3 ( backoffLimit). helm.sh/hook helm.sh/hook-delete-policy. Helm, , .


interaction-test.yaml:


apiVersion: batch/v1
kind: Job
metadata:
  name: "{{ .Release.Name }}-interaction-test"
  labels:
    {{- include "msvc-chart.labels" . | nindent 4 }}
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": hook-succeeded,hook-failed
spec:
  template:
    spec:
      containers:
      - name: test
        image: busybox
        command: ['wget']
        args:  ['{{ include "msvc-chart.gateway.service.name" . }}:{{ .Values.gateway.service.port }}/']
      restartPolicy: Never
  backoffLimit: 3

Helm


Helm : , , Helm, kubectl. Helm , . kubectl, Helm , .



, Kubernetes-:


helm install --dry-run --debug <name> <chart-folder>

, , , <name>:


helm install <name> <chart-folder>

values.yaml, --set <key>=<value>. . , yaml- -f <overriden-values.yaml>. -n <namespace> , .



:


helm upgrade <name> <chart-folder>

-f <file> --set <key>=<value>. Helm , . , --reuse-values. , Helm , --set --reuse-values , - . :


helm install <name> <chart-folder>
helm upgrade <name> <chart-folder> --set key=value --reuse-values

, , - , , :


helm install <name> <chart-folder> --set _=null


helm list — Helm' .
helm status <name> — .
helm history <name> — . . .
helm get values <name> — . --revision n, n- .
helm show values <chart-folder> — . values.yaml.
helm get manifest <name> — Helm' .



helm rollback <name> <revision-num> — .
helm uninstall <name> — .



Helm- . HTTP- S3-. GitHub Pages.


index.yaml , . — .


GitHub docs :


helm package msvc-chart/ -d docs/
helm repo index docs/ --url https://<github-username>.imtqy.com/<github-repo-name>/

docs. index.yaml . url URL- . , . , GitHub Pages, 'Source' 'master branch /docs folder'.



, GKE, Helm.


:


kubectl create namespace msvc-ns
kubectl config set-context --current --namespace=msvc-ns

. Helm msvc-repo (URL , Helm-):


helm repo add msvc-repo https://anshelen.imtqy.com/microservices-deploy/
helm repo update

:


helm install msvc-project msvc-repo/msvc-chart \
--set backend.container.resources.requests.cpu=50m \
--set backend.hpa.enabled=true \
--set gateway.container.resources.requests.cpu=50m \
--set gateway.hpa.enabled=true \
--set secrets.secret=secret

URL- . , , .


:


helm test msvc-project

:


helm upgrade msvc-project msvc-repo/msvc-chart \
--set secrets.secret=new-secret \
--reuse-values

- , .



Helm- , GKE. Helm , Git — , Docker, Helm.


Afin d'automatiser le processus de livraison de code au cluster, dans le prochain article, nous allons configurer le serveur d'intégration continue Jenkins et créer un pipeline pour notre projet.

More articles:


All Articles