从Kubernetes的生活中看：我们如何将DBMS（不仅是从评论环境）删除为静态

1. 可以从开发人员定义的Git存储库中的独立分支（所谓的功能分支）中创建评论环境。
2. 它们可以具有单独的DBMS实例，队列处理器，缓存服务等。-总的来说，一切都是为了充分再现生产环境。
3. 它们允许并行开发，从而大大加速了应用程序中新功能的发布。同时，每天可能需要数十个这样的环境，这就是为什么它们的创建速度至关重要的原因。

问题和解决方法

1. 为了简化质量检查工程师和其他一些员工的工作，请在单独的静态环境中删除审阅中使用的所有数据库（和相应的虚拟主机）。由于项目中普遍存在的原因，这种与他们互动的方式是最佳的。
2. 减少审查环境的创建时间。从头开始暗示了它们创建的整个过程。包括数据库克隆，迁移等。

• 创建审阅环境时，应执行以下操作一次：在两个DBMS（MongoDB和PostgreSQL）中创建数据库，从备份/模板还原数据库，以及在RabbitMQ中创建vhost。这将需要一种方便的方式来加载当前转储。（如果您之前有审查环境，那么很可能已经有一个现成的解决方案。）
• 查看环境完成后，必须删除RabbitMQ中的数据库和虚拟主机。

• 对于初始化任务，我们将pre-install在创建发行版中的所有资源之前使用挂钩启动它；
• 对于删除任务，使用一个hook post-delete。

实际实施

{{- if .Values.global.review }}
---
apiVersion: batch/v1
kind: Job
metadata:
  name: db-create-postgres-database
  annotations:
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-weight": "5"
spec:
  template:
    metadata:
      name: init-db-postgres
    spec:
      volumes:
      - name: postgres-scripts
        configMap:
          defaultMode: 0755
          name: postgresql-configmap
      containers:
      - name: init-postgres-database
        image: private-registry/postgres 
        command: ["/docker-entrypoint-initdb.d/01-review-load-dump.sh"]
        volumeMounts:
        - name: postgres-scripts
          mountPath: /docker-entrypoint-initdb.d/01-review-load-dump.sh
          subPath: review-load-dump.sh
        env:
{{- include "postgres_env" . | indent 8 }}
      restartPolicy: Never
{{- end }}

1. Job review-. review CI/CD Helm- (. if .Values.global.review ).
2. Job — , ConfigMap. , , . , hook-weight.
3. cURL , PostgreSQL, .
4. PostgreSQL : , shell- .

PostgreSQL

1. 从备份中“标准”恢复；
2. 使用模板进行恢复。

---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: update-postgres-template
spec:
  schedule: "50 4 * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  startingDeadlineSeconds: 600
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          imagePullSecrets:
          - name: registrysecret
          volumes:
          - name: postgres-scripts
            configMap:
              defaultMode: 0755
              name: postgresql-configmap-update-cron
          containers:
          - name: cron
            command: ["/docker-entrypoint-initdb.d/update-postgres-template.sh"]
          image: private-registry/postgres 
            volumeMounts:
            - name: postgres-scripts
              mountPath: /docker-entrypoint-initdb.d/update-postgres-template.sh
              subPath: update-postgres-template.sh
            env:
{{- include "postgres_env" . | indent 8 }}

#!/bin/bash -x

CREDENTIALS="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}/postgres"

psql -d "${CREDENTIALS}" -w -c "REVOKE CONNECT ON DATABASE ${POSTGRES_DB_TEMPLATE} FROM public"
psql -d "${CREDENTIALS}" -w -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = '${POSTGRES_DB_TEMPLATE}'"

curl --fail -vsL ${HOST_FORDEV}/latest_${POSTGRES_DB_STAGE}.psql -o /tmp/${POSTGRES_DB}.psql

psql -d "${CREDENTIALS}" -w -c "ALTER DATABASE ${POSTGRES_DB_TEMPLATE} WITH is_template false allow_connections true;"
psql -d "${CREDENTIALS}" -w -c "DROP DATABASE ${POSTGRES_DB_TEMPLATE};" || true
psql -d "${CREDENTIALS}" -w -c "CREATE DATABASE ${POSTGRES_DB_TEMPLATE};" || true
pg_restore -U ${POSTGRES_USER} -h ${POSTGRES_HOST} -w -j 4 -d ${POSTGRES_DB_TEMPLATE} /tmp/${POSTGRES_DB}.psql

psql -d "${CREDENTIALS}" -w -c "ALTER DATABASE ${POSTGRES_DB_TEMPLATE} WITH is_template true allow_connections false;"

rm -v /tmp/${POSTGRES_DB}.psql

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgresql-configmap
  annotations:
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  review-load-dump.sh: |
    #!/bin/bash -x
    
 
 
    CREDENTIALS="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}/postgres"

    if [ "$( psql -d "${CREDENTIALS}" -tAc "SELECT CASE WHEN EXISTS (SELECT * FROM pg_stat_activity WHERE datname = '${POSTGRES_DB}' LIMIT 1) THEN 1 ELSE 0 END;" )" = '1' ]
      then
          echo "Open connections has been found in ${POSTGRES_DB} database, will drop them"
          psql -d "${CREDENTIALS}" -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = '${POSTGRES_DB}' -- AND pid <> pg_backend_pid();"
      else
          echo "No open connections has been found ${POSTGRES_DB} database, skipping this stage"
    fi

    psql -d "${CREDENTIALS}" -c "DROP DATABASE ${POSTGRES_DB}"

    if [ "$( psql -d "${CREDENTIALS}" -tAc "SELECT 1 FROM pg_database WHERE datname='${POSTGRES_DB}'" )" = '1' ]
      then
          echo "Database ${POSTGRES_DB} still exists, delete review job failed"
          exit 1
      else
          echo "Database ${POSTGRES_DB} does not exist, skipping"
    fi


    psql ${CREDENTIALS} -d postgres -c 'CREATE DATABASE ${POSTGRES_DB} TEMPLATE "loot-stage-copy"'

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgresql-configmap-on-delete
  annotations:
    "helm.sh/hook": "post-delete, pre-delete"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation
data:
  review-delete-db.sh: |
    #!/bin/bash -e

    CREDENTIALS="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}/postgres"

    psql -d "${CREDENTIALS}" -w postgres -c "DROP DATABASE ${POSTGRES_DB}"

1. 每天晚上，都会进行数据库备份，以便可以从群集的本地网络下载它。
2. 在创建审阅环境时，将从转储中加载和还原数据库。
3. 部署转储后，将执行所有其他操作。

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgresql-configmap
  annotations:
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  review-load-dump.sh: |
    #!/bin/bash -x

    CREDENTIALS="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}/postgres"
    psql -d "${CREDENTIALS}" -w -c "DROP DATABASE ${POSTGRES_DB}" || true
    psql -d "${CREDENTIALS}" -w -c "CREATE DATABASE ${POSTGRES_DB}"

    curl --fail -vsL ${HOST_FORDEV}/latest_${POSTGRES_DB_STAGE}.psql -o /tmp/${POSTGRES_DB}.psql

    psql psql -d "${CREDENTIALS}" -w -c "CREATE EXTENSION ip4r;"
    pg_restore -U ${POSTGRES_USER} -h ${POSTGRES_HOST} -w -j 4 -d ${POSTGRES_DB} /tmp/${POSTGRES_DB}.psql
    rm -v /tmp/${POSTGRES_DB}.psql

Mongodb

1. 他已经过时了。
2. 根据我们的测试结果，我们发现数据库恢复时间与通常的恢复时间没有太大的不同mongo_restore。但是，我注意到测试是作为一个项目的一部分进行的-在您的情况下，结果可能会完全不同。

1. 例如，如果您的DBMS位于网络文件系统上（对于没有生产环境的情况），则恢复可能会花费很长时间。然后，您可以简单地将DBMS从舞台转移到单独的节点并使用本地存储。由于这不是正式产品，因此创建评论的速度对我们而言更为关键。
2. 您可以将每个作业恢复带到一个单独的容器中，从而允许您预先执行迁移和依赖于DBMS操作的其他过程。因此，我们通过提前完成它们来节省时间。
3. 有时，您可以通过删除旧的/不相关的数据来减小转储的大小-在一定程度上足以仅保留数据库结构。当然，这不适用于需要完全转储的情况（例如，对于QA测试任务）。

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb-scripts-on-delete
  annotations:
    "helm.sh/hook": "post-delete, pre-delete"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation
data:
  review-delete-db.sh: |
    #!/bin/bash -x

    mongo ${MONGODB_NAME} --eval "db.dropDatabase()" --host ${MONGODB_REPLICASET}/${MONGODB_HOST}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb-scripts
  annotations:
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  review-load-dump.sh: |
    #!/bin/bash -x

    curl --fail -vsL ${HOST_FORDEV}/latest_${MONGODB_NAME_STAGE}.gz -o /tmp/${MONGODB_NAME}.gz

    mongo ${MONGODB_NAME} --eval "db.dropDatabase()" --host ${MONGODB_REPLICASET}/${MONGODB_HOST}
    mongorestore --gzip --nsFrom "${MONGODB_NAME_STAGE}.*" --nsTo "${MONGODB_NAME}.*" --archive=/tmp/${MONGODB_NAME}.gz --host ${MONGODB_REPLICASET}/${MONGODB_HOST}

the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead

Rabbitmq

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: rabbitmq-configmap
  annotations:
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  rabbitmq-setup-vhost.sh: |
    #!/bin/bash -x

    /usr/local/bin/rabbitmqadmin -H ${RABBITMQ_HOST} -u ${RABBITMQ_USER} -p ${RABBITMQ_PASSWORD} declare vhost name=${RABBITMQ_VHOST}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: rabbitmq-configmap-on-delete
  annotations:
    "helm.sh/hook": "post-delete, pre-delete"
    "helm.sh/hook-weight": "1"
    "helm.sh/hook-delete-policy": before-hook-creation
data:
  rabbitmq-delete-vhost.sh: |
    #!/bin/bash -x

    /usr/local/bin/rabbitmqadmin -H ${RABBITMQ_HOST} -u ${RABBITMQ_USER} -p ${RABBITMQ_PASSWORD} delete vhost name=${RABBITMQ_VHOST}

缺点

1. 它以阶段环境的形式证明了单点故障。
2. 如果舞台环境中的应用程序仅在一个副本中运行，那么我们将更加依赖于该应用程序在其上运行的主机。因此，随着查看环境数量的增加，节点上的负载会成比例地增加，而无法平衡该负载。

结论

聚苯乙烯

• “ Kubernetes提示和技巧：加快大型数据库的引导速度 ”；
• “ 畅通无阻的RabbitMQ迁移到Kubernetes ”；
• “将MongoDB顺利迁移到Kubernetes ”；
• “正在运行的团队正在向Kubernetes提供新的应用程序发布。”