Cara memulai pengujian Ansible, refactor proyek selama satu tahun dan tidak terbang dari gulungan

Ini adalah transkrip kinerja pada DevOps-40 2020-03-18 :

Mulai dari komit kedua, kode apa pun menjadi warisan, karena ide-ide awal mulai menyimpang dari kenyataan pahit. Ini tidak baik atau buruk, itu adalah pemberian yang sulit untuk diperdebatkan dan perlu untuk bergaul. Bagian dari proses ini adalah refactoring. Infrastruktur Refactoring sebagai Kode. Biarkan cerita dimulai bagaimana refactor mungkin dalam setahun dan tidak terbang dari gulungan.

Asal Warisan

Hari 1: Nol Pasien

Sekali waktu ada proyek bersyarat. Itu memiliki tim pengembangan Dev dan insinyur Ops. Mereka memecahkan masalah yang sama: bagaimana cara menyebarkan server dan menjalankan aplikasi. Masalahnya adalah bahwa masing-masing tim menyelesaikan masalah ini dengan caranya sendiri. Proyek memutuskan untuk menggunakan Ansible untuk menyinkronkan pengetahuan antara tim Dev dan Ops.

Hari No. 89: Kelahiran Legacy

, , legacy. ?

• , — .
• .
• Ansible / Python / Bash / Terraform! !
• Full Stack Overflow Developer stackoverflow, , .

, , , , , , , , .

- hosts: localhost
  tasks:
    - shell: echo -n Z >> a.txt && cat a.txt
      register: output
      delay: 1
      retries: 5
      until: not output.stdout.find("ZZZ")

№ 109:

IaC / / , . , .

IaC

№ 139: ?

:

1. ?
2. ?
3. ?

, . .. ( , 200 000 ), .

№ 149:

e . . , . - , confluence, " ?" " ?" . : / . , , - .

, . , , : , , .

Ansible

Ansible , , .

№ -997: SDS provision

Ansible SDS (Software Defined Storage).
, , 60-90 , . e2e , .. , . . .

№ -701: Ansible test kitchen

Ansible , test kitchen / kitchen-ci inspec. Ruby ( : YML ansible?) 40 10 . .

, - . 13 2 , 70 , 2 . XP (extreme programming) .. 70 .

№ -601: Ansible molecule

testkitchen, docker . , 20-25 7 .

17 45 28 2 jenkins slave.

№ 167: Ansible

, . , . , .

, , , jira, google docs . , , . , - , , .

:

• Eat.
• Sleep.
• Code.
• IaC test.
• Repeat

.

, .

№ 181: Green Build Master

Green Build Master. , jenkins. , :

• .
• - , .

№ 193: unit

— , . , .

№ 211: unit integration

unit , . .. , , .

jenkins , / , .

Jenkins + Docker + Ansible = Tests

1. Checkout repo and generate build stages.
2. Run lint playbook stages in parallel.
3. Run lint role stages in parallel.
4. Run syntax check role stages in parallel.
5. Run test role stages in parallel.
   
   1. Lint role.
   2. Check dependency on other roles.
   3. Check syntax.
   4. Create docker instance
   5. Run molecule/default/playbook.yml.
   6. Check idempotency.
6. Run integration tests
7. Finish

№ 271: Bus Factor

- . e. code review . , , , .. .

. , , . jenkins + bitbucket + jira.

, -, , :

- get_url:
    url: "{{ actk_certs }}/{{ item.1 }}"
    dest: "{{ actk_src_tmp }}/"
    username: "{{ actk_mvn_user }}"
    password: "{{ actk_mvn_pass }}"
  with_subelements:
    - "{{ actk_cert_list }}"
    - "{{ actk_certs }}"
  delegate_to: localhost

- copy:
    src: "{{ actk_src_tmp }}/{{ item.1 }}"
    dest: "{{ actk_dst_tmp }}"
  with_subelements:
    - "{{ actk_cert_list }}"
    - "{{ actk_certs }}"

, .

get_url:
    url: "{{ actk_certs }}/{{ actk_item }}"
    dest: "{{ actk_src_tmp }}/{{ actk_item }}"
    username: "{{ actk_mvn_user }}"
    password: "{{ actk_mvn_pass }}"
  loop_control:
    loop_var: actk_item
  with_items: "{{ actk_cert_list }}"
  delegate_to: localhost

- copy:
    src: "{{ actk_src_tmp }}/{{ actk_item }}"
    dest: "{{ actk_dst_tmp }}"
  loop_control:
    loop_var: actk_item
  with_items: "{{ actk_cert_list }}"

№ 311:

, . " , ". docker, . testinfra ansible verifier - .

:

1. docker.
2. , .
3. - .
4. .
5. .

Pipeline jenkins

1. Generate build stages.
2. Lint all in parallel.
3. Run test role stages in parallel.
4. Finish.

Lessons learned

Avoid global variables

Ansible , workaround private_role_vars, .

. role_a role_b

# cat role_a/defaults/main.yml
---
msg: a

# cat role_a/tasks/main.yml
---
- debug:
    msg: role_a={{ msg }}

# cat role_b/defaults/main.yml
---
msg: b

# cat role_b/tasks/main.yml
---
- set_fact:
    msg: b
- debug:
    msg: role_b={{ msg }}

- hosts: localhost
  vars:
    msg: hello
  roles:
    - role: role_a
    - role: role_b
  tasks:
    - debug:
        msg: play={{msg}}

, , . Ansible , - , .

BAD: .

# cat roles/some_role/tasks/main.yml
---
debug:
  var: java_home

GOOD: defaults .

# cat roles/some_role/defaults/main.yml
---
r__java_home:
 "{{ java_home | default('/path') }}"

# cat roles/some_role/tasks/main.yml
---
debug:
  var: r__java_home

Prefix role variables

BAD: .

# cat roles/some_role/defaults/main.yml
---
db_port: 5432

GOOD: , inventory .

# cat roles/some_role/defaults/main.yml
---
some_role__db_port: 5432

Use loop control variable

BAD: item, / -

---
- hosts: localhost
  tasks:
    - debug:
        msg: "{{ item }}"
      loop:
        - item1
        - item2

GOOD: loop_var.

---
- hosts: localhost
  tasks:
    - debug:
        msg: "{{ item_name }}"
      loop:
        - item1
        - item2
      loop_control:
        loop_var: item_name

Check input variables

, , ,

GOOD: .

- name: "Verify that required string variables are defined"
  assert:
    that: ahs_var is defined and ahs_var | length > 0 and ahs_var != None
    fail_msg: "{{ ahs_var }} needs to be set for the role to work "
    success_msg: "Required variables {{ ahs_var }} is defined"
  loop_control:
    loop_var: ahs_var
  with_items:
    - ahs_item1
    - ahs_item2
    - ahs_item3

Avoid hashes dictionaries, use flat structure

hash/dictionary , , hash/dictionary, .

BAD: hash/dictionary.

---
user:
  name: admin
  group: admin

GOOD: .

---
user_name: admin
user_group: "{{ user_name }}"

Create idempotent playbooks & roles

, .. configuration drift -. molecule, .

Avoid using command shell modules

shell , , Ansible.

Test your roles via molecule

Molecule , .

Molecule Multiple instances

molecule.yml platforms .

---
    driver:
      name: docker
    platforms:
      - name: postgresql-instance
        hostname: postgresql-instance
        image: registry.example.com/postgres10:latest
        pre_build_image: true
        override_command: false
        network_mode: host
      - name: app-instance
        hostname: app-instance
        pre_build_image: true
        image: registry.example.com/docker_centos_ansible_tests
        network_mode: host

, converge.yml :

---
- name: Converge all
  hosts: all
  vars:
    ansible_user: root
  roles:
    - role: some_role

- name: Converge db
  hosts: db-instance
  roles:
    - role: some_db_role

- name: Converge app
  hosts: app-instance
  roles:
    - role: some_app_role

Ansible verifier

molecule ansible , , 3 . testinfra/inspec, , :

---
- name: Verify
  hosts: all
  tasks:
    - name: copy config
      copy:
        src: expected_standalone.conf
        dest: /root/wildfly/bin/standalone.conf
        mode: "0644"
        owner: root
        group: root
      register: config_copy_result

    - name: Certify that standalone.conf changed
      assert:
        that: not config_copy_result.changed

, smoke test:

---
  - name: Verify
    hosts: solr
    tasks:
      - command: /blah/solr/bin/solr start -s /solr_home -p 8983 -force
      - uri:
          url: http://127.0.0.1:8983/solr
          method: GET
          status_code: 200
        register: uri_result
        until: uri_result is not failed
        retries: 12
        delay: 10
      - name: Post documents to solr
        command: /blah/solr/bin/post -c master /exampledocs/books.csv

Put complex logic into modules & plugins

Ansible , , , shell , . , , .

Summarize Tips & Tricks

1. Avoid global variables.
2. Prefix role variables.
3. Use loop control variable.
4. Check input variables.
5. Avoid hashes dictionaries, use flat structure.
6. Create idempotent playbooks & roles.
7. Avoid using command shell modules.
8. Test your roles via molecule.
9. Put complex logic into modules & plugins.

, IaC. , .

Links

• Slides How to test Ansible and don't go nuts
• Video How to test Ansible and don't go nuts
• , 200 000
• Ansible: 120 VM c Coreos Centos 18
• YML ansible?
• A list of awesome IaC testing articles, speeches & links
• English version

UPD1 2020.05.01 20:30 — callback_whitelist = profile_tasks . ansible. mitogen
UPD2 2020.05.03 16:34 — English version