Sauvegarde fine des systèmes de fichiers Linux. Comment créer des copies de travail d'un SGBD MySQL de trois téraoctets en 20 secondes

, . (thin provisioning) Linux , CI/CD- . , production MySQL, «» , .

: ?

, . ? . .

( 2018 15 ), , , . :

• MySQL 1000 2,5 , .
• . « », JOIN’ .
• .
• .

. Postman PHP-, . , : , , . .

, : . .

. , .

« »?
( — sparse volumes), , . just-enough (, ) just-in-time ( ). , , . , , LVM2, ZFS, BTRFS. . , (data- MySQL).

, Thin LVM

« Thin LVM, MySQL ».

LVM , . — MySQL, MySQL. , . :

• 2 x Intel Silver 4114 (10x2,2 HT)
• 8 x 32 DDR4
• 8 x 1920 Intel SSD RAID- Adaptec RAID-10

RAID- RAID MD . , :

• RAID-, , .
• MySQL .

RAID-10: Volume Group (VG) ( 6,7 ) (Logical Volume, LV) 50 . c MySQL. , pool, /var/lib/mysql 3,5 ( ):

lvcreate -l 100%FREE -T vga/thin
lvcreate -V 3.5T -T vga/thin -n mysql

ext4, , . API, , MySQL . , bash, API HTTP → bash open source- goexpose, Go.

- bash- open source, :

snapmain:

1. .
2. snapmain.
3. snapmain.
4. MySQL .

snapmain:

1. ().
2. . , 5 .
3. , LV- .
   3.1 , kill -9 MySQL LV-.
4. snapmain .
5. .
6. () MySQL.
7. .
8. .

:

1. ().
2. MySQL kill -9.
3. .
4. LV- .

:

lvcreate -n stage_3307 -s vga/snapmain
lvchange -ay -K vga/stage_3307
mount -o noatime,nodiratime,data=writeback /dev/mapper/vga-stage_3307 /mnt/stage_3307

, . SSD-. - Thin LVM: 4 . :

1. /var/lib/mysql.
2. , .
3. , .
4. , .
5. - 100% , .
6. .

, ( ):

RAID-:

• .
• writeback ( , ).

:

• /var/lib/mysql noatime,nodiratime,data=writeback
• ext4 tune2fs.

MySQL:

• innodb_flush_method = O_DSYNC ( , ).
• , .
• innodb_buffer_pool_size = 4G ( InnoDB, MySQL , ).

, MySQL. , . , innodb_parallel_doublewrite_path /dev/shm, 5 .

MySQL , ? . , . , .

. , , :

Thin LVM: iostat, , , .

, : YOLO-. - ext4 , . , .

Thin LVM:

• Thin pool data %
• Thin pool metadata %

( ), .

. fstrim -v /var/lib/mysql.

:

• , LVM, .
• . , , .
• , .

, ZFS

- ZFS, ZFS Solaris. FreeBSD . Linux, . - B-tree (, InnoDB MySQL) ZFS . . ext4 xfs, . , ZFS , Linux-, , ( , - ZFS ), .

( RAID-). SSD- 1920 . ZFS, 50 MD RAID-10 . 1950 ZFS- RAID-10:

zpool create zpool mirror /dev/sda2 /dev/sdb2 mirror /dev/sdc2 /dev/sdd2 mirror /dev/sde2 /dev/sdf2 mirror /dev/sdg2 /dev/sdh2

MySQL:

zfs create zpool/mysql
zfs set compression=gzip zpool/mysql
zfs set recordsize=128k zpool/mysql
zfs set atime=off zpool/mysql
zfs create zpool/mysql/data
zfs set recordsize=16k zpool/mysql/data
zfs set primarycache=metadata zpool/mysql/data
zfs set mountpoint=/var/lib/mysql zpool/mysql/data

, gzip. 3 1,6 , , , , — , ZFS! gzip 4 , .

. c LVM- MySQL. ZFS, . :

zfs set snapdir=visible zpool/mysql/data
zfs create zpool/stage_3307
zfs clone zpool/mysql/data@snapmain zpool/stage_3307/data
zfs set mountpoint=/mnt/stage_3307 zpool/stage_3307/data

: ZFS- l2arc zil. , , , , . — . . zpool status:

logs
      /dev/shm/zil_slog.img  ONLINE       0     0     0
cache
      /dev/shm/l2arc.img     ONLINE       0     0     0

: ( ) 50-60 %.

, ( Thin LVM):

: 40 , MySQL 20 . , .

:

• .
• : , ZFS .
• ZFS ( 1 ) . , , - .

?

, . , , , . BTRFS . , , . , , — , , . , .

, . , , :

• Thin LVM — ZFS.
• ZFS — .

— . , , , , , . - , , , . , - ( — -) . , , , , . , . , .