Get best of the week

Linux文件系统的精简备份。如何在20秒内创建三TB的MySQL DBMS的工作副本


, . (thin provisioning) Linux , CI/CD- . , production MySQL, «» , .


: ?


, . ? . .


( 2018 15 ), , , . :


  • MySQL 1000 2,5 , .
  • . « », JOIN’ .
  • .
  • .

. Postman PHP-, . , : , , . .


, : . .


. , .


« »?
( — sparse volumes), , . just-enough (, ) just-in-time ( ). , , . , , LVM2, ZFS, BTRFS. . , (data- MySQL).


, Thin LVM


« Thin LVM, MySQL ».


LVM , . — MySQL, MySQL. , . :


  • 2 x Intel Silver 4114 (10x2,2 HT)
  • 8 x 32 DDR4
  • 8 x 1920 Intel SSD RAID- Adaptec RAID-10

RAID- RAID MD . , :


  • RAID-, , .
  • MySQL .

RAID-10: Volume Group (VG) ( 6,7 ) (Logical Volume, LV) 50 . c MySQL. , pool, /var/lib/mysql 3,5 ( ):


lvcreate -l 100%FREE -T vga/thin
lvcreate -V 3.5T -T vga/thin -n mysql

ext4, , . API, , MySQL . , bash, API HTTP → bash open source- goexpose, Go.


- bash- open source, :


snapmain:


  1. .
  2. snapmain.
  3. snapmain.
  4. MySQL .

snapmain:


  1. ().
  2. . , 5 .
  3. , LV- .
    3.1 , kill -9 MySQL LV-.
  4. snapmain .
  5. .
  6. () MySQL.
  7. .
  8. .

:


  1. ().
  2. MySQL kill -9.
  3. .
  4. LV- .

:


lvcreate -n stage_3307 -s vga/snapmain
lvchange -ay -K vga/stage_3307
mount -o noatime,nodiratime,data=writeback /dev/mapper/vga-stage_3307 /mnt/stage_3307

, . SSD-. - Thin LVM: 4 . :


  1. /var/lib/mysql.
  2. , .
  3. , .
  4. , .
  5. - 100% , .
  6. .

, ( ):


RAID-:


  • .
  • writeback ( , ).

:


  • /var/lib/mysql noatime,nodiratime,data=writeback
  • ext4 tune2fs.

MySQL:


  • innodb_flush_method = O_DSYNC ( , ).
  • , .
  • innodb_buffer_pool_size = 4G ( InnoDB, MySQL , ).

, MySQL. , . , innodb_parallel_doublewrite_path /dev/shm, 5 .


MySQL , ? . , . , .


. , , :


Thin LVM: iostat, , , .


, : YOLO-. - ext4 , . , .


Thin LVM:


  • Thin pool data %
  • Thin pool metadata %

( ), .


. fstrim -v /var/lib/mysql.


:


  • , LVM, .
  • . , , .
  • , .

, ZFS


- ZFS, ZFS Solaris. FreeBSD . Linux, . - B-tree (, InnoDB MySQL) ZFS . . ext4 xfs, . , ZFS , Linux-, , ( , - ZFS ), .


( RAID-). SSD- 1920 . ZFS, 50 MD RAID-10 . 1950 ZFS- RAID-10:


zpool create zpool mirror /dev/sda2 /dev/sdb2 mirror /dev/sdc2 /dev/sdd2 mirror /dev/sde2 /dev/sdf2 mirror /dev/sdg2 /dev/sdh2

MySQL:


zfs create zpool/mysql
zfs set compression=gzip zpool/mysql
zfs set recordsize=128k zpool/mysql
zfs set atime=off zpool/mysql
zfs create zpool/mysql/data
zfs set recordsize=16k zpool/mysql/data
zfs set primarycache=metadata zpool/mysql/data
zfs set mountpoint=/var/lib/mysql zpool/mysql/data

, gzip. 3 1,6 , , , , — , ZFS! gzip 4 , .


. c LVM- MySQL. ZFS, . :


zfs set snapdir=visible zpool/mysql/data
zfs create zpool/stage_3307
zfs clone zpool/mysql/data@snapmain zpool/stage_3307/data
zfs set mountpoint=/mnt/stage_3307 zpool/stage_3307/data

: ZFS- l2arc zil. , , , , . — . . zpool status:


logs
      /dev/shm/zil_slog.img  ONLINE       0     0     0
cache
      /dev/shm/l2arc.img     ONLINE       0     0     0

: ( ) 50-60 %.


, ( Thin LVM):


: 40 , MySQL 20 . , .


:


  • .
  • : , ZFS .
  • ZFS ( 1 ) . , , - .

?


, . , , , . BTRFS . , , . , , — , , . , .



, . , , :


  • Thin LVM — ZFS.
  • ZFS — .

— . , , , , , . - , , , . , - ( — -) . , , , , . , . , .

More articles:


All Articles