I suggest you familiarize yourself with the transcript of the report of the beginning of 2020 by Georgy Rylov "WAL-G: new opportunities and expanding the community"
Open-source maintainers have many problems as they grow. How to write more and more required features, fix more and more issues and manage to watch more and more pull requests? Using WAL-G (backup-tool for PostgreSQL) as an example, I’ll talk about how we solved these problems by launching a course on Open-source development at the university, what we have achieved and where we will move on.
Hello again to everyone! I am a Yandex developer from Yekaterinburg. And today I will talk about WAL-G.
The title of the report did not say that it was something about backups. Does anyone not know what a WAL-G is? Or does everyone know? Raise your hand, who does not know. Ofiget, you came to the report and do not know what it is about.
, . , . , , , , .
, . . WAL-G .
clck.ru/F8ioz — https://www.highload.ru/moscow/2018/abstracts/3964
clck.ru/Ln8Qw — https://www.highload.ru/moscow/2019/abstracts/5981
, , , , . , .
WAL-G , Citus Data. . .
WAL-G :
WAL-G .
2020- . , :
- 1 000 GitHub.
- 150 .
- 15 PR.
- .
- issues . , , - .
, , , - Managed Databases .
- 2018- . , - , , . , . - - . , . , .
. . - . , , , , , . – . , . , , . , . , , , , , . . . – , .
PR
- . CI.
- 2 . .
- , , , . end-to-end , .
Open Source
- , , , .
:
?
, , , , , .
. :
- Open Source.
- CV.
- .
- GSoC.
- +1 , .
, . , WAL-G . Odyssey, PostgreSQL ClickHouse.
, .
?
, , , . ? , . request, .
, .
tablespaces. Tablespaces WAL-G , , WAL-G, WAL-G WAL-E, tablespaces.
, . , Postgres , . , Postgres.
Tablespaces , Postgres, . , tablespac’ .
Postgres? pg_tblspc. , Postgres .
, - . . . - tablespace , . , . , - . – . tablespace. , .
, WAL-E, , WAL-G, . .
, , catchup. catchup , , , Oracle, Postgres.
, . - . . , write-ahead log. LSN . - . . -.
? , , . , . - .
– , , . , 10 TB – - . , . catchup.
Catchup -, . LSN catchup , - LSN LSN, . , .
. . . Yandex Postgres, MySQL, MongoDB, Redis, ClickHouse, - , point-in-time recovery MySQL, .
- , WAL-G. , .
. , - . , – postgres’, WAL-G. - . . . Postgres MySQL, MySQL, Postgres.
, , , extensions PostgreSQL. , , MySQL - .
. Postgres, Postgres . MySQL , Postgres MySQL. - . , , .
Postgres, MySQL, MongoDB, Redis
, , – . , , API, WAL-G.
, – , . . . , - , , - .
. . .
MySQL, WAL-G .
MongoDB production, . , . - . , production.
, backup tools . . , point-in-time recovery . - , . backup tools, - , WAL-G, . point-in-time recovery.
? WAL-G Libsodium.
. . - .
?
100 . , – . . 100 . - , 30 .
, , . - - . .
14 issues, 10 . , , - .
. 12 . 6 «5». , , .
?
-requests, . :
- HA-. WAL-G . , , , .
- WAL’ .
- , WAL-G , .
? , , 4- , , , . , . WAL-G WAL-G , .
qr- . . , - . - , - , . .
! ! WAL-G, Postgres. WAL-G MySQL -. CentOS yum install MySQL, MariDB. 10.3 - , MariDB-. ?
MariDB. FoundationDB, , , , . , .
! ! . WAL-G , ?
?
.
, .
() , ! . . — . , Storages. , Storoges . Storage , , , . pull request. , . Go, 50 . WAL-G .
! . – , . , , , ?
. , . . unit-. , . . – , cron, . . . .
. ?
, . . . , , , , - , , . . . , .
! , archive_command WAL-G . - , , . ?
? , , ? ?
() , WAL-G shell-. - , , WAL-G. , , , . – . . , . !
(: shell- issue)
! ! catchup, . , . WAL-G .
Catchup appeared literally on the 20th of January 2020. The documentation might be worth a while. We ourselves write it and write not so super-perfectly. And perhaps it’s worth starting from students to demand that they write it.
Is she already in release?
Pull request is already frozen, i.e. I checked it. I tried this on a test cluster. So far, we have not had a situation where we could test this with a military example.
When to expect?
I dont know. Wait a month, we’ll check for sure.