Saya mengusulkan untuk berkenalan dengan transkrip laporan awal 2020 oleh Andrei Borodin "Odyssey: arsitektur, konfigurasi, pemantauan"
Baru-baru ini, kami telah merilis versi 1.0 dari koneksi puller kami Odyssey . Ini dirancang untuk memecahkan masalah mengelola koneksi instalasi PostgreSQL yang sangat dimuat. Dalam laporan ini, saya ingin berbicara tentang arsitektur dan pengoperasian Odyssey. Masalah yang diselesaikan dalam transisi yang cukup panjang antara 1.0rc dan 1.0 juga akan terpengaruh.
Halo semuanya! Saya mengerjakan Postgres di Yandex. Dan dalam setengah tahun terakhir saya telah melakukan manajemen koneksi. Hal yang cukup kritis, karena itu menghalangi setiap permintaan. Dalam bahasa Rusia kita mengatakan "Odyssey."
Connection pooler. . , . connection pooler . . – , . , connection pooler , . Connection pooler . , .
Postgres . , Postgres ., - Postrgres’. . . .
., Postgres. , , , 1 .
. , Postgres. , , Odyssey .
? , . Postgres , , – , .
, .
, Postgres . . - . , procArray.
- , , Application-side pooling.
- – Proxy pooler.
- -, , pooler.
- , .
Application-side pooling – , , .
, , instances , .
, . , , .
, , .
instance Postgres. , . , . , - – .
Application-side pooling . , ?
pooler, . .
Pgpool II. , . . . - , , .
Crunchy-Proxy. Crunchy Data Proxy pooler. Go, - .
PgBouncer. pooler , pooling. . , , .
- PgBouncer.
.
PgBouncer . PgBouncer , . . , - , , , : , , . , PgBouncer .
, , , , Bouncer .
, , PgBouncer . . CPU 97 % . , , Bouncer .
, . , PgBouncer PgBouncer. PgBouncer, . reuseport. .
PgBouncer, Bouncer, - .
PgBouncer, connection pool PgBouncer. , .
- PgBouncer, , c PgBouncer - .
PgBouncer TLS-. TLS- – , , . . . , , . TLS handshake . PgBouncer, , TLS , . , . . , , .
, , , Odyssey – , PgBouncer .
- , PgBouncer , . . pull request. . . , PgBouncer .
Odyssey PgBouncer . .
connection pooler, Odyssey.
Odyssey . Odyssey. Kiwi, Machinarium.
Kiwi Postgres. Postgres Proto3. Kiwi , , . libpq Postgres, , . connection pooler Postgres’ .
Machinarium – , .
Machinarium . – , , thread . . – , . , , .
, . Machinarium. – , , , . sleep. , csw_worker Mashinarium’, - 0 .
Machinarium - , .
, , PgBouncer. PgBouncer libevent, Machinarium – libevent, connection pooling, .
Odyssey . main-, . . main-, , . . TCP-, , . - , , , .
, router, wait_read sleep, , . -. . , . . . . , , , , , . , , . , .
, , , , PgBouncer PgBouncer. . . , show stats, show clients, show pulls .
Odyssey production instances Postgres. . , PgBouncer, , . PgBouncer .
, . Odyssey production- , Postgres profession Odyssey. , production , . . , , .
. Odyssey, Odyssey TLS-. TLS- . .
Odyssey , , . connection pooler , , TLS-, , connection pooler, , TLS-.
, , .
, , , 20 000-30 000 TLS-, pooler TLS- . TLS handshake . - TLS handshake, , 3 30 000 TLS handshake, .
accept , . . , -, , , handshake -.
, , , Postgres , GSSAPI.
, Postgres , -, TLS-, GSSAPI.
PSQL GSSAPI. PgBouncer, Odyssey 12 Postgres . , , . .
Pooler , GSSAPI, , . TLS-.
Odyssey . 1.0 storages, , storage . . , . , walsender, : « ?». : «, basebackup» «, , walsender ».
, break inEnhaced. 1.1, , , , , . . . , . , .
– connection pooler. Connection pooler . . pooler’ , 1 000 .
Odyssey , , 1 000 , 1 000 , . , . PgBouncer , , OLTP-. pooler , .
, , . , , , , 4- . , PgBouncer, .
, – ML. PgBouncer, Odyssey – . , . .
pooler , . - . . ML-, . , .
, , . , , , .
, , , . , connection pooler, . , Python - . , , . .
PgBouncer , – , . TLS handshake, 15 , PgBouncer . . , 1.0. , , .
client_max_routing TLS handshakes. , TLS-.
, , .
Odyssey , , TLS-. listen.
Odyssey – , . , , .
Listen , .
Storage – . , remote, local. Local – , . . , PgBouncer PgBouncer.
, – database user, , . , , . .
, Postgres – client_fwd_error. «no», Bouncer. . . - , , Bouncer. «yes», , Postgres.
Pool_cancel pool_rollback – , , , , cancel .
Application_name_add_host – Bouncer, , ip- application_name .
, , quantiles quantiles .
? Bouncer . , . 10 % , , , - . 10 % .
– , . 0,99 , 99 % 1,273 . , - , - - , - cron-. . , , – .
Odyssey. PgBouncer.
Utilization – Odyssey. 4 , , Odyssey 4- . 400 %, , , .
Saturation Errors – , Odyssey, , . . - , .
Odyssey , online restart. PgBouncer PgBouncer – PgBouncer. , , Odyssey .
reload Odyssey , , Odyssey , . , reload , restart , .
Odyssey , , – IPv4 IPv6 - PgBouncer Odyssey PgBouncer , Odyssey IPv6. , , IPv4 , . , , . , IPv4 , , .
Odyssey Open Source . . . , PgBouncer source-port, , - , . - , .
, SCRAM .
merge pause resume. , . . , . - pooler. , . , 11 12, 13 Postgres.
! . . Odyssey? . . hba , Postgres pg_hba.conf . .? GitHub , , .
, .
ip, hostname . ., . . , .
iptables .
, iptables – , . Odyssey – Bouncer, , , pg_hba .
. Issue GitHub.
, .
-. , .
, !
! metrics endpoint, Prometheus .
, Prometheus. , . : , . , , . – , . . Prometheus - , ? .
, Prometheus - , , , . endpoints metrics , Prometheus’. , , .
?
Zabbix . , , , , Bouncer , . endpoint .
. . Odyssey http-?
-, . , Bouncer.
. .
?
– , . .
. - Prometheus, , , , ?
– Odyssey , - , , , . , , . - ? , .
. – , . , . . . - , , . , , . , .
, , «connection pooler». - . , . .
, .
, .
, . , , , , , , . , pooler, . , . ? 2 . pooler? , pooler .
.
.
! . , - Prometheus, . , , - , , Prometheus, Cloudwatch, , ?
Odyssey?
call Odyssey. , . , , , . , , .
show stats. ?
- , .
, exporter Prometheus.
exporter ?
Exporter, .
. . , Postgres Odyssey : « ?». Odyssey , . exporter - http . , Odyssey. Postgres CentOS . , . Exporter – . . , , C, - , , Go. . ?
! ? ?
? , . . open source. , . -, . GitHub. . , .
, ?
GCC Clang.
. ?
C99.
, - ? ? , ?
20 000 . 10 000 – . . . - , , , , , - .
? 500 MB , - , - . , , .
10 000 , ?
, , . . . 10 000 , – ? . . TLS handshake – 100 . 100 , 10 . . 10 , 100 . , - . 100 – . 10 . TLS handshake – .
Odyssey.conf max clients. ? , instance Odyssey per_worker? ? . . user1 , , 50 Odyssey, user2 , , 100. ?
, . . , TLS handshake, , . . . , . TLS - , , . max clients – . , .
max clients 20 000 20 000 TCP . 20 001- , : «» TLS handshake. pool size. pool size «1», , , , , - . , 5300: too many clients for this pool, . . sorry, .
. . wait - pool size?
Database. database user. user pool size. Postgres. Postgres : «too many clients». Odyssey . , round trip postmaster. Postmaster , . , , pool size. pool size.
! . Bouncer : MD5 hash. , MD5 hash?
.
. . ?
, . . include .
, ?
, . Storage , listen , . . GitHub : « , . . - , ». , . , - . , , . .