Proponho me familiarizar com a transcrição do relatório de início de 2020 de Andrei Borodin "Odyssey: arquitetura, configuração, monitoramento"
Mais recentemente, lançamos a versão 1.0 de nossas conexões extratoras Odyssey . Ele foi desenvolvido para resolver os problemas de gerenciamento de conexões de instalações PostgreSQL altamente carregadas. Neste relatório, gostaria de falar sobre a arquitetura e operação da Odyssey. Problemas que foram resolvidos em uma transição bastante longa entre 1.0rc e 1.0 também serão afetados.
Olá a todos! Eu faço o Postgres no Yandex. E no último ano e meio, venho fazendo o gerenciamento de conexões. Coisa bastante crítica, porque fica no caminho de cada solicitação. Em russo, dizemos "Odisséia".
Connection pooler. . , . connection pooler . . – , . , connection pooler , . Connection pooler . , .
Postgres . , Postgres ., - Postrgres’. . . .
., Postgres. , , , 1 .
. , Postgres. , , Odyssey .
? , . Postgres , , – , .
, .
, Postgres . . - . , procArray.
- , , Application-side pooling.
- – Proxy pooler.
- -, , pooler.
- , .
Application-side pooling – , , .
, , instances , .
, . , , .
, , .
instance Postgres. , . , . , - – .
Application-side pooling . , ?
pooler, . .
Pgpool II. , . . . - , , .
Crunchy-Proxy. Crunchy Data Proxy pooler. Go, - .
PgBouncer. pooler , pooling. . , , .
- PgBouncer.
.
PgBouncer . PgBouncer , . . , - , , , : , , . , PgBouncer .
, , , , Bouncer .
, , PgBouncer . . CPU 97 % . , , Bouncer .
, . , PgBouncer PgBouncer. PgBouncer, . reuseport. .
PgBouncer, Bouncer, - .
PgBouncer, connection pool PgBouncer. , .
- PgBouncer, , c PgBouncer - .
PgBouncer TLS-. TLS- – , , . . . , , . TLS handshake . PgBouncer, , TLS , . , . . , , .
, , , Odyssey – , PgBouncer .
- , PgBouncer , . . pull request. . . , PgBouncer .
Odyssey PgBouncer . .
connection pooler, Odyssey.
Odyssey . Odyssey. Kiwi, Machinarium.
Kiwi Postgres. Postgres Proto3. Kiwi , , . libpq Postgres, , . connection pooler Postgres’ .
Machinarium – , .
Machinarium . – , , thread . . – , . , , .
, . Machinarium. – , , , . sleep. , csw_worker Mashinarium’, - 0 .
Machinarium - , .
, , PgBouncer. PgBouncer libevent, Machinarium – libevent, connection pooling, .
Odyssey . main-, . . main-, , . . TCP-, , . - , , , .
, router, wait_read sleep, , . -. . , . . . . , , , , , . , , . , .
, , , , PgBouncer PgBouncer. . . , show stats, show clients, show pulls .
Odyssey production instances Postgres. . , PgBouncer, , . PgBouncer .
, . Odyssey production- , Postgres profession Odyssey. , production , . . , , .
. Odyssey, Odyssey TLS-. TLS- . .
Odyssey , , . connection pooler , , TLS-, , connection pooler, , TLS-.
, , .
, , , 20 000-30 000 TLS-, pooler TLS- . TLS handshake . - TLS handshake, , 3 30 000 TLS handshake, .
accept , . . , -, , , handshake -.
, , , Postgres , GSSAPI.
, Postgres , -, TLS-, GSSAPI.
PSQL GSSAPI. PgBouncer, Odyssey 12 Postgres . , , . .
Pooler , GSSAPI, , . TLS-.
Odyssey . 1.0 storages, , storage . . , . , walsender, : « ?». : «, basebackup» «, , walsender ».
, break inEnhaced. 1.1, , , , , . . . , . , .
– connection pooler. Connection pooler . . pooler’ , 1 000 .
Odyssey , , 1 000 , 1 000 , . , . PgBouncer , , OLTP-. pooler , .
, , . , , , , 4- . , PgBouncer, .
, – ML. PgBouncer, Odyssey – . , . .
pooler , . - . . ML-, . , .
, , . , , , .
, , , . , connection pooler, . , Python - . , , . .
PgBouncer , – , . TLS handshake, 15 , PgBouncer . . , 1.0. , , .
client_max_routing TLS handshakes. , TLS-.
, , .
Odyssey , , TLS-. listen.
Odyssey – , . , , .
Listen , .
Storage – . , remote, local. Local – , . . , PgBouncer PgBouncer.
, – database user, , . , , . .
, Postgres – client_fwd_error. «no», Bouncer. . . - , , Bouncer. «yes», , Postgres.
Pool_cancel pool_rollback – , , , , cancel .
Application_name_add_host – Bouncer, , ip- application_name .
, , quantiles quantiles .
? Bouncer . , . 10 % , , , - . 10 % .
– , . 0,99 , 99 % 1,273 . , - , - - , - cron-. . , , – .
Odyssey. PgBouncer.
Utilization – Odyssey. 4 , , Odyssey 4- . 400 %, , , .
Saturation Errors – , Odyssey, , . . - , .
Odyssey , online restart. PgBouncer PgBouncer – PgBouncer. , , Odyssey .
reload Odyssey , , Odyssey , . , reload , restart , .
Odyssey , , – IPv4 IPv6 - PgBouncer Odyssey PgBouncer , Odyssey IPv6. , , IPv4 , . , , . , IPv4 , , .
Odyssey Open Source . . . , PgBouncer source-port, , - , . - , .
, SCRAM .
merge pause resume. , . . , . - pooler. , . , 11 12, 13 Postgres.
! . . Odyssey? . . hba , Postgres pg_hba.conf . .? GitHub , , .
, .
ip, hostname . ., . . , .
iptables .
, iptables – , . Odyssey – Bouncer, , , pg_hba .
. Issue GitHub.
, .
-. , .
, !
! metrics endpoint, Prometheus .
, Prometheus. , . : , . , , . – , . . Prometheus - , ? .
, Prometheus - , , , . endpoints metrics , Prometheus’. , , .
?
Zabbix . , , , , Bouncer , . endpoint .
. . Odyssey http-?
-, . , Bouncer.
. .
?
– , . .
. - Prometheus, , , , ?
– Odyssey , - , , , . , , . - ? , .
. – , . , . . . - , , . , , . , .
, , «connection pooler». - . , . .
, .
, .
, . , , , , , , . , pooler, . , . ? 2 . pooler? , pooler .
.
.
! . , - Prometheus, . , , - , , Prometheus, Cloudwatch, , ?
Odyssey?
call Odyssey. , . , , , . , , .
show stats. ?
- , .
, exporter Prometheus.
exporter ?
Exporter, .
. . , Postgres Odyssey : « ?». Odyssey , . exporter - http . , Odyssey. Postgres CentOS . , . Exporter – . . , , C, - , , Go. . ?
! ? ?
? , . . open source. , . -, . GitHub. . , .
, ?
GCC Clang.
. ?
C99.
, - ? ? , ?
20 000 . 10 000 – . . . - , , , , , - .
? 500 MB , - , - . , , .
10 000 , ?
, , . . . 10 000 , – ? . . TLS handshake – 100 . 100 , 10 . . 10 , 100 . , - . 100 – . 10 . TLS handshake – .
Odyssey.conf max clients. ? , instance Odyssey per_worker? ? . . user1 , , 50 Odyssey, user2 , , 100. ?
, . . , TLS handshake, , . . . , . TLS - , , . max clients – . , .
max clients 20 000 20 000 TCP . 20 001- , : «» TLS handshake. pool size. pool size «1», , , , , - . , 5300: too many clients for this pool, . . sorry, .
. . wait - pool size?
Database. database user. user pool size. Postgres. Postgres : «too many clients». Odyssey . , round trip postmaster. Postmaster , . , , pool size. pool size.
! . Bouncer : MD5 hash. , MD5 hash?
.
. . ?
, . . include .
, ?
, . Storage , listen , . . GitHub : « , . . - , ». , . , - . , , . .