Ich schlage vor, mich mit der Entschlüsselung des Berichts von Ende 2019 durch Alexander Valyalkin "Go Optimizations in VictoriaMetrics" vertraut zu machen.
VictoriaMetrics ist ein schnelles und skalierbares Datenbankverwaltungssystem zum Speichern und Verarbeiten von Daten in Form einer Zeitreihe (ein Datensatz bildet die Zeit und eine Reihe von Werten, die dieser Zeit entsprechen, z. B. durch regelmäßige Abfrage des Sensorstatus oder Erfassung von Metriken).
Folien
. – . GitHub . Go . . fast, quick .
VictoriaMetrics. ? .
:
- , VictoriaMetrics.
- , .
- , .
- : .
- , VictoriaMetrics. bitset- Go.
VictoriaMetrics - ? , . . , – . ClickHouse, ClickHouse. , , : MergeTree, , .
VictoriaMetrics .
— . . , . VictoriaMetrics .
VictoriaMetrics — . . VictoriaMetrics, .
, VictoriaMetrics , . Go, .
, ? . — (timestamp, ), . – float64.
. ? -.
. : __name__="cpu_usage" – , instance="my-server" — , , datacenter="us-east" — , .
, -. (timestamp, value). t1, t3, t3, ..., tN — timestamps, 10, 20, 12, ..., 15 — . cpu-usage .
? - ?
- DevOps CPU, RAM, , rps, . .
- IoT – , geo coordinates -.
- – .
- , . , VictoriaMetrics , .
- . , ; ; , ; . .
- . , . .
- Healthcare – , ..
, , , , , , . .
? ?
, . . (timestamp, value) . API , -, , regexp. , , .
, SQL . , SQL , . , PromQL, InfluxQL, Flux, Q. , - . PromQL, , . Prometheus.
VictoriaMetrics.
. . .
, , label=value . .
- TSDB, . timeseries_ids , label=value. , timeseries_ids.
, select-.
timeseries_ids , label=value, .- data points
timeseries_ids. - - data points, . .
. timeseries_ids . VictoriaMetrics, , :)
. , . , ? , . , .
. , . ? label=value, label value — . – timeseries_ids, label=value.
timeseries_ids, label=value.
timeseries_ids label=value, label=regexp. ? timeseries_ids label=value.
. . .
getMetricIDs . label=value. metricIDs.
? , invertedIndex. (map), slice int-. label=value.
: metricIDs label=value, label=value, metricIDs . intersectInts, . .
, . . ? , . . . .
. . , , – out of memory error. .
, LevelDB, RocksDB.
, , .
LevelDB RocksDB – Google Facebook. LevelDB. Facebook LevelDB , RocksDB. RocksDB Facebook , RocksDB MySQL. MyRocks.
LevelDB. ? label=value. – , label=value.
c label=value, timeseries_ids. timeseries_ids, label=prefix, range scan, . . . , label=prefix timeseries_ids.
, Go. . LevelDB.
, . . , map . label=value. label=value metricIDs . .
, . VictoriaMetrics LevelDB. .
? LevelDB , . slice metricIDs. — slice .
LevelDB GetValues , label=value. timeseries_ids. timeseries_ids slice timeseries_ids. , , map' .
, LevelDB C. C- Go . . , , go, 1-5 , . VictoriaMetrics – :)
. mergeset.
Mergeset MergeTree. ClickHouse. , mergeset timeseries_ids . Mergeset Go. VictoriaMetrics GitHub. mergeset /lib/mergeset. , .
API mergeset LevelDB RocksDB. . . .
mergeset . , VictoriaMetrics production .
?
– high churn rate. – . , . .
– . , , . , , , . 4 . , , 100 400 . .
, . , , . 40 , , , 40 .
. , idle, . user space, kernel space . , . 7-8 .
40 8 = 320 . 100, 32 000 400.
Kubernetes. , Kubernetes . Kubernetes . . deployment . . , high-cardinality. VictoriaMetrics .
high churn rate. - high churn rate production? .
, Kubernetes, deployment, . . . Kubernetes - id’ deployment' label.
? , deployment' , deployment_id. .
, , , , , . – high churn rate.
high churn rate , - . .
? . . . . - , . . , .
, . , , id .
, , . . . , , .
. id , .
, , . .
, . VictoriaMetrics . . , . - , partition , , . . . VictoriaMetrics , . , .
. , timeseries_ids , , .
? – . . . label=value, . timeseries_ids .
. N , , , . id .
10 . , label=value N . , labels , Kubernetes.
– . . key=value . . . , . , . . . , ClickHouse. .
– timeseries_ids. , . timeseries_ids label=value.
– . . intersectInts slices — a b. slices.
. slice a, slice b. . , , . result.
? — . , slice a b , . , .
map. map. map slice a. slice b. – slice b map. , .
? , . . . slices. slice 2 , , .
, , map.
– overhead . . , VictoriaMetrics intersection map. , map map.
? Go . . . hash , HashMap. hash’ . VictoriaMetrics.
bitset, . slices. bitset. slice. slice. . . . . , map add has.
, , map, - , , 10 , map VictoriaMetrics.
, map. .
, , .
, – , . . . , ids VictoriaMetrics. , . , , out of memory error .
. , VictoriaMetrics, lib/uint64set. VictoriaMetrics, timeseries_id 64- , 32 32 .
, .
API. . API VictoriaMetrics. . . . , VictoriaMetrics.
add, . has, . del, . len, . clone . appendto slice timeseries_ids.
. set :
ItemsCount – , set. , , VictoriaMetrics bitset.
– buckets. slice bucket32. hi . 32 . slice — b16his buckets bucket16 .
16 64- . bitsets 16 .
Bucket64 uint64. . bucket16 2^16=65536 . 8, 8 . 8, 1000 uint64 . . . Bucket16 – 8- .
, .
uint64 . 32 , 32 . buckets. 32 bucket . , add b32 buckets. 32 . true, , . false, . .
bucket hi-, addAlloc, bucket, bucket-.
b32.add. . 16 , 16 .
16 . . add, bucket16.
, . uint64 id slice bit, bitmask. 64- , , . , , . , ids 10 maps.
VictoriaMetrics . , production.
– , , , , . . production, . , VictoriaMetrics , .
bitset. C++ vector bool, bitset. ?
, . bitset ids timeseries, VictoriaMetrics. , 32 . 32 . , . . C++ , , . , , .
. - ClickHouse . , C++ - . , , , , 32 .
. InfluxDB?
. , InfluxDB 10 high cardinality , , , . , VictoriaMetrics 1 GB , InfluxDB 10 Gb. .
, InfluxDB – Flux InfluxQL. PromQL, VictoriaMetrics. PromQL – Prometheus.
– , InfluxDB , fields . . . .
VictoriaMetrics . -. – – (timestamp, value), – label=value. fields measurements. , , , , , InfluxDB, , . , , .
. , - , , , ?
Go' map'. , , , . . , .
! ! . Wildberries. . . , , , , , , , . . , - ? 2 — bitset , , , ? , ?
. . , . , ?
. , . , .
. , , . churn rate , . . , .
– , – . Prometheus . Prometheus . , - label timeseries. , , , , VictoriaMetrics. VictoriaMetrics , - , .
, , . timeseries-, . . – intersect . , , .
?
.
LevelDB mergeset . . ? – . , . – 5 . 100 000 000 , , .
, . . , , . ? - VictoriaMetrics, ClickHouse . ., ?
, . , ? , , , – . . . . .
, . . , , . , , - ClickHouse , , , . , - , , , ? , , .
, . , Prometheus recording rules. . . , , . VictoriaMetrics , Prometheus, recoding rules.
, . , Go, . . . , . , - . , , . , ClickHouse timeseries-, .
. , , - Graphite Carbon. , . . , . . , , , , , . Prometheus, VictoriaMetrics . ? , ?
. . , (downsampling). . -, downsampling - : - , - , , . , . , . .
– , VictoriaMetrics, ClickHouse, , , , . VictoriaMetrics – 50 000 000 . . . . 20 , , . VictoriaMetrics ClickHouse downsamling.
, VictoriaMetrics . production 0,4 0,8 . – timestamp + . , .
. . ?
. . – . Graphite, , . OpenTSDB . InfluxDB – . VictoriaMetrics – , Prometheus . VictoriaMetrics remote storage Prometheus . .
, , , — , , . DevOps- , 30 , , , . high frequency trading systems, .
VictoriaMetrics DevOps-, , . , – high frequency trading systems.
! . PromQL?
. VictoriaMetrics PromQL. PromQL, MetricsQL. YouTube. Monitoring Meetup .
Telegram VictoriaMetrics.