I propose to get acquainted with the decoding of the report of the end of 2019 by Alexander Valyalkin "Go optimizations in VictoriaMetrics"
VictoriaMetrics is a fast and scalable DBMS for storing and processing data in the form of a time series (a record forms time and a set of values corresponding to this time, for example, obtained through periodic polling of sensor status or collection of metrics).
Slides
. – . GitHub . Go . . fast, quick .
VictoriaMetrics. ? .
:
- , VictoriaMetrics.
- , .
- , .
- : .
- , VictoriaMetrics. bitset- Go.
VictoriaMetrics - ? , . . , – . ClickHouse, ClickHouse. , , : MergeTree, , .
VictoriaMetrics .
— . . , . VictoriaMetrics .
VictoriaMetrics — . . VictoriaMetrics, .
, VictoriaMetrics , . Go, .
, ? . — (timestamp, ), . – float64.
. ? -.
. : __name__="cpu_usage" – , instance="my-server" — , , datacenter="us-east" — , .
, -. (timestamp, value). t1, t3, t3, ..., tN — timestamps, 10, 20, 12, ..., 15 — . cpu-usage .
? - ?
- DevOps CPU, RAM, , rps, . .
- IoT – , geo coordinates -.
- – .
- , . , VictoriaMetrics , .
- . , ; ; , ; . .
- . , . .
- Healthcare – , ..
, , , , , , . .
? ?
, . . (timestamp, value) . API , -, , regexp. , , .
, SQL . , SQL , . , PromQL, InfluxQL, Flux, Q. , - . PromQL, , . Prometheus.
VictoriaMetrics.
. . .
, , label=value . .
- TSDB, . timeseries_ids , label=value. , timeseries_ids.
, select-.
timeseries_ids , label=value, .- data points
timeseries_ids. - - data points, . .
. timeseries_ids . VictoriaMetrics, , :)
. , . , ? , . , .
. , . ? label=value, label value — . – timeseries_ids, label=value.
timeseries_ids, label=value.
timeseries_ids label=value, label=regexp. ? timeseries_ids label=value.
. . .
getMetricIDs . label=value. metricIDs.
? , invertedIndex. (map), slice int-. label=value.
: metricIDs label=value, label=value, metricIDs . intersectInts, . .
, . . ? , . . . .
. . , , – out of memory error. .
, LevelDB, RocksDB.
, , .
LevelDB RocksDB – Google Facebook. LevelDB. Facebook LevelDB , RocksDB. RocksDB Facebook , RocksDB MySQL. MyRocks.
LevelDB. ? label=value. – , label=value.
c label=value, timeseries_ids. timeseries_ids, label=prefix, range scan, . . . , label=prefix timeseries_ids.
, Go. . LevelDB.
, . . , map . label=value. label=value metricIDs . .
, . VictoriaMetrics LevelDB. .
? LevelDB , . slice metricIDs. — slice .
LevelDB GetValues , label=value. timeseries_ids. timeseries_ids slice timeseries_ids. , , map' .
, LevelDB C. C- Go . . , , go, 1-5 , . VictoriaMetrics – :)
. mergeset.
Mergeset MergeTree. ClickHouse. , mergeset timeseries_ids . Mergeset Go. VictoriaMetrics GitHub. mergeset /lib/mergeset. , .
API mergeset LevelDB RocksDB. . . .
mergeset . , VictoriaMetrics production .
?
– high churn rate. – . , . .
– . , , . , , , . 4 . , , 100 400 . .
, . , , . 40 , , , 40 .
. , idle, . user space, kernel space . , . 7-8 .
40 8 = 320 . 100, 32 000 400.
Kubernetes. , Kubernetes . Kubernetes . . deployment . . , high-cardinality. VictoriaMetrics .
high churn rate. - high churn rate production? .
, Kubernetes, deployment, . . . Kubernetes - id’ deployment' label.
? , deployment' , deployment_id. .
, , , , , . – high churn rate.
high churn rate , - . .
? . . . . - , . . , .
, . , , id .
, , . . . , , .
. id , .
, , . .
, . VictoriaMetrics . . , . - , partition , , . . . VictoriaMetrics , . , .
. , timeseries_ids , , .
? – . . . label=value, . timeseries_ids .
. N , , , . id .
10 . , label=value N . , labels , Kubernetes.
– . . key=value . . . , . , . . . , ClickHouse. .
– timeseries_ids. , . timeseries_ids label=value.
– . . intersectInts slices — a b. slices.
. slice a, slice b. . , , . result.
? — . , slice a b , . , .
map. map. map slice a. slice b. – slice b map. , .
? , . . . slices. slice 2 , , .
, , map.
– overhead . . , VictoriaMetrics intersection map. , map map.
? Go . . . hash , HashMap. hash’ . VictoriaMetrics.
bitset, . slices. bitset. slice. slice. . . . . , map add has.
, , map, - , , 10 , map VictoriaMetrics.
, map. .
, , .
, – , . . . , ids VictoriaMetrics. , . , , out of memory error .
. , VictoriaMetrics, lib/uint64set. VictoriaMetrics, timeseries_id 64- , 32 32 .
, .
API. . API VictoriaMetrics. . . . , VictoriaMetrics.
add, . has, . del, . len, . clone . appendto slice timeseries_ids.
. set :
ItemsCount – , set. , , VictoriaMetrics bitset.
– buckets. slice bucket32. hi . 32 . slice — b16his buckets bucket16 .
16 64- . bitsets 16 .
Bucket64 uint64. . bucket16 2^16=65536 . 8, 8 . 8, 1000 uint64 . . . Bucket16 – 8- .
, .
uint64 . 32 , 32 . buckets. 32 bucket . , add b32 buckets. 32 . true, , . false, . .
bucket hi-, addAlloc, bucket, bucket-.
b32.add. . 16 , 16 .
16 . . add, bucket16.
, . uint64 id slice bit, bitmask. 64- , , . , , . , ids 10 maps.
VictoriaMetrics . , production.
– , , , , . . production, . , VictoriaMetrics , .
bitset. C++ vector bool, bitset. ?
, . bitset ids timeseries, VictoriaMetrics. , 32 . 32 . , . . C++ , , . , , .
. - ClickHouse . , C++ - . , , , , 32 .
. InfluxDB?
. , InfluxDB 10 high cardinality , , , . , VictoriaMetrics 1 GB , InfluxDB 10 Gb. .
, InfluxDB – Flux InfluxQL. PromQL, VictoriaMetrics. PromQL – Prometheus.
– , InfluxDB , fields . . . .
VictoriaMetrics . -. – – (timestamp, value), – label=value. fields measurements. , , , , , InfluxDB, , . , , .
. , - , , , ?
Go' map'. , , , . . , .
! ! . Wildberries. . . , , , , , , , . . , - ? 2 — bitset , , , ? , ?
. . , . , ?
. , . , .
. , , . churn rate , . . , .
– , – . Prometheus . Prometheus . , - label timeseries. , , , , VictoriaMetrics. VictoriaMetrics , - , .
, , . timeseries-, . . – intersect . , , .
?
.
LevelDB mergeset . . ? – . , . – 5 . 100 000 000 , , .
, . . , , . ? - VictoriaMetrics, ClickHouse . ., ?
, . , ? , , , – . . . . .
, . . , , . , , - ClickHouse , , , . , - , , , ? , , .
, . , Prometheus recording rules. . . , , . VictoriaMetrics , Prometheus, recoding rules.
, . , Go, . . . , . , - . , , . , ClickHouse timeseries-, .
. , , - Graphite Carbon. , . . , . . , , , , , . Prometheus, VictoriaMetrics . ? , ?
. . , (downsampling). . -, downsampling - : - , - , , . , . , . .
– , VictoriaMetrics, ClickHouse, , , , . VictoriaMetrics – 50 000 000 . . . . 20 , , . VictoriaMetrics ClickHouse downsamling.
, VictoriaMetrics . production 0,4 0,8 . – timestamp + . , .
. . ?
. . – . Graphite, , . OpenTSDB . InfluxDB – . VictoriaMetrics – , Prometheus . VictoriaMetrics remote storage Prometheus . .
, , , — , , . DevOps- , 30 , , , . high frequency trading systems, .
VictoriaMetrics DevOps-, , . , – high frequency trading systems.
! . PromQL?
. VictoriaMetrics PromQL. PromQL, MetricsQL. YouTube. Monitoring Meetup .
Telegram VictoriaMetrics.