Entschlüsselung des Berichts 2016 von Vitaliy Levchenko "Prometheus-Überwachung von Microservice-Anwendungen"
Prometheus macht es im Gegensatz zu klassischen Systemen einfach, die Überwachung sich schnell ändernder und komplex organisierter Systeme zu erhöhen und aufrechtzuerhalten. Ich werde über Implementierungserfahrungen, Fallstricke und unerwartetes Verhalten sprechen und zeigen, wie das gesamte System, einschließlich Benachrichtigungen und Dashboards, schnell konfiguriert werden kann.
, . , , , job' — . : — , , .. , , , , .
Prometheus Google Borgmon , . , , — . — , . — .
, Prometheus .
, , , . , . Prometheus , . :
. . . .
, , developer. . . .
, , , .
? . . docker. , , , .
, . . , . , , – . ? , – . , production .
– production . production 10 . , continuous delivery. stages. . . .
, . . .
, . , , .
. , – . , - environments – . stage environment, environment, production.
, , . . , 20 , .
. The twelve factor app – Production. , , . – .
, .
. . . , , , . – . . . . , . . , . . .
– . . , , , Graphite, StatsD. , , push based, . .
– , , . . , , .
, , . , . Graphite, . . , 100 000 – . – , 100 000 . – . , 21 . , , , .
production Zabbix - ? . – . . . Zabbix – , . . , .
, . , , Zabbix . Nagios . , , . . . , , - .
, , , . Zabbix . , , Redis. .
, , . InfluxDB . . Graphite, InfluxDB , , .
InfluxDB , . . production. . . , . . .
, . , , . . . (?) . . .
, , , . . . , . . , – . .
Riemann . : « CPU ». CPU – 100 %, 0 %. .
– , CollectD . , . InfluxDB. - , - , - , CollectD , . , , .
InfluxDB , time-series , .
Prometheus. ?
Google, , , Google Borgmom, , Facebook. , , , . . .
, . . production ( , , , ) , . . Prometheus .
. . Prometheus , . .
, . deprecate- . Grafana. Grafana , .
, Prometheus . . . . , 20 000 – . 24- 400 000 . 3 , 1 200 000. .
production . - . , . . , - .
, , , . . . environment. server, . handler . . . Prometheus . , .
, , . , , , . . , . .
? . : « environment , ». .
. - , , . , , , – .
Prometheus , . . , .
. . , 4 . . , . Zookeeper, , … 95 % , 20 100 %. . – . . . , .
- pull metrics push metrics. , pull metrics .
StatsD – Graphite, . StatsD , . . .
, exporter . , , 50 000 . . , , production, .
StatsD exporter Prometheus . StatsD, StatsD, gauge etc, .
push gateway push’. Prometheus. StatsD, push gateway .
. , Prometheus, , . . . , , . . Amazon, Kubernetes, Mesos, Consul . . . , .
, Ansible, . . . - , , , . , Prometheus. reload , .
, . , , . javascript . , . .
. - .
Prometheus . . open source. , Postgres exporter – , . Node exporter, .
, . , . , . .
systemd. .
, . , .
. , . . , , . , , . Prometheus . . . Influx, .
. .
. , - , . , , . – –rm – r <storage path/*. . . , , production. , , .
federation, , federation , .
. . . Prometheus . .
openTSDB . , . openTSDB , , , , Hadoop.
, , , , histogram summary. , .
, , 100 – . 100 300 – . , 300 – . histogram. , … , , , . . . , histogram , . 10 , 10 . - . , recording rules. , , . , .
Summary – . Summary . Summary , . , , , . . , . , 99- , , . . , . .
summary . . . , .
-. . .
? ? , , , :
- . , , . . , latency , , . 99 % . , , - . . .
- .
- .
- .
- + .
?
- . , . . 10 , 10 , 10 , . ( Service Discovery)
- , . Grafana . , . : « ». 10 . , . . , , .
- – , . . . . , , , , .
- – , , , , .
- , Prometheus . . , Prometheus docker. . . – . . .
! ! Zabbix. , . , . , , Zabbix. environment, ?
. – environment, . ( Kubernetes Service Discovery)
. ., , developer, ? . . environment?
.
environment?
environment. environment. environment. , environment, stable – environment. ( Kubernetes Service Discovery)
, . . . - , , . , - . . . , - , , . , ?
Zabbix production, , . , , , -. full time , . Zabbix, , , 3-4- . 3-4- , Zabbix – . .
, 3 Zabbix?
.
. . developers Prometheus?
.
? ?
.
. environment, - sender, Prometheus .
sender. Prometheus . API http’, . Consul, Prometheus , . .
. , , - Prometheus?
, . .
. docker’, , , , - , . overhead, . . . , ?
?
.
health handler, . , , . , , .
! ! 4 . StatsD, , , . StatsD, Influx’ Telegraf?
, StatsD, Influx.
Ok. , Grafana . , . Go, . , .
open source, .
GitHub , . . federation, ?
, . .
Ok. -?
, .
, . ? ?
. , . 100 000 . , 200 000-300 000. , , , , . Zabbix, . Influx , . . , queries .
?
?
, - , , .
, - . , . . .