Monitoring Apache Ignite. Done right

We made 2 subsystems inside Apache Ignite.

In the article I will tell about their architecture:

• How did the metrics subsystem and the system view subsystem.
• What has been done and what are we going to do?

Hello everyone! My name is Nikolai Izhikov. I work at Sberbank Technologies to develop Open Source products.

Note 1 : “done” means that this functionality is available in master Apache Ignite , you can build and see how everything works. The functionality will be released in the 2.8 Apache Ignite release, which will be released in early 2020.


 2: , , Apache Ignite.


 3: 2.8, , - .

Apache Ignite in-memory . API: , , ( ), .


, - . : " , ? , , ." :

• ? — , , .
• ? — , , : , , , . , .
• ? — .
• ? — " " — ?

.

:

• .
• — , .
• . : , .
• : , , , , .

, . , , . :

• API : transaction manager, cache manager, etc.
• - .
• API: JMX, SQL, Java. API.

:

• : , ? . , , , - . , .
• «» : , API.
• : jmx, sql, http(Prometheus) , , .
• : , .

Open Source community . , Ignite Enhancement Proposal (IEP), , , API, .
:

Ignite : , . , . , . , Ignite . , Ignite — . — . (prometheus, zabbix) .

Ignite . Ignite : jvm, , , . Ignite . , .

: — .

: , data region', .. , , .

: io.dataregion.myregion.TotalAllocatedPages — Data Region "myregion".

:

• Metric, Gauge — .
• MetricRegistry — (, ) .
• MetricExporterSpi — .
• ReadOnlyMetricRegistry — , exporter’-. read only access + listeners.
• GridMetricManager — .

:

• Metric — . .
  : LongAddterMetric, DoubleMetricImpl, ObjectMetricImpl.
• Gauge — . , Supplier. . , , .
  : LongGauge, DoubleGauge, ObjectGauge.
• HistogramMetric — + . , .
  : , 250 , 500 , 1 .
• HitRateMetric — X .
  : , .

MetricExporterSpi — , . , . JMX, SQL, Log, OpenCensus.

GridMetricManager mmgr = ...;
MetricRegistry mreg = mmgr.registry("io.dataregion." + name);
LongAdderMetric replacedPages = mreg.longAdderMetric("PagesReplaced",
            "Number of pages replaced from last restart.");
//....
replacedPages.increment();

JMX

SQL

, " ?" .
.

System View — ?

— . system view, SQL. ( JMX).

• SystemView — (). : , , compute task', sql table . .
• SystemViewExporterSpi — .
• ReadOnlySystemViewRegistry — exporter'. read only access + listeners.
• GridSystemViewManager —

:

• . , , - . .
• overhead. .
• . . .

: . java-. :

POJO. , . . . : reflection?

:

• compile time.
• .
• .
• .

:
(SystemViewRowAttributeWalker.java), . . system view . .

        ctx.systemView().registerView(CACHES_VIEW, CACHES_VIEW_DESC,
            new CacheViewWalker(),
            registeredCaches.values(),
            CacheView::new);

        ctx.systemView().registerView(CACHE_GRPS_VIEW, CACHE_GRPS_VIEW_DESC,
            new CacheGroupViewWalker(),
            registeredCacheGrps.values(),
            CacheGroupView::new);

Reflection — , . dev-list JMH benchmark. reflection . walker . Walker 4 .

SQL

JMX

, " ?" .

• : , . .
• « » (prometheus, zabbix)
• . , . — .
• . Oracle , , : , , , .
• — Ignite , . . threshold «compute 30 », , .

• dev@ignite.apache.org
• user@ignite.apache.org
  Come! We have almost a Russian-speaking community, almost all contributors are from Russia, so you can easily join and make a contribution to Open Source.

Presentation - https://www.highload.ru/moscow/2019/abstracts/6111
Video: