![](https://habrastorage.org/webt/kz/1o/dl/kz1odlxib3nbua9hw17spxs-qwy.png)
A pesquisa elĂĄstica Ă© enfrentada por muitos. Mas o que acontece quando vocĂȘ deseja usĂĄ-lo para armazenar logs "em um volume particularmente grande"? Sim, e sem esforço sobreviver Ă falha de qualquer um dos vĂĄrios data centers? Que arquitetura vale a pena fazer e quais sĂŁo as armadilhas?
elasticsearch -, : , .
â , . , Manticore Search, Sphinx search, Elasticsearch. , - ...search, . .
, , Elasticsearch. , - , -, .
:
- Graylog. , , .
- : 50-80 , - , , 2-3
- , , : .
- , , , .
- , â - .
- , : , , - (, ).
, .
-, - Elasticsearch ( ).
- 18 â , , .
: Podman , one-cloud. 2 , 2.0Ghz v4 , .
:
![](https://habrastorage.org/webt/p9/3t/in/p93tinvpqlgzst0ef81rdxz63ii.jpeg)
:
- 3-4 VIP - Graylog, , .
- VIP LVS.
- Graylog, GELF, syslog.
- Elasticsearch.
- , , -.
![](https://habrastorage.org/webt/uo/ny/3k/uony3kuusk3jcczrdtuo2-atrrk.jpeg)
, , .
Elasticsearch â master, coordinator, data node. , .
Master
, , , cluster wide housekeeping.
Coordinator
- : . , , , master, , .
Data node
, , , .
Graylog
- Kibana Logstash ELK-. Graylog UI . Graylog Kafka Zookeeper, Graylog . Graylog (Kafka) Elasticsearch , . Logstash, Graylog Elasticsearch.
, Graylog service discovery, Elasticsearch , .
:
![](https://habrastorage.org/webt/cs/-d/qw/cs-dqwa67gjezp1mqohibpkqj4a.jpeg)
. , .
, , , .
: Elasticsearch data nodes.
â , Elasticsearch. , Lucene index. Lucene index, , .
![](https://habrastorage.org/webt/br/39/ub/br39ub-mmn2nqzzl8gliudk4hau.jpeg)
, «» -.
, ( ) -. -, , replication factor, ( ). , -, , , , .
30 .
:
![](https://habrastorage.org/webt/f7/rd/pk/f7rdpkxd79_0cwzig6b_e-6yace.jpeg)
- â . â primary-, . â replica-. -.
, -. , , , :
![](https://habrastorage.org/webt/lw/0j/jq/lw0jjqdhp0x8pcr0zkds_vrrjga.jpeg)
, .. , 48 ( : 48 ).
:
- , , , , . ââ . â â , .
, - , . - , - . .
latency , SSD. , , 56 . 56 - , , Elasticsearch . Elasitcsearch thread pool , - " â ".
, - 20 , 1 360 . , 48 , 15 . 2 .
, .
, Graylog - . , 2-3 .
, Graylog , : « , â ».
Master : « 71», -, primary-shard 71.
replica-shard, -.
![](https://habrastorage.org/webt/6o/8h/tq/6o8htqw69mkv93ypgil2xidxyxu.jpeg)
Graylog . , Elasticsearch round-robin primary-shard replica-shard.
![](https://habrastorage.org/webt/rf/1h/qh/rf1hqhqde1xniyf2wp_zndd-0rm.jpeg)
180 , , , , «» -. , , , .
48 300-400ms, , leading wildcard.
«» Elasticsearch: Java
![](https://habrastorage.org/webt/c1/bt/zx/c1btzxkqrav3llzzjhntt98waae.jpeg)
, , .
, Elasticsearch Java.
, Lucene, background job', Lucene . , OutOfMemoryError-. , , , .
, Lucene- . . ( heap.size RAM), - off-heap , - ~500MB, .
: RAM , , .
4-5 , - 10-20.
, , off-heap Elasticsearch . , direct buffer pools , , explicit GC Elasticsearch.
- , . .
: Java . 16 (-XX:MaxDirectMemorySize=16g), , explicit GC , , .
, «, » , .
, mmapfs, . , mmapfs , mapped-. - , GC safepoint, , . , master , . 5-10 garbage collector, , . â, â - .
, niofs, , Elastic , hybridfs, . .
, . 2-3 , .
Full GC, - , . GC : , , , â .
, , - , . , , .
, , - , - Elasticsearch, , .
, , , . GC , . GC, , .
, , â JDK13 Shenandoah. , .
Java .
«» Elasticsearch:
![](https://habrastorage.org/webt/lh/8x/ry/lh8xryhh1otgj99hhi5zhhdgv-e.jpeg)
, , - .
: - «» , , Graylog es_rejected_execution.
- , thread_pool.write.queue - , Elasticsearch , 200 . Elasticsearch . - .
, : 300 , , Full GC.
, , , Graylog , , 3 , . , , Elasticsearch , , ( ), , , .
, - - , Elastic, - â - Graylog.
, , : , , .
. , , , , , , .
, Elasticsearch , - round-robin (, primary-shard, , ), replica-shard, . , use_adaptive_replica_selection: true.
:
![](https://habrastorage.org/webt/ci/dy/mk/cidymktcic-xg9gn2f5sgbqupz0.jpeg)
query time , .
, -.
:
- - master, .
- .
- : - - primary-, replica- -, .
- , , .
, - :
![](https://habrastorage.org/webt/so/kd/nb/sokdnbryweq4yqhz50ikx54p2mk.jpeg)
:
![](https://habrastorage.org/webt/iq/lv/h-/iqlvh-nojl1kwbe8hafvd7cr-ii.jpeg)
?
- .
?
, TaskBatcher, , . , replica primary, - - â TaskBatcher, .
- , - - « - - -».
- , . , , . , , .
, - , full GC. - , , .
, 6.4.0, , 10 - 360 , .
:
![](https://habrastorage.org/webt/h-/rm/hw/h-rmhw1kyfmivtzdzmt0x3wn3xm.jpeg)
6.4.0, , - . «» . : 2, 3 10 ( , ) -, - , , , B, C, D.
- - - , - 20-30 , - .
, , , « » . , , 7.2.
, - , , , , - primary-shard ( replica-shard - primary, ).
, «», - stale . , , - , -, - - . .
- 5 . .
:
- 360 - 700 .
- 60 -.
- 40 , 6.4.0 â -, ,
- , .
- heap.size, 31 : , leading wildcard - , circuit breaker Elasticsearch.
- , , , .
, , :
- - , , - . - - , 2-3 , 2, 3, 4 â , - , .
- , pending-. , , - , , , replica- primary, - .
- garbage collector, .
- , , « ».
- , heap, RAM I/O.
Thread Pool Elasticsearch. Elasticsearch , , thread_pool.management. , , _cat/shards , . , , thread_pool.management , , 5 , , .
: ! , .
, - , , , , .
![](https://habrastorage.org/webt/eo/-i/_j/eo-i_jeqv7iqnialrja9bnbwrpc.jpeg)