Cloister -> easy OTP cluster management

Almost every successful business application sooner or later enters a phase when horizontal scaling is required. In many cases, you can simply start a new instance and reduce the average load. But there are less trivial cases when we must ensure that different nodes know about each other and accurately distribute the workload.

It so happened that erlang , which we chose for its nice syntax and hype around, has first-class support for distributed systems . In theory, this sounds generally trivial:

Messaging between processes on different nodes, as well as between links and monitors is transparent [...]

In practice, things are a little more complicated. Distributed erlang was developed when “container” meant such a large iron box for transportation, and “docker” was simply synonymous with a port loader. In IP4 there were many unoccupied addresses, in network gaps - rats that were gnawed were usually to blame, and the average uptime of a production system was measured for decades.

, , erlang , IP- , . , erlang, , .

: libcluster. , ,  — , .  —  . , , . .

, , :

• , erlang;
• ( , , , );
• , :nonode@nohost;
• , .

, , :nonode@nohost, - test_cluster_task, docker-compose up --scale my_app=3 , - . , , mnesia — , - .

Cloister , : . , , , CS . , . , .

Cloister , (assembly and maintainance) , Cloister.Manager .

, config, :

config :cloister,
  otp_app: :my_app,
  sentry: :"cloister.local", # or ~w|n1@foo n2@bar|a
  consensus: 3,              # number of nodes to consider
                             #    the cluster is up
  listener: MyApp.Listener   # listener to be called when
                             #    the ring has changed

: Cloister OTP :my_app, erlang service discovery , , MyApp.Listener ( @behaviour Cloister.Listener) . .

, Cloister , ( , .) , , . ( , ), MyApp.Listener.on_state_change/2. , %Cloister.Monitor{status: :up}, : «, ».

consensus: 3 , , , status: :rehashing → status: :up .

, consensus: 1 Cloister , :nonode@nohost, :node@host, :node@host.domain — , (:none | :shortnames | :longnames).

, mnesia. on_state_change/2. , , , mnesia Cloister.

Cloister , . , , , IP- , , , /. , :nonode@nohost. .

, MyApp.Listener, , . , libcluster, , . , , , -.

Note: at this point in the original was the phrase “Happy clustering!”, And Yandex, which I translate (not to climb the dictionaries myself), offered me the option “Happy cluster!” Perhaps the best translation, especially in the light of the current geopolitical situation, is impossible to imagine.