Kubernetes load balancing and scaling long-lived connections

A bit about how traffic is redistributed in Kubernetes 

1. In the diagram below, you see three instances of the same application and a load balancer:
   
   
   
2. The load balancer is called the Service, it is assigned an IP address. Any incoming request is redirected to one of the pods:
   
   
   
3. The deployment script determines the number of application instances. You will almost never have to deploy directly under:
   
   
   
4. Each pod is assigned its own IP address:
   
   
   

1. There is a curl request 10.96.45.152 to the service:
   
   
   
2. The service selects one of the three pod addresses as the destination:
   
   
   
3. Traffic is redirected to a specific pod:
   
   
   

1. Under 1 requests the backend internal component. Instead of choosing a specific one for the backend, it performs a service request:
   
   
   
2. The service selects one of the backend pods as the destination address:
   
   
   
3. Traffic goes from hearth 1 to hearth 5 selected by the service:
   
   
   
4. Under 1, it does not know exactly how many such hearths as under 5 are hidden behind the service:
   
   
   

Balancing in Kubernetes Services

1. Consider a cluster of three nodes. There are pods on each node:
   
   
   
2. Knitted hearths painted in beige are part of the service. Since the service does not exist as a process, it is grayed out:
   
   
   
3. The first one asks for the service and should fall on one of the related headers:
   
   
   
4. But the service does not exist, there is no process. How does it work?
   
   
   
5. Before the request leaves the node, it goes through the iptables rules:
   
   
   
6. The iptables rules know that there is no service, and replace its IP address with one of the IP addresses of the pods associated with this service:
   
   
   
7. The request receives a valid IP address as the destination address and is normally processed:
   
   
   
8. Depending on the network topology, the request eventually reaches the hearth:
   
   
   

Are iptables able to balance the load?

1. Choose the first one with a probability of 33%, otherwise go to the next rule.
2. Choose the second one with a probability of 50%, otherwise go to the next rule.
3. Choose the third under.

Long-lived connections in Kubernetes do not scale by default

• Keep-alive in Node.js
• Keep-alive in spring boot
• Keep-alive in python
• Keep-alive in .NET

1. The first sub sends a request to the service:
   
   
   
2. You already know what will happen next. The service does not exist, but there are iptables rules that will handle the request:
   
   
   
3. One of the backend pods will be selected as the destination address:
   
   
   
4. The request reaches the hearth. At this point, a permanent TCP connection between the two pods will be established:
   
   
   
5. Any next request from the first pod will go through an already established connection:
   
   
   

1. Get the list of endpoints from the service.
2. For each endpoint, open a persistent connection.
3. When you need to make a request, use one of the open connections.
4. Regularly update the list of endpoints, create new ones, or close old persistent connections if the list changes.

1. Instead of sending the first request to the service, you can balance requests on the client side:
   
   
   
2. You need to write code that asks which pods are part of the service:
   
   
   
3. As soon as you receive the list, save it on the client side and use it to connect to the pods:
   
   
   
4. You yourself are responsible for the load balancing algorithm:
   
   
   

Client side load balancing

var mysql = require('mysql');
var poolCluster = mysql.createPoolCluster();

var endpoints = /* retrieve endpoints from the Service */

for (var [index, endpoint] of endpoints) {
  poolCluster.add(`mysql-replica-${index}`, endpoint);
}

// Make queries to the clustered MySQL database

• WebSockets and secured WebSockets
• HTTP / 2
• gRPC
• RSockets
• AMQP

Balancing long-lived connections in Kubernetes

1. Clusterip
2. NodePort
3. Loadbalancer
4. Headless

1. The management layer assigns it an IP address.
2. Kube-proxy forms the necessary iptables rules.

Service Mesh will help you

1. Gets a list of IP addresses from the service.
2. Opens and maintains a connection pool.
3. Periodically updates the pool, adding or removing endpoints.

1. Selects an available connection using some kind of logic (e.g. round-robin).
2. Fulfills the request.

1. Automatically searches for IP addresses of services.
2. Checks connections such as WebSockets and gRPC.
3. Balances requests using the correct protocol.

What happens if you ignore balancing issues?

Conclusion

1. Three levels of autoscaling in Kubernetes and how to use them effectively . 
2. Kubernetes in the spirit of piracy with an implementation template .
3. Kubernetes .