Kubernetes pod shutdown delay
This is the third part of our path (approx. Lane - link to the first article) to achieve zero downtime when updating a Kubernetes cluster. In the second part, we reduced the downtime that arose due to the forced termination of applications running in pods, terminating them correctly using lifecycle hooks. However, we also learned that pod can continue to receive traffic after the application in it has started shutting down. That is, the client may receive an error, because its request will be directed to the pod, which can no longer serve the traffic. Ideally, we would like pods to stop accepting traffic immediately after the start of the eviction. To reduce the risk of downtime, we first need to understand why this is happening.
“Kubernetes in Action”. . , , Kubernetes, .
pod’
lifecycle pod’. , — pod’, , pod’ . , , pod endpoint Service .
pod’ Service? , pod’ .
pod API, — . pod’ , :
kubelet , .kube-proxy IP pod’ iptables .- Endpoints controller pod endpoints, pod
Service
. , , . , pod preStop TERM , pod . pod , .
, , pod , . , - Kuberenetes. , ? ? , online? , ? ?
, , 0. , , 99% . sleep preStop, . .
, preStop. “Kubernetes in Action”, Lukša 5-10 , 5 :
lifecycle:
preStop:
exec:
command: [
"sh", "-c",
# Introduce a delay to the shutdown sequence to wait for the
# pod eviction event to propagate. Then, gracefully shutdown
# nginx.
"sleep 5 && /usr/sbin/nginx -s quit",
]
. , kubectl drain, pod’ . event, pod’ kubelet Endpoint Controller ( Service endpoints) . , , preStop pod.
Drain node pod, event
preStop , . Endpoint Controller pod:
Pod ,
, pod , , . , - pod’, , pod. , , event , .
, , preStop sleep pod Nginx, pod :
Node 1, . , , .
pod’
, , pod’, . pod’, , pod’? Deployment .
Deployment controller, . resource config, pod’ . Deployment pod’, pod’. :
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.15
ports:
- containerPort: 80
, pod’ Deployment app: nginx pod’ nginx:1.15, 80 .
pod’, spec Deployment, , :
spec:
replicas: 2
, 2 pod’ . , pod’ , , pod. , , pod’ drain, Deployment .
, preStop , pod’ . Deployment pod’. ?
drain , , pod’. , stateful-, , , pod’ , .
drain , pod’, . , pod’ , , drain, pod’.
, Kubernetes PodDisruptionBudgets, pod, . , drain, drain .
Kubernetes AWS , Gruntwork.io.
: