Get best of the week

Traffic management in Kubernetes cluster with Calico


, DevOps, - . Kubernetes-, Calico . , k8s, Calico.


Calico — — IP-, , HTTP-, .


Calico GlobalNetworkPolicy. pod’, //pod’, , preDNAT, doNotTrack applyOnForward.


: pod’ . Kubernetes NetworkPolicies , api, Calico. GlobalNetworkPolicy. Zero Trust Networking.


Zero Trust Networking


, , , Kubernetes, Calico Zero Trust Networking, .


:


  1. ( , ).
  2. endpoint , . , , IP- , , .
  3. . , , .
  4. workload (pod/VM/container) .
  5. Zero Trust Networks . ( ), Zero Trust Networks .

, , : - , , . Policy, , , , .


HostEndpoint, , , , . , , , :


PortProtocolDirectionPurpose
22TCPInboundSSH access
53UDPOutboundDNS queries
67TCPOutboundDHCP access
68TCPInboundDHCP access
179TCPInbound & OutboundBGP access (Calico networking)
2379TCPInbound & Outboundetcd access
2380TCPInbound & Outboundetcd access
6666TCPInbound & Outboundetcd self-hosted service access
6667TCPInbound & Outboundetcd self-hosted service access

, Felix. Felix — Calico, , , Calico.


Calico


Calico. , , , Calico — , , , Calico , Kubernetes. ( ) .


Felix


— Felix, , , , endpoint . :


  • , , .
  • FIB (Forwarding Information Base) Linux.
  • ACL Linux.
  • , Felix etcd.


, (, OpenStack, Kubernetes) Calico . Kubernetes — CNI plugin.


etcd


Calico etcd. etcd — -, Calico. , Kubernetes, Calico etcd.


BGP (BIRD)


Calico BGP , Felix. BGP , , Felix , , .


BGP (BIRD)


, Calico BGP , , (N ^ 2). , BIRD. , , BGP - , BGP-.


Network Policy


NetworkPolicy, Calico ( api projectcalico.org) , , Kubernetes ( api networking.k8s.io).


NetworkPolicy , endpoints, (labels). :


apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-tcp-6379
  namespace: production
spec:
  selector: role == 'database'
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: role == 'frontend'
    destination:
      ports:
      - 6379
  egress:
  - action: Allow

/, role == 'frontend' 6379 .


Host Endpoint


() , Calico. HostEndpoint. (labels) pod’, HostEndpoint, endpoints pod’.


, , 22, 80, 443 . , , HostEndpoint. :


apiVersion: projectcalico.org/v3
kind: HostEndpoint
metadata:
  name: node4-ens160
  labels:
    type: production
role: worker
        node: 4
spec:
  interfaceName: ens160
  node: k8s-s4
  expectedIPs:
  - 10.213.0.11
ports:
 - name: http
   port: 80
   protocol: TCP
 - name: https
   port: 443
   protocol: TCP

ports, . , (http, https), . interfaceName — IP- expectedIPs. 22 , Felix.


Global Network Policy


, , GlobalNetworkPolicy, , 80/443 HostEndpoint:


kind: GlobalNetworkPolicy
apiVersion: projectcalico.org/v3
metadata:
  name: allow-s4
spec:
  selector: role==worker
  order: 10
  applyOnForward: true
  types:
  - Egress
  - Ingress
  ingress:
  - action: Allow
    protocol: TCP
    source:
      nets:
            - 10.213.0.0/24
  - action: Allow
    protocol: TCP
    destination:
      ports: [http,https]
  - action: Allow
    protocol: ICMP
  egress:
  - action: Allow

, GlobalNetworkPolicy NetworkPolicy . order — , , .


, GlobalNetworkPolicy , , : preDNAT, doNotTrack applyOnForward, .


applyOnForward, preDNAT doNotTrack


, , . Bikram Gupta.


applyOnForward


applyOnForward , , iptabels FORWARD. , , pod’. pod ( ), , PREROUTING – FORWARD – POSTROUTING.


applyOnForward false, GlobalNetworkPolicy workload (/pod/, ) . , . .


applyOnForward true, GlobalNetworkPolicy (forwarded) , :


  • , HostEndpoint workload.
  • workload, HostEndpoint.
  • , HostEndpoint HostEndpoint.

applyOnForward false. , doNotTrack preDNAT, applyOnForward true, , FORWARD.


(FORWARD), - . : HostEndpoint, applyOnForward: true HostEndpoint , . applyOnForward:true, HostEndpoint , , .


: GlobalNetworkPolicy, ICMP HostEndpoint applyOnForward:false. , ping 8.8.8.8 pod’ , , forwarded , GNP . . HostEndpoint, applyOnForward:true, , TCP-, ping 8.8.8.8 . applyOnForward:true ( HostEndpoint) .


preDNAT


, DNAT (Destination Network Address Translation) .


, , NodePorts , . , NodePort DNAT (kube-proxy). , , , NodePort, preDNAT true.


:


  • preDNAT , .
  • , host endpoint, , workload (pod/VM/container).
  • preDNAT . HostEndpoint, preDNAT , “ ”.

doNotTrack


doNotTrack (conntrack) , , ( pod / VM / container).


conntrack — Linux, , , .
, . , :


  • , conntrack ( 128k ).
  • . conntrack ( 120). , conntrack 128 , 1100 , , (128k / 120s = 1092 connections/s ).

, , memcached, . Calico , , , doNotTrack. ,


doNotTrack OUTPUT PREROUTING, (, doNotTrack:false), order. doNotTrack order:1, doNotTrack order:1000, doNotTrack. rder .



, , Calico Kubernetes- .


:



:


More articles:


All Articles