Get best of the week

Fine-tuning routing for MetalLB in L2 mode


Not so long ago, I was faced with a very non-standard task of configuring routing for MetalLB. All would be nothing, because MetalLB usually does not require any additional actions, but in our case there is a fairly large cluster with a very simple network configuration.


In this article I will tell you how to configure source-based and policy-based routing for the external network of your cluster.


I will not dwell on installing and configuring MetalLB in detail, as I assume you already have some experience. I propose to go straight to the point, namely, to configure routing. So we have four cases:


Case 1: When no configuration is required


Let's analyze a simple case.



An additional routing configuration is not required when the addresses issued by MetalLB are on the same subnet as the addresses of your nodes.


For example, you have a subnet 192.168.1.0/24, it has a router 192.168.1.1, and your nodes get addresses: 192.168.1.10-30then for MetalLB you can configure the range 192.168.1.100-120and be sure that they will work without any additional configuration.


Why is that? Because your nodes already have configured routes:


# ip route
default via 192.168.1.1 dev eth0 onlink 
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.10

And the addresses from the same ridge will reuse them without any additional gestures.


2:



, IP- MetalLB .


. MetalLB , :


ip addr add 10.9.8.7/32 dev lo

, :


  • a) /32 ( )
  • b) ( loopback). , Linux. , arp- arp- , , , Kubernetes.

, strict arp:


echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

arp- , IP-. MetalLB kube-proxy IPVS.


MetalLB arp-, user-space, MetalLB.


. , :


ip route add 10.9.8.0/24 dev eth1

3: source-based routing


Source-based routing , gateway, , gateway.


, 192.168.1.0/24 , MetalLB. , 1.2.3.0/24 VLAN 100, Kubernetes- .



1.2.3.4 1.2.3.0/24 . , MetalLB 1.2.3.4, 1.2.3.1, , 1.2.3.1.


default gateway 192.168.1.1, , 1.2.3.1, .


?


, . VLAN- :


ip link add link eth0 name eth0.100 type vlan id 100
ip link set eth0.100 up

:


ip route add 1.2.3.0/24 dev eth0.100 table 100
ip route add default via 1.2.3.1 table 100

100 1.2.3.1, eth0.100.


:


ip rule add from 1.2.3.0/24 lookup 100

: 1.2.3.0/24, 100. 1.2.3.1


4: policy-based routing


, 1.2.3.0/24 :



1.2.3.0/24, 1.2.3.0/24 eth0.100, - Kubernetes , .


, policy-based routing.


, , :


ip route add 1.2.3.0/24 dev eth0.100 table 100
ip route add default via 1.2.3.1 table 100

amarao

, - :


ip rule add from 1.2.3.0/24 lookup 100
ip rule add from 1.2.3.0/24 to 10.112.0.0/12 lookup main

:


  • 1.2.3.0/24 —
  • 10.112.0.0/12 — podNetwork

, .



netfilter:


iptables:


iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j RETURN
iptables -t mangle -A PREROUTING -i bond0.100 -j MARK --set-mark 0x100
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark

eth0.100, 0x100, .


:


ip rule add from 1.2.3.0/24 fwmark 0x100 lookup 100

1.2.3.0/24 0x100 100.


, , , Kubernetes.


, Linux reverse path filter, : - - , , .


, :


echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0.100/rp_filter

, rp_filter, , . rp_filter.


rp_filter netfilter. rpfilter iptables , :


iptables -t raw -A PREROUTING -i eth0.100 -d 1.2.3.0/24 -j RETURN
iptables -t raw -A PREROUTING -i eth0.100 -m rpfilter --invert -j DROP

rp_filter eth0.100 1.2.3.0/24.

More articles:


All Articles