Get best of the week

Networking in Flexiant Cloud Orchestrator

In the continuation of the series of articles on the cloud orchestra, Flexiant would like to talk about how the organization of the network for virtual machines and containers is being managed under this orchestra.

Flexiant identifies four types of networks that can be accessed by the client in his virtual machine.

PVIP is a network in which a single / 32 address is allocated to a client. Routing to this IP address is performed on the host machine on which the
Public IP virtual machine is located - the network in which the client receives a separate VLAN and / 29 subnet. This is the main network that we use. Routing for it is performed on special router nodes that are managed by Flexiant Cloud Orchestrator.

Private IP - network, organization similar to the previous one, except that it is not displayed externally on router nodes. A client can use it to create internal networks between their virtual machines. IP addresses can be assigned both from the interface and on the machine itself.

Interworking VLAN is, again, very similar to Private IP, except that it is not available to the client directly without the participation of the provider. We give this network to the client when he needs to connect the existing L2 infrastructure to the Flexiant Cloud Orchestrator infrastructure.

All networks except the first operate in a separate isolated VLAN; accordingly, all L2 protocols are available to the client: arp, dhcp, lldp, etc. The PVIP network runs on L3, and we use it for clients who want to get protection from DDoS. If they want to protect IP addresses, then they need to use it.

Each network is assigned to a virtual machine adapter. Thus, in one virtual machine there can be several types of networks at the same time.

IP addresses for Private IP and Interworking VLAN can be chosen by clients arbitrarily. IP addresses for Public IP and PVIP are selected from the list in the control panel.

image

As soon as the adapter receives the network, and optionally an IP address, information about this gets into the orchestra’s database, and the corresponding settings are entered on the routing nodes (for the Public IP network), computing nodes (for PVIP), as well as in the DHCP server located on the management node, in case an IP address has been set. The correspondence table between the virtual mac-address of the card and the selected IP-address is entered into the PgSQL database, from where it is further taken by the DHCP server.

On each node, whether it is a router node or a compute node, the Flexiant agent is installed, which receives commands from the orchestra. Inside the nodes themselves, a virtual router is raised in a separate network namespace. Thus, the Linux network stack on the nodes and the virtual router network stack are independent of each other.

Router-Nodes that ensure the functioning of Public IP duplicate each other using the VRRP protocol in the implementation of carp. If one of the nodes fails, the traffic will go through the other.

Here is an example of routing on the route node for one:

Router0# ip a l
…
766: VLAN367: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 00:1e:67:d3:67:5c brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e0fe:25ff:fef9:7304/64 scope link
     valid_lft forever preferred_lft forever
…
Router0# brctl show VLAN367
bridge name     bridge id               STP enabled     interfaces
VLAN367         8000.001e67d3675c       no              bond0.367
                                                                               evrl-000190

#         ,     shell  screen,       

Router0# evrs
Router0# ip a l evrr-000190
768: evrr-000190: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether e6:52:0f:21:83:b0 brd ff:ff:ff:ff:ff:ff
    inet 217.23.xxx.xxx/29 scope global evrr-000190
       valid_lft forever preferred_lft forever
    inet6 fe80::e452:fff:fe21:83b0/64 scope link
       valid_lft forever preferred_lft forever

Router0# ip r l 217.23.xxx.xxx
217.23.xxx.xxx/29 dev evrr-000190  proto kernel  scope link  src 217.23.xxx.xxx
Router#0 ip r l ip r l 0.0.0.0/0
default via 10.158.192.3 dev evrr-000000  proto bird

A pair of evrr / evrl interfaces are regular veths on Linux that interconnect two network namespaces. Thus, it is seen how incoming traffic getting into the virtual router is routed there to a specific interface.

Firewall management also takes place in a virtual router. Separate tables and chains are created that allow or do not allow traffic to a particular network interface.

Router0# iptables -nvL evrr-000190-4i
Chain evrr-000190-4i (1 references)
 pkts bytes target     prot opt in     out     source               destination
 435M  124G RETURN     all  --  *      *       217.23.xxx.xxx        0.0.0.0/0
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0
Router0# iptables -nvL evrr-000190-4o
Chain evrr-000190-4o (1 references)
pkts bytes target     prot opt in     out     source               destination
587M  782G ACCEPT     all  --  *      *       0.0.0.0/0            217.23.xxx.xxx
6038K  350M DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

We use OSPF as the routing protocol, the configuration with compute and the router node gets to the core of the network, and from there to the core routers. As software on the nodes, bird is used, which is controlled by Flexiant. We also make changes to configuration templates, since Flexiant allows us to do this. Flexiant also allows you to use statics and bgp as routing protocols.

In terms of payment, all networks are free. Payment is only for IP addresses for PVIP and Public IP networks. However, IP addresses for PVIP are more expensive because they are protected from DDoS attacks.

More articles:


All Articles