Data Centre Design IPv6 Microsegmentation
The source for this topic is taken from Ivan Pepelnjak IPv6 micro-segmentation webinar.
Layer-2 security mechanisms for IPv6 are still as complicated as they are with IPv4. Nothing has changed. We are still building the foundation of our IPv6 and IPv4 networks on the same forwarding paradigm; relying on old technologies that emulates thick coaxial cable, known as Ethernet. Ethernet should be limited to where Ethernet was designed: data link layer between adjacent devices. Unfortunately, IP+Ethernet mentality is tightly coupled together with every engineers mind.
What is Layer 2 ? And why do we need it?
Layer 2 is the layer that allows adjacent network devices to exchange frames. Every layer 2 technology has at least 3 components a) Start-of-frame indication, b) End-of-frame indication, c ) Error correction mechanism in case the physical layer cannot guarantee error-free transmission of zeroes and ones.
You may have realized that I haven’t mentioned layer 2 MAC address as a required component?
MAC addresses are required when you have more than two devices attached to the same physical network. MAC addresses are in Ethernet frames because the original Ethernet standard used coax cable with multiple nodes attached to the same physical medium. Layer 2 addressing on point-to-point Fiber Channel networks is not required while you do need layer 2 addressing on shared cable-based Ethernet networks. One of the main reasons for the continuation of MAC addresses in Ethernet frames is for backward compatibility. More importantly, no one wants to change device drivers in every host deployed in a data center or Internet for that matter.
The first time you need unique addresses is Layer-3
“IPv6 microsegmentation is an approach used to solve security challenges in IPv6”
IPv6 has many layer-2 security challenges. Similar to IPv4 world, the assumption is one subnet is one security zone. This can be represented as a traditional VLAN with a corresponding VLAN ID or more recent technology of VXLAN with a corresponding VXLAN ID. Devices in that domain are in one security domain and all enjoy the same level of trust, representing a number of IPv6 security challenges. If an intruder breaks into that segment they can take advantage of that implicit trust between all devices in that particular segment. The main disadvantage is intra-subnet communication is not secured and multiple IPv6 first-hop vulnerabilities ( RA and NA spoofing, DHCPv6 spoofing, DAD DoS attack and ND DoS attacks) exists.
the attacker can spoof the neighbour advertisement messages and affect ND cache on host. Thus, take over and intercept traffic sent to other hosts. It can also intercept DHCP request and pretend to be DHCP server, redirecting traffic to itself or DoS attack with incorrect DNS records. The root cause is that everything we operate on today is simulating the thick coaxial cable we used for Ethernet. In the early days, Ethernet segments were with one segment of coaxial cable and all stations could attach to this segment resulting in one large security domain. Networks evolved, new technologies were introduced. The coaxial cable was later replaced with thin cable and; then, hubs to switches. Unfortunately, we haven’t changed the same basic forwarding paradigm we used 40 years ago. We are still emulating thick coaxial cable while relying on same traditional basic forwarding paradigm. The networking industry trying to fix the problem without addressing and fixing the actual source of the problem. The networking industry is retaining existing forwarding paradigm while implementing layer-2 security mechanism to overcome its limitations. All these layer-2 security measures ( first hop security ) are leading to complex networks both from design and operational aspect. They are adding more kludges; hence, every technology trying to fix the shortcomings when they should be addressing the actual source of the problem.
In layer 2 world, everyone is trying to retain existing forwarding paradigm, even with most recent data centre overlay technologies. For example, they are still trying to emulate the thick coaxial cable over VXLAN segment over IP. VXLAN uses historic flooding behavior. In IPv6 world to overcome shortcomings with layer 2; vendors started to implement a list of first-hop layer-2 security mechanisms. You need to implement all of these to secure layer 2 IPv6 domain. All these features are complicated technologies to implement and they are used solely to fix the broken forwarding paradigm of layer 2. Recent issues with MLD ( multicast listener discovery protocol ) which is part of IPv6. MLD can break into multicast streams on Local Area Networks ( LAN ) and gain control of first hop router communication. So, in the future we will now need to implement MLD guard as a new first hop security mechanism. The list goes on and on. A constant cat and mouse game. So, we need to ask ourselves can we do better than that? And what can we implement or design to overcome these shortcomings? Just get rid of layer 2? 🙂 We can simply remove layer 2 from “some” networks. If the first hop router is an actual layer 3 device then we don’t need to implement all the security kludges mentioned above. We would still need Ethernet between end host and router as end hosts have Ethernet cards. Using a layer 3 device as the first hop, we immediately remove all IPv6 spoofing attacks. For example, RA Gaurd is not needed as the router will not listen to RA messages. ND spoofing is impossible as you can’t bridge ND across segments. However, DoS attacks are still possible. This type of layer 3 only design is implemented on xDSL and Mobile networks. Design by putting every host in a /64 subnet. But; now, we are going back to 64-bit segments in order to implement security between segments.
Is this possible to use in the data centre when moving VM’s across mobility domains?
Microsegmentation for Data Centre
In data centres, we have issues with live VM migration. We have to move VMs between servers while retaining IPv6 address so that all Transmission Control Protocol ( TCP ) sessions stay intact. Layer 3 solutions exist but they are much slower ( as layer 3 routing protocol convergence is slower than layer 2 convergence ) than what we can get with simple flooding of MAC address with reverse Address Resolution Protocol ( ARP ) and gratuitous ARP. We usually have some VLAN that spans the domain with a real VLAN or VXLAN segment. VLANs must span the entire mobility domain, expanding the broadcast domain throughout the network. Expanding the broadcast domain also expands the scope of layer 2-security attacks. Private VLANs exists but on a large-scale private VLANs are messy and complex. You can use one VLAN per VM but this would cause an explosion of VLAN numbers. You still need to terminate layer 3 on Core switches which means all traffic between two VM has to traverse to Core. Inter-VLAN communication is sent to Core ( layer 3 device ). Even when VM are sitting on the same hypervisor. Clearly not a good design.
Also, if you want mobility across more than one core switch, then you can’t aggregate traffic and you have to pass the IPv6 prefixes to support VM mobility. Now, we have loads of /64 prefix in IPv6 forwarding table when using one prefix per VM. Vendors like Brocade only support 3k IPv6 prefixes and Juniper support up to 1k. In the future, this scale limitation will represent design problems. So, we need some other type of design? We need to change the forwarding paradigm. In an ideal world, use layer 3 only networks, layer-3 device as first hop device and still support VM mobility. At the same time not generate not than many IPv6 prefixes.
Intra-Subnet ( Host Route ) Layer 3 forwarding
Is it possible to design and build layer-3 only IPv6 networks without assigning a /64 prefix to every host?
Intra-subnet layer 3 forwarding implements /128 for hosts and this is propagated with updates across the network. At a host level, nothing changes. It can use DHCP or whatever other mechanisms to get its address. Now, that we are using /128 so we don’t need to use IPv6 forwarding table for this prefix. We can put the /128 into IPv6 Neighbor Discovery ( ND ) entries. This is how the ND cache be implement on hardware-based platforms. No difference between ND entities and /128-host routes in the IPv6 routing table. The Important point here is that you can use ND entries instead of IPv6 forwarding table, which by default in most platforms have small table sizes. Juniper EX series can have 32k ND entities but only 1K IPv6 entries. With this design trick we can significantly increase the number of hosts we can have under anIPv6 microsegmentation design.
IPv6 Microsegmetation and Cisco Dynamic Fabric Automation ( DFA )
Virtual Machine microsegmentation with Cisco DFA allows you to implement VLAN per VM addressing scheme without the worry of VLAN sprawl and all those problems experienced with provisioning. More importantly, all layer 3 traffic is not terminated on core switch, it is terminated on the leaf switch.