Carriers Based on Open Ethernet with SONiC
At a recent Nvidia GPU Technology Conference (GTC) session, specialists from Nvidia and Comcast summarized how and where the open-source NOS can fit in to sustain 400 gigabit Ethernet deployment. Comcast discusses leveraging NVIDIA’s open Ethernet networking technologies in this session for their Cloud Fabric services deployment.
Comcast uses the open Ethernet approach to build its data centers and carrier-neutral facilities. These deployments consist of SONiC-enabled Nvidia SN4700 networking switches that support 32 ports of 400G.
One can consider this type of network transformation as moving from a static and conservative mindset that results in cost overrun and inefficiencies to a dynamic routed environment. Now we can build more stable data centers based on an underlay and overlay technology with the stability of Layer 3 routing.
A key point: Additional technical information
The NVIDIA® Spectrum®-3 based SN4000 is an open ethernet switch supporting all speeds ranging from 1GbE through 400GbE, SN4000. This switch is the perfect building block for building large layer-2, layer-3, and virtualized network fabrics for greenfield and brownfield environments.
Core technical highlights include:
- Max Port Speed 400Gb/s
- Max Flow Counters 512K entries
- Max Access-Control Lists (ACLs) 512K entries
- Network Address Translation (NAT) 100K+ entries
- Max IPV4 Routes 512K entries
Additional Networking capabilities:
- Visibility: NVIDIA Spectrum SN4000 switches support detailed and contextual telemetry with NVIDIA What Just Happened® (WJH). Spectrum switches implement hardware-accelerated histograms to track and summarize queue depths at a sub-microsecond granularity. Hardware-accelerated histograms avoid false alerts common to simple watermarks and threshold-based methods.
- Performance: NVIDIA Spectrum SN4000 switches feature a fully shared packet and monolithic buffer architecture. This unique buffer architecture delivers a fair, predictable, high-performance data path essential for scaling out software-defined storage and modern multi-tenant cloud deployments. Spectrum switches have a robust high bandwidth and a low latency data path for remote direct-memory access over converged Ethernet (RoCE) and machine learning applications that leverage GPUDdirect®.
- Scale: NVIDIA Spectrum SN4000 switches support best-in-class Virtual Extensible LAN (VXLAN) with the backing for 10X more tunnels and tunnel endpoints. Its 512K shared forwarding entries can be flexibly used across access-control lists (ACL), longest prefix match (LPM) routes, host routes, MAC tables, and equal-cost multi-path (ECMP) and tunnel applications.
These features allow network and data center managers to design and implement a cost-effective switch fabric based on the “pay-as-you-grow” principle. Therefore, a material consisting of a few servers can slowly grow to include hundreds or thousands of servers. The Spectrum Ethernet switch family is powered by the Spectrum application-specific integrated circuit (ASIC). As a result, spectrum switches feature dynamic, flexible shared buffers and predictable wire-speed performance.
The SN2000, SN3000, and SN4000 series are available in three versions:
- Pre Installed with NVIDIA Cumulus® Linux, NVIDIA’s flagship network operating system that takes the Linux user experience from servers to switches and provides a rich routing and automation functionality for large-scale applications
- Bare-metal switches, including an Open Network Install Environment (ONIE) image ready to be installed with the SONiC or another ONIE-mounted operating system
- Pre Installed with NVIDIA Onyx™, a home-grown operating system utilizing an industry-standard command-line interface (CLI)
One significant advantage is that leaf and spine architectures can quickly scale up to 10K+ nodes in two tiers.
SONiC: Open-Source Network Operating System
SONiC is an open-source network operating system based on Linux that works on switches from multiple vendors and ASICs. In addition, SONiC offers a full suite of network functionality, like Border Gateway Protocol (BGP).
SONiC core features:
- Decouples Hardware & Software: SONiC is built on Switch Abstraction Interface that helps accelerate hardware innovation.
- Accelerates Software Evolution: First solution to break monolithic switch software into multiple containerized components that accelerate software evolution.
- Rapidly Growing Ecosystem: SONiC has gained broad industry support over the last year, including significant network chip vendors.
SONiC System Architecture
SONiC system’s architecture comprises various modules that interact with each other through a centralized and scalable infrastructure. This infrastructure relies on a redis-database engine: a key-value database to provide a language-independent interface, a method for data persistence, replication, and multi-process communication among all SONiC subsystems.
By relying on the publisher/subscriber messaging paradigm offered by the redis-engine infrastructure, applications can subscribe only to the data views they require and avoid implementation details irrelevant to their functionality. SONiC positions each module in independent docker containers to maintain cohesion among semantically-affine components while reducing coupling between disjointed ones. Each piece is written to separate from the platform-specific details required to interact with lower-layer abstractions.
SONiC is the leading open-source community network operating system for cloud data centers. David Iles, senior director of Ethernet Switching at Nvidia, said. “It’s modular, containerized, and built from the ground up to be scalable for large network deployments.”
- A key point: Final comments on SONiC
SONiC was initially built by Microsoft and formally became an open-source project in 2015 when it joined the Open Compute Project (OCP). In 2022, the project moved to its current home at the Linux Foundation.
- Fortinet’s new FortiOS 7.4 enhances SASE - April 5, 2023
- Comcast SD-WAN Expansion to SMBs - April 4, 2023
- Cisco CloudLock - April 4, 2023