Network Traffic Engineering

Network Traffic Engineering

Network Traffic Engineering

In today's interconnected world, network traffic engineering plays a crucial role in optimizing the performance and efficiency of computer networks. This blog post aims to provide a comprehensive overview of network traffic engineering, its importance, and the techniques used to manage and control traffic flow.

Network traffic engineering efficiently manages and controls the flow of data packets within a computer network. It involves analyzing network traffic patterns, predicting future demands, and implementing strategies to ensure smooth data transmission.

Table of Contents

Highlights: Network Traffic Engineering

Network Flow Model

In a computer network, an important function is to carry traffic efficiently, given the routing paradigm in place. This efficiency is achieved through traffic engineering. Network flow models are used for network traffic engineering and can help determine routing decisions. Network Traffic engineering (TE) is the engineering of paths that can carry traffic flows that vary from those chosen automatically by the routing protocol(s) used in that network.

The Role of MPLS Networking

Therefore, we can engineer the paths that better suit our application. We can do this in several ways, such as standard IP routing, MPLS, or OpenFlow protocol. When considering network traffic engineering and MPLS OpenFlow, let’s start with the basics of traffic engineering and MPLS networking.

Related: You may find the following posts helpful for pre-information:

  1. Transport SDN
  2. Network Visibility
  3. Load Balancing
  4. Chaos Engineering Kubernetes
  5. Segment Routing
  6. What is OpenFlow
  7. DMVPN Phases



MPLS OpenFlow.

Key Network Traffic Engineering  Discussion Points:


  • Introduction to network traffic engineering and what is involved.

  • Highlighting the different components of an MPLS network and how they work.

  • MPLS traffic engineering (TE).

  • Controller-based networking and its advantages.

  • Discussing a predictable Traffic Engineering solution.

Back to Basics: Network traffic engineering

Traffic Engineering

Main Traffic Engineering Components

Network Traffic Engineering

  • Optimal Resource Utilization

  • Enhanced Quality of Service (QoS)

  • Scalability

  • Example: MPLS TE

Importance of Network Traffic Engineering

Efficient network traffic engineering is essential for several reasons:

1. Optimal Resource Utilization: By balancing network resources, traffic engineering helps minimize congestion and maximize bandwidth utilization, improving network performance.

2. Enhanced Quality of Service (QoS): Traffic engineering techniques prioritize critical applications, ensuring they receive the necessary bandwidth and reduce latency, improving user experience and customer satisfaction.

3. Scalability: With proper traffic engineering, networks can accommodate increased traffic demands, scalability, and future growth without significant performance degradation.

Techniques Used in Network Traffic Engineering

Here are some commonly used techniques in network traffic engineering:

1. Traffic Monitoring and Analysis: Network administrators employ tools to monitor and analyze traffic patterns, helping them identify bottlenecks, congestion points, and potential network vulnerabilities.

2. Traffic Shaping: Traffic shaping involves regulating network traffic flow to optimize performance. It can prioritize certain types of traffic, delay less critical traffic, and prevent data bursts that may overload network resources.

3. Load Balancing: Load balancing distributes network traffic across multiple paths or devices, preventing congestion and ensuring efficient use of available resources.

4. Quality of Service (QoS): QoS mechanisms prioritize specific types of traffic, ensuring that critical applications receive the necessary resources and reduce latency.

5. Traffic Engineering Protocols: Network engineers utilize RSVP (Resource Reservation Protocol) and MPLS (Multiprotocol Label Switching) to manage network traffic and allocate resources effectively and dynamically.

Lab Guide: MPLS Forwarding

In the following guide, we have an MPLS network. MPLS networks have devices with different roles. So, we have the core node called the “P” provider and the “PE” provider edge nodes. The beauty of MPLS forwarding is that we can have scale in the network’s core. The P nodes do not need customer routes from the CE devices. These are usually carried out in BGP.

Note:

However, with an MPLS network, we have MPLS forwarding between the loopbacks. Notice the diagram below. The loopback address of 2.2.2.2/32 and 4.4.4.4/32 belong to the PE nodes. The P node is entirely unaware of any BGP routing.

MPLS overlay
Diagram: MPLS Overlay

Traffic Engineering: Inbound and Outbound

Before you can understand how to use MPLS to do traffic engineering, you must understand what traffic engineering is. So, we have network engineering that manipulates your network to suit your traffic. You make the most reasonable predictions about how traffic will flow across your network and then order the right components.

Then we have traffic engineering. Network traffic engineering is manipulating your traffic to fit your network. Traffic engineering is not MPLS-specific and is a general practice among all networking and security technologies. Traffic engineering could be a simple or complex implementation. Something as simple as tweaking IP metrics on the interface can be implemented in its simplest for traffic engineering. Then, we have traffic engineering specific to MPLS.

Network traffic engineering
Diagram: Network Traffic Engineering. Source AWS

Lab Guide: MPLS TE

In this lab, we will look at MPLS TE with ISIS configuration. Routers PE1, P1, P2, P3, and PE2 are our MPLS core network. The CE1 and CE2 routers use regular IP routing. All routers are configured to use IS-IS L2. 

Tip: There are four main items we have to configure:

  • Enable MPLS TE support:
    • Globally
    • Interfaces
  • Configure IS-IS to support MPLS TE.
  • Configure RSVP.
  • Configure a tunnel interface.
MPLS TE
Diagram: MPLS TE

Understanding MPLS and MPLS forwarding

MPLS is the de facto technology for service provider WAN networks. Its scalable architecture moves complexity and decision-making to the network’s edges, leaving the core to label switch packets efficiently. The PE nodes sit at the edge and perform path calculations and encapsulations. The P nodes sit in the core and label switch packets. They only perform MPLS switching and have no visibility of customer routes.

Edge MPLS routers map incoming packets into forwarding equivalence classes (FEC) and use a different label-switched path (LSP) for each forwarding class. Keeping the network core simple enables scalable network designs. Many of today’s control planes encompass a distributed architecture and can make forwarding decisions independently.

MPLS control plane still needs a distributed IGP (OSPF and ISIS) to run in the core and a distributed label allocation protocol (LDP) to label packets. Still, it shifted how we think of control planes and distributed architectures. MPLS reduced the challenges of some early control plane approaches but proposes challenges by not having central visibility, especially for traffic engineering (TE).

MPLS forwarding
Diagram: MPLS Forwarding. The source is NetworkInterview.

Example Technology: DMVPN Phase 3 Traffic Manipulation

DMVPN Phase 3 is the third and final phase of a Dynamic Multipoint Virtual Private Network DMVPN setup. This phase is focused on implementing the DMVPN tunnel and enabling dynamic routing. The tunnel is built between multiple network points, allowing communication between them.

In DMVPN Phase 1, the spoke devices rely on the configured tunnel destination to identify where to send the encapsulated packets. Phase 3 DMVPN uses mGRE tunnels and depends on NHRP redirect and resolution request messages to determine the NBMA addresses for destination networks.

Packets flow through the hub in a traditional hub-and-spoke manner until the spoke-to-spoke tunnel has been established in both directions. Then, as packets flow across the hub, the hub engages NHRP redirection to find a more optimal path with spoke-to-spoke tunnels.

NHRP Routing Table Manipulation

NHRP tightly interacts with the routing/forwarding tables and installs or modifies routes in the Routing Information Base (RIB), also known as the routing table, as necessary. If an entry exists with an exact match for the network and prefix length, NHRP overrides the existing next hop with a shortcut. The original protocol is still responsible for the prefix, but the percent sign (%) indicates overwritten next-hop addresses in the routing table.

DMVPN Phase 3
Diagram: DMVPN Phase 3 configuration

Lab Guide: DMVPN Phase 3

The following example shows DMVPN Phase 3 running on the network.

DMVPN Phase 3 is the latest iteration of the DMVPN technology, offering enhanced scalability and flexibility compared to its predecessors. It builds upon the foundation of Phase 1 and Phase 2, incorporating improvements that address the limitations of these earlier versions.

One of the critical features of DMVPN Phase 3 is the addition of a hub-and-spoke network topology. This allows for a centralized hub connecting multiple remote spokes, creating a dynamic and efficient network infrastructure. The hub is a central point for all spokes, enabling secure communication. In our case below, R11 is the hub, and R31 and R41 are the spokes.

Note:

Once the hub site receives traffic indicating spoke to spoke traffic, it sends back a “Traffic Indication” message. Notice the output from the debug command below. Via NHRP, the spoke knows a better path to reach the other spoke, not via the hub. The spoke then proceeds to build spoke-to-spoke tunnels.

DMVPN Phase 3
Diagram: DMVPN Phase 3 configuration

Network Traffic Engineering and MPLS 

MPLS was very successful, and significant service provider networks could support many customers by employing MPLS-style architecture. End-to-end Label Switch Paths (LSP) are extended to interconnect multiple MPLS service providers, route reflectors, and BGP confederations for large-scale deployments and complexity reduction.

However, no matter how scalable the MPLS architecture could be, you can’t escape the fact that Inter-DC circuit upgrades are time-consuming and expensive. To help alleviate this, MPLS providers introduced MPLS Traffic engineering (TE). TE moves traffic to other parts of the network to underutilized sections.

While simple TE can be done with IGP metrics, they don’t satisfy unique traffic class requirements. Therefore, provider networks commonly deploy MPLS RSVP/ TE. This type of TE enhances IGP metric tuning, allowing engineers to forward core traffic over non-shortest paths. The non-shorted path is used to avoid network “hot spots.” Since the traffic is now moved to other underutilized network parts, it prevents the lengthy process of upgrading congested core links. MPLS TE distributes traffic optimally across a network. “MPLS RSVP/ TE is a widely adopted and well-defined technology. Can SDN and OpenFlow do a better job?”

Network Traffic Engineering
Diagram: Network traffic engineering.

Holistic visibility – Controller-based networking

MPLS/TE is a distributed architecture. There is no real-time global view of the end-to-end network path. The lack of a global view may induce incorrect traffic engineering decisions, lack of predictability, and deterministic scheduling of LSPs.

Some tools work with MPLS TE to create a holistic view, but they are usually expensive and do not offer a “real-time” picture. They often make an offline topology. They also don’t change the fact that MPLS is a distributed architecture.

The significant advantage of a centralized SDN and OpenFlow framework, commonly called MPLS OpenFlow, is that you have a holistic view of the network, controller-based networking. The centralized software sits on the controllers, analyzing and controlling the production network forwarding paths. It has a real-time network view and gains insights into various network analytics about link congestion, delay, latency, drops, and other performance metrics.

mpls openflow
Diagram: MPLS OpenFlow

MPLS OpenFlow can push down rules to the nodes per-flow basis, offering a granular approach to TE. Per-flow TE state is challenging to achieve with the traditional TE mechanism. OpenFlow’s finer granularity is also evident in service insertion use casesIn addition, OpenFlow 1.4 supports better statistics that give you visibility into application performance.

This metric and a central viewpoint can only enhance traffic engineering decisions. Let’s face it: MPLS RSVP/TE, while widely deployed, involves several control plane protocols. All these protocols need to interact and work together.

The OpenFlow MPLS protocol steers traffic over MPLS using OpenFlow.

You can direct traffic from OpenFlow networks over MPLS LSP tunnel cross-connects and logical tunnel interfaces over MPLS networks. By stitching OpenFlow interfaces to MPLS label-switched paths (LSPs), you can direct OpenFlow traffic onto MPLS networks. In addition, through MPLS LSP tunnel cross-connects between interfaces and LSPs, you can connect the OpenFlow network to a remote network by creating MPLS tunnels that use LSPs as conduits.

MPLS OpenFlow
Diagram: MPLS OpenFlow. The source is Juniper.

Network state vs. Centralized end-to-end visibility

RSVP requires that some state is stored on the Label Switch Router (LSR). The state is always bad for a network and imposes control plane scalability concerns. The network state is also a target for attack. Hierarchical RSVP was established to combat the state problem, but in my opinion, it adds to network complexity. All these kludges become an operational nightmare and require skilled staff to design, implement, and troubleshoot.

Removing MPLS signaling protocols from the network and the state they need to maintain eliminates some of the scale concerns with MPLS TE. Distributed control planes must maintain many tables and neighbor relationships (LSDB and TED). They all add to network complexity.

Predictable and deterministic TE solution

Using SDN and OpenFlow for traffic engineering provides a more predictable and deterministic TE solution. Informing the OpenFlow controller that you want the traffic redirected toward a specific MAC address, the necessary forwarding entries are programmed and automatically appear across the path. There are possibilities with NETCONF and MPLS-TP, but they operationally cause problems and don’t alleviate the distributed signaling protocols.

Having a central controller view, the contents of the network allow for particular network touchpoints. New features are implemented in the software and pushed down to the individual nodes. Similar to all SDN architectures, fewer network touchpoints increase network agility. The box-by-box and manual culture is slowly disappearing.

Challenges and Future Trends

Network traffic engineering faces several challenges, including ever-increasing data volumes, evolving network architectures, and the rise of new technologies such as cloud computing and the Internet of Things (IoT). However, emerging trends like Software-Defined Networking (SDN) and Artificial Intelligence (AI) are promising to address these challenges and optimize network traffic.

 

Summary: Network Traffic Engineering

Understanding Network Traffic Engineering

Network traffic engineering analyzes and manipulates traffic to enhance performance and meet specific objectives. It involves various techniques such as traffic shaping, route optimization, and load balancing. By intelligently managing the flow of data packets, network administrators can ensure optimal utilization of available bandwidth and minimize latency issues.

Traffic Engineering Techniques

Traffic Shaping

Traffic shaping is a technique used to control network traffic flow by enforcing predetermined bandwidth limits. It allows administrators to prioritize critical applications or services, ensuring smooth operation during peak traffic hours. By regulating the rate at which data packets are transmitted, traffic shaping helps prevent congestion and maintain a consistent user experience.

Route Optimization

Route optimization focuses on selecting the most efficient paths for data packets to travel across a network. Network engineers can determine the optimal routes that minimize delays and packet loss by analyzing various factors such as latency, bandwidth availability, and network topology. This ensures faster data transmission and improved overall network performance.

Load Balancing

Load balancing is a technique that distributes network traffic across multiple paths or devices, avoiding bottlenecks and optimizing resource utilization. By evenly distributing the workload, load balancers ensure that no single component is overwhelmed with traffic, thereby improving network efficiency and preventing congestion.

Benefits of Network Traffic Engineering

Enhanced Performance

By implementing traffic engineering techniques, network administrators can significantly enhance network performance. Reduced latency, improved throughput, and minimized packet loss contribute to a smoother and more efficient network operation.

Scalability and Flexibility

Network traffic engineering enables scalability and flexibility in network design. It allows for the efficient allocation of resources and the ability to adapt to changing traffic patterns and demands. This ensures that networks can handle increasing traffic volumes without sacrificing performance or user experience.

Effective Resource Utilization

Optimized network traffic engineering ensures that network resources are utilized effectively, maximizing the return on investment. By efficiently managing bandwidth and routing paths, organizations can avoid unnecessary expenses on additional infrastructure and improve overall cost-effectiveness.

Challenges and Considerations

While network traffic engineering offers numerous benefits, it also comes with its own set of challenges. Factors such as dynamic traffic patterns, evolving network technologies, and security considerations must be considered. Network administrators must stay updated with industry trends and continuously monitor and analyze network performance to address these challenges effectively.

Conclusion: Network traffic engineering is a critical discipline that ensures computer networks’ efficient and reliable functioning. By employing various techniques and protocols, network administrators can optimize resource utilization, enhance the quality of service, and pave the way for future network scalability. As technology evolves, staying updated with emerging trends and best practices in network traffic engineering will be crucial for organizations to maintain a competitive edge in today’s digital landscape.

OpenFlow Service Chaining

OpenFlow and SDN Adoption

 

SDN Adoption report

 

SDN Adoption

In networking, OpenFlow and Software-Defined Networking (SDN) have emerged as groundbreaking technologies that are transforming how networks are designed, managed, and operated. This blog post aims to provide a comprehensive understanding of OpenFlow and SDN adoption, shedding light on their benefits, challenges, and the future of network infrastructure.

 

Highlights: SDN Adoption

  • The Role of OpenFlow

Firstly, the basis of the SDN adoption report is the OpenFlow protocol, an existing technology derived from academic labs. Its origins can be traced back to 2006 when Martin Casado, part of the “Clean Slate” program, developed Ethane. They were trying to figure out ways to manage the network states via a centrally managed global policy.

The idea that networks are dynamic and non-symmetrical poses challenges in keeping track of their state to enforce programmability. The program has stopped but produced several follow-up programs, including OpenFlow and SDN.

SDN OpenFlow is not something that is revolutionary new. Similar ideas have been available, and previous projects tried to solve the same problems OpenFlow is trying to solve today. Besides the central viewpoint use case, whatever you can do with OpenFlow today is possible with Policy Based Routing (PBR) and ACL. The problem is that these tools are clumsy and do not scale well.

 

You may find the following useful for pre-information:

  1. Virtual Overlay Network
  2. SDN Router
  3. What is OpenFlow
  4. BGP SDN
  5. SDN BGP
  6. Hyperscale Networking
  7. SDN Data Center

 



SDN Adoption Report.

Key SDN Adoption Discussion Points:


  • Introduction to SDN OpenFlow and what is involved.

  • Highlighting the SDN architecture.

  • Critical points on the virtual switching fabric.

  • Technical details on the use of OSPF.

  • Technical details for programming the forwarding paths.

  • Final comments on SDN OpenFlow.

 

Back to basics with the SDN.

What is OpenFlow?

OpenFlow is an open standard that enables the separation of the control plane and the data plane in network devices. It allows network administrators to centrally control and manage the behavior of network switches and routers, resulting in increased network programmability, flexibility, and scalability. OpenFlow provides a standardized protocol that facilitates communication between the control and data planes, enabling the network to be programmed and controlled through software.

Understanding SDN Adoption:

SDN is a paradigm shift in network architecture that leverages OpenFlow and other technologies to virtualize and abstract network resources. With SDN, the control plane is decoupled from the underlying physical infrastructure, allowing network administrators to configure and manage networks through a centralized controller dynamically. This centralized control simplifies network operations, enhances automation, and creates innovative network services.

The use of APIs

Besides the network abstraction, the SDN architecture will deliver a set of APIs that streamline the implementation of standard network services. These network services include routing, security, access control, and traffic engineering. Consequently, we can achieve exceptional programmability, automation, and network control, enabling us to build highly scalable and flexible networks that readily adapt to changing business needs. Then we have OpenFlow and the SDN story. OpenFlow is the first standard interface explicitly designed for SDN, providing high-performance and granular traffic control across multiple networking devices.

 

Benefits of OpenFlow and SDN Adoption:

The adoption of OpenFlow and SDN comes with numerous benefits for organizations of all sizes:

1. Enhanced Network Programmability: OpenFlow and SDN enable network administrators to program and control networks through software, making implementing new network services and policies easier.

2. Increased Flexibility and Scalability: SDN allows for dynamic network reconfiguration and resource allocation, ensuring networks can adapt to changing requirements and scale efficiently.

3. Centralized Network Management: With SDN, network administrators can manage and configure multiple network devices from a centralized controller, simplifying network operations and reducing the complexity of managing traditional networks.

4. Improved Network Security: SDN facilitates the implementation of granular security policies, enabling network administrators to detect and respond to security threats quickly, enhancing overall network security.

Challenges and Considerations:

While OpenFlow and SDN offer significant advantages, their adoption comes with a few challenges that organizations need to address:

1. Compatibility: Not all network devices and vendors fully support OpenFlow and SDN, requiring organizations to consider device compatibility carefully before implementation.

2. Skillset and Training: SDN introduces new concepts and requires network administrators to acquire skills and knowledge to deploy and manage SDN-based networks effectively.

3. Transition from Legacy Infrastructure: Migrating from traditional networking solutions to SDN-based architectures requires careful planning and a phased approach to minimize disruptions and ensure a smooth transition.

 

Starting Points for SDN Adoption

SDN Architectures and OpenFlow

SDN architectures and OpenFlow offer several advantages. You can influence traffic forwarding behavior at a more granular flow level. A holistic view instead of a partial view of distributed devices simplifies the network. Traffic engineering with SDN becomes easier to implement when you have a centralized view; this is how Google implemented SDN. Google has two network backbones; Internet-facing and data center backbone. 

They noticed that the cost/bit was not decreasing as the network grew. It was doing the opposite. Their solution was to implement a centralized controller and manage the WAN as a fabric, not as a collection of individual nodes.

 

SDN adoption report: Virtual switching fabric

SDN architectures allow networks to move from loosely coupled systems to a virtual switching fabric. One extensive flat virtualized network that appears and can be managed as a single switch has many operational advantages. The switch fabric consists of multiple physical nodes but behaves like one big switch. For example, a port on any underlying switch fabric nodes or virtual switch appears as a port to the single switching fabric.

The entire data plane becomes an abstraction. By employing this architecture, we manage the data plane as a whole entity instead of a set of loosely coupled connected devices. The control and data planes are distributed to the same locations if we study existing networks. No central point controls individual nodes, resulting in complex cross-network interactions.

 

sdn adoption

Open Shortest Path First (OSPF)

Open Shortest Path First (OSPF) calculates the shortest path tree from each node to every other node. Each OSPF neighbor must establish an adjacency, build, and synchronize the link-state databases (LSB). The complexity can be reduced by designing OSPF areas with ABRs, but by sacrificing some precision of route information. Imagine that every node reports and synchronizes its LSB to a central controller with an OSPF SDN application instead of individual nodes.

The controller can perform the Shortest Path First (SPF) calculation and directly update each node’s forwarding information base (FIB). The network now becomes programmable. While it does bring advantages, the laws of physics have not changed.

OpenFlow does not decrease latency or let you push more bits through a link. It does, however, let you manage and control your network better. It removes the box-by-box mentality and introduces automation and programmability.

SDN CONTROLLER

Do you think OpenFlow will be derailed?

SDN OpenFlow has come up against some market adoption barriers, such as silicon challenges and numerous vendor-specific extensions. In addition, the lack of conformance tests has led to some inconsistencies. It depends on how you define it. To explain it, you need to know what it is not. It is not a controller or a forwarding switch but a communication between the two.

It has a distinct place in the SDN architecture and does not run anywhere except between the control (controller) and the data plane, such as the OVS bridge acting as the switch infrastructure. SDN OpenFlow is also not alone in this space; other technologies provide control and data plane communications, such as BGP, Open vSwitch Database Management Protocol (OVSDB), NETCONF, and Extensible Message and Presence Protocol (XMPP).

Juniper’s OpenContrail uses XMPP.

 

SDN ADOPTION

It is evolving, and emerging technologies are sometimes slow to adopt. For example, in the early days of Novell networks, there were 4-frame types. Likewise, OpenFlow is changing and adapting as time progresses. For example, the original version of OpenFlow did not have multiple flow tables; now, versions 1.3 and 1.4 have multiple tables with various actions and many additional features.

 

Will it be used for program forwarding paths instead of BGP? 

Probably not, but it will augment BGP and other traditional technologies. It is not strictly a YES or NO answer as the SDN adoption falls into two buckets: one with OpenFlow and one without. Take the IPv6 adaptations as the IPv4 “replacement.” There was a “D” day of IPv4 address exhaustion, but IPv4 is still widely used. New “transition” mechanisms such as 6to4 and NAT64 are still widely deployed. It is the same with SDN and OpenFlow.

There will be ways to make traditional networks communicate with SDN and OpenFlow. BGP was invented as an EBGP, but people use EBGP Internal in their network. BGP is also used as an SDN control plane. It will be the case that you have controllers that provide automation and a holistic view but can speak BGP or OSPF to program the forwarding devices. SDN migrations will come incrementally, similar to what we see with IPv4 and IPv6

The lack of clarity in the controller space has limited the progress in OpenFlow. But; now, the controller market is consolidating, which gives users a clear path forward. This emergence is a good thing and will move OpenFlow forward. Maintaining SDN applications on different controllers is a dead-end, but now that OpenDaylight is emerging, we have controller unity.

A market with numerous open-source controllers would make SDN application development difficult. There will always be business drivers for proprietary controllers serving a particular niche and corner case problems the open community did not invest in. Even today, specialized UNIX platforms exist when you look at open Linux. Similarly, this flow of technology adoption will be evident for OpenFlow controllers.

 

The Future of OpenFlow and SDN:

The adoption of OpenFlow and SDN has gained significant momentum in recent years, and the future looks promising for these technologies. With the increasing demand for flexible, scalable, and programmable networks, OpenFlow and SDN are vital in deploying 5G networks, Internet of Things (IoT) applications, and network virtualization.

Conclusion:

OpenFlow and SDN adoption revolutionizes network infrastructure, offering increased programmability, flexibility, and centralized management. While challenges exist, the benefits of OpenFlow and SDN far outweigh the drawbacks. As organizations continue to embrace digital transformation, OpenFlow and SDN will continue to shape the future of networking, enabling agile, scalable, and secure networks that can adapt to the evolving needs of modern businesses.

 

 

SDN Adoption report

openstack foundations

Openstack Architecture in Cloud Computing

OpenStack Architecture in Cloud Computing

Cloud computing has revolutionized businesses' operations by providing flexible and scalable infrastructure for hosting applications and storing data. OpenStack, an open-source cloud computing platform, has gained significant popularity due to its robust architecture and comprehensive services.

In this blog post, we will explore the architecture of OpenStack and how it enables organizations to build and manage their own private or public clouds.

At its core, OpenStack comprises several interconnected components, each serving a specific purpose in the cloud infrastructure. The architecture follows a modular approach, allowing users to select and integrate the components that best fit their requirements.

Table of Contents

Highlights: OpenStack Architecture in Cloud Computing

The role of decoupling

The key to cloud computing is decoupling virtual resources from physical ones. The ability to abstract processors, memory, etc., from the underlying hardware enables on-demand/elastic provisioning and increased efficiency. This abstraction process has driven the cloud and led to various popular cloud flavors such as IaaS – Infrastructure-as-as-Service, PaaS – Platform-as-as-Service, and SaaS – Software-as-as-service, a base for OpenStack foundations.

The fundamentals have changed, and the emerging way of consuming I.T. ( compute, network, storage ) is the new “O.S.” for the data center in the cloud. The cloud cannot operate automatically and needs a management suite to control and deploy service-oriented infrastructures. Different companies deploy different teams specialized only in managing cloud computing. Those without an in-house team get it outsourced by firms like Global Storage. 

Related: You may find the following post of interest:

  1. OpenStack Neutron Security Groups
  2. OpenStack Neutron
  3. Network Security Components
  4. Hyperscale Networking



Openstack Architecture in Cloud Computing.

Key Openstack Architecture in Cloud Computing Discussion Points:


  • Introduction to OpenStack architecture in cloud computing and what is involved.

  • Highlighting the components of cloud computing.

  • Critical points on OpenStack foundations and operations.

  • Technical details on the use of APIs.

  • Technical details for the OpenStack deployment details.

Back to Basics: Cloud Adoption.

The adoption of cloud technology has transformed how companies run their IT services. By leveraging new strategies for resource use, several cloud solutions came into play with different categories: private, public, hybrid, and community.

OpenStack falls into the private cloud category. However, deploying OpenStack is still a difficult step, which requires a good understanding of its beneficial returns to a given organization concerning automation, orchestration, and flexibility.

The main components of OpenStack are:

1. Nova: Nova is the compute service responsible for managing and provisioning virtual machines (VMs) and other instances. It provides an interface to control and automate the deployment of instances across multiple hypervisors.

2. Neutron: Neutron is a networking service that enables creating and managing virtual networks within the cloud environment. It offers a range of networking options, including virtual routers, load balancers, and firewalls, allowing users to customize their network configurations.

3. Cinder: Cinder provides block storage to OpenStack instances. It allows users to create and manage persistent storage volumes, which can be attached to cases for data storage. Cinder supports various storage backends, including local disks and network-attached storage (NAS) devices.

4. Swift: Swift is an object storage service that provides scalable and durable storage for unstructured data. It enables users to store and retrieve large amounts of data, making it suitable for applications that require high scalability and fault tolerance.

5. Keystone: Keystone serves as the identity service for OpenStack, providing authentication and authorization mechanisms. It manages user credentials and assigns access rights to the various components and services within the cloud infrastructure.

6. Glance: Glance is an image service that enables users to discover, register, and retrieve virtual machine images. It provides a catalog of images that can be used to launch instances, making it easy to create and manage VM templates.

7. Horizon: Horizon is the web-based dashboard for OpenStack, providing a graphical user interface (GUI) for managing and monitoring the cloud infrastructure. It allows users to perform administrative tasks like launching instances, managing networks, and configuring security settings.

These components work together to provide a comprehensive cloud computing platform that offers scalability, high availability, and efficient resource management. OpenStack’s architecture is designed to be highly modular and extensible, allowing users to add or replace components per their specific requirements.

SDN Abstraction

These platforms rely on a new networking architecture known as software-defined networking. Traditional networking relies on manual administration, and its culture is based on a manual approach. Networking gear is managed box by box, and administrators maintain singular physical network hardware and connectivity. SDN, on the other hand, abstracts the network.

The switching infrastructure may still contain physical switch components but is managed like one switch. The data plane is operated as an entire entity rather than a loosely coupled connected device. SDN approach is often regarded as a prerequisite and necessary foundation for scalable cloud computing.

 

OpenStack Architecture in Cloud Computing

OpenStack Fundations and Origins

OpenStack Foundations is a software platform for orchestrating and automating data center environments. It provides APIs enabling users to create virtual machines, network topologies, and scale applications to business requirements. It does not just let you control your cloud; you may make it available to customers for unique self-service and management.

It’s a collection of projects (each has a specific mission) to create a shared cloud infrastructure maintained by a community. It enables any organization type to build its public or private cloud stack. A key differentiator from OpenStack and other platforms is that it’s open-source, run by an independent community continually updating and reviewing publicly accessible information. The key to its adoption is that customers do not fear vendor lock-in.

The pluggable framework is supported by multiple vendors, allowing customers to move away from the continuous path of yearly software license renewal costs. There is real momentum behind it. The lead-up to OpenStack and cloud computing started with Amazon Web Service (AWS) in 2006. They offered a public IaaS and virtual instances with an API. However, there was no SLA or data guarantee, so research academies mainly used it.

NASA and Rackspace

Historically, OpenStack was founded by NASA and Rackspace. NASA was creating a project called Nebula, which was used for computing. Rackspace was involved in a storage project ( object storage platform ) called Cloud Files. The two projects mentioned above led to a community of collaborating developers working on open projects and components.

There are plenty of vendors behind it and across the entire I.T. stack. For servers, we have Dell and H.P.; Storage consists of NetApp and SolidFire; Networking has Cisco and Software with VMware and IBM.

Initially, OpenStack foundations started with three primary services: NOVA computer service, SWIFT storage service, and GLANCE virtual disk image service. Soon after, many additional services, such as network connectivity, were added. The initial networking suffered simple implementations, providing only basic networking via Linux Layer 2 VLANs and IPtables.

Now, with the Neutron networks, you can achieve a variety of advanced topologies and rich network policies. Most networking is based on tunneling ( GRE or VXLAN ). Tunnels are used within the hypervisor, so it fits nicely with multi-tenancy. Tunnels are created between the host over the Layer 3 network within the hypervisor. As a result, tenancy V.M.s can spin up where they want and communicate over the tunnel.

 

What is an API?

The application programming interface ( API ) is the engine under the cloud hood. The messenger takes requests, tells the systems what you want to do, and then returns the response to you—ultimately creating connectivity.

openstack foundations

Each core project (compute, network, etc.) will expose one or more HTTP/RESTful interfaces for public or managed access. This is known as a Northbound REST API. Northbound API faces some programming interfaces. It conceptualizes lower-level detail functions. Southbound faces the forwarding plane and allows components to communicate with a lower-level part.

For example, a southbound protocol could be OpenFlow or NETCONF. Northbound and southbound are software directions from the reference point of the network operating systems. We now have an East-West interface. At the time of writing, this protocol is not fully standardized, but eventually, it will be used to communicate between federations of controllers for state synchronization and high availability.

 

OpenStack Architecture: The Foundations

  1. OpenStack Compute – Nova is comparable to AWS EC2. She is used to provisioning instances for applications.
  2. OpenStack Storage – Swift is comparable to AWS S3. Provides object storage functions for application objects.
  3. OpenStack Storage – Cinder is comparable to AWS Elastic Block Storage. Provides persistent block storage functions for stateless instances.
  4. OpenStack Orchestration – Heat is comparable to AWS Cloud formation. Orchestrates deployment of cloud services
  5. OpenStack Networking – Neutron Network is comparable to AWS VPC and ELB. Creates networks, topologies, ports, and routers.

There are others, such as Identity, Image Service, Trove, Ceilometer, and Sahara.

Each OpenStack foundation component has an API that can be called from either CURL, Python, or CLI. CURL is a command-line tool that lets you send HTTP requests and receive responses. Python is a widely-used programming language, and within the OpenStack ecosystem, Python automates scripts to create and manage resources in your OpenStack cloud. Finally, command-line interfaces (CLI) can access and send requests to APIs.

 

OpenStack Architecture & Deployment

OpenStack has a very modular design, and the diagram below displays key OpenStack components. Logically, it can be divided into three groups: a) Control, b) Network, and c) Compute. All of the features use a database or a message bus. The database can either be MySQL, MariaDB, or PostgreSQL. The message bus can be RabbitMQ, Qpid, and ActiveMQ.

The messaging and database could run on the same control node for small or DevOps deployments but could be separated for redundancy. The cloud controller on the left consists of numerous components, which are often disaggregated into separate nodes. It is the logical interface to the cloud and provides the API service.

Openstack Deployment

The network controller includes the networking service known as Neutron. It offers an API for orchestrating network connectivity. Extension plugins provide additional network services such as VPNs, NAT, security firewalls, and load balancing. Generally, it is separate from the cloud controller as traffic may flow through it. The compute nodes are the instances. This is where the application instances are deployed. 

 

Leverage vagrant 

A vagrant is a valuable tool for setting up Dev OpenStack environments to automate and build virtual machines ( with OpenStack ). It’s a wrapper around a virtualization platform, so you are not running the virtualization in Vagrant. The Vagrant V.M. gives you a pure environment to work with as it isolates dependencies from other V.M. applications. Nothing can interfere with the V.M., offering full testing scope. An excellent place to start is Devstack. It’s the best tool for setting up small single-node non-production/testing installs.

Summary: OpenStack Architecture in Cloud Computing

In the fast-evolving world of cloud computing, OpenStack has emerged as a powerful open-source platform that enables efficient management and deployment of cloud infrastructure. Understanding the architecture of OpenStack is essential for developers, administrators, and cloud enthusiasts alike. This blog post delved into the various components and layers of OpenStack architecture, providing a comprehensive overview of its inner workings.

Section 1: OpenStack Components

OpenStack comprises several key components, each serving a specific purpose in the cloud infrastructure. These components include:

1. Nova (Compute Service): Nova is the heart of OpenStack, responsible for managing and provisioning virtual machines (VMs) and controlling compute resources.

2. Neutron (Networking Service): Neutron handles networking functionalities, providing virtual network services, routers, and load balancers.

3. Cinder (Block Storage Service): Cinder offers block storage capabilities, allowing users to attach and manage persistent storage volumes to their instances.

4. Swift (Object Storage Service): Swift provides scalable and durable object storage, ideal for storing large amounts of unstructured data.

Section 2: OpenStack Architecture Layers

The OpenStack architecture is structured into multiple layers, each playing a crucial role in the overall functioning of the platform. These layers include:

1. Infrastructure Layer: This layer comprises the physical hardware resources such as servers, storage devices, and switches that form the foundation of the cloud infrastructure.

2. Control Layer: The control layer comprises services that manage and orchestrate the infrastructure layer. It includes components like Nova, Neutron, and Cinder, which control and coordinate resource allocation and network connectivity.

3. Application Layer: At the topmost layer, the application layer consists of software applications and services that run on the OpenStack infrastructure. These can range from web applications to databases, all utilizing the underlying resources OpenStack provides.

Section 3: OpenStack Deployment Models

OpenStack offers various deployment models to cater to different needs and requirements. These models include:

1. Public Cloud: OpenStack is operated and managed by a third-party service provider in a public cloud deployment, offering cloud services to multiple organizations or individuals over the internet.

2. Private Cloud: A private cloud deployment involves setting up an OpenStack infrastructure exclusively for a single organization. It provides enhanced security and control over data and resources.

3. Hybrid Cloud: A hybrid cloud deployment combines both public and private clouds, allowing organizations to leverage the benefits of both models. This provides flexibility and scalability while ensuring data security and control.

Conclusion:

OpenStack architecture is a complex yet robust framework that powers cloud computing environments. Understanding its components, layers, and deployment models is crucial for effectively utilizing and managing OpenStack infrastructure. Whether you are a developer, administrator, or simply curious about cloud computing, exploring OpenStack architecture opens up a world of possibilities for building scalable and efficient cloud environments.

network-automation3

Network Configuration Automation

 

automate network configuration

 

Network Configuration Automation

In today’s fast-paced digital landscape, businesses rely heavily on efficient network operations to stay competitive. Network configuration automation has emerged as a game-changer, enabling organizations to streamline their network management processes and enhance overall operational efficiency. This blog post will delve into network configuration automation, its benefits, and how it revolutionizes how businesses manage their networks.

Network configuration automation is the practice of using software tools to automate the management and configuration of network devices. Traditionally, network administrators have manually configured and managed network devices, which can be time-consuming, error-prone, and resource-intensive. With network configuration automation, these tasks are automated, enabling administrators to manage and control their network infrastructure centrally, reducing the risk of human error and accelerating network deployment and updates.

 

Highlights: Network Configuration Automation

  • Application Changes

How applications are deployed today is so different from the way applications were deployed 10-15 years ago. So much has changed with the app. The problem we are seeing today is that the network is not tightly coupled with these other developments. The provision of various network policies and corresponding configurations are not tightly associated with the application.

Most of the time, they are loosely coupled and reactive. For example, analyzing firewall rules and providing a network assessment is nearly impossible with old security devices driving the need for network configuration automation and the ability to automate network configuration.

 

Before you proceed, you may find the following articles of interest:

  1. Open Networking
  2. A10 Networks
  3. Brownfield Network Automation

 



Automate Network Configuration.

Key Network Configuration Automation Discussion Points:


  • Introduction to Network Configuration Automation and what is involved.

  • Highlighting the components of Automate Network Configuration.

  • Critical points on the use of Ansible and Ansible variables.

  • Technical details on how virtualization changes the manual approach.

  • Technical details on SDN as a companion to automation.

 

Back to basics with the Network Automation.

One of the easiest and quickest ways to get started with network automation is to automate the creation of the device configuration files used for initial device provisioning and push them to network devices. You can also extra a lot of information with automation. For example, network devices have enormous static and ephemeral data buried inside, and using open-source tools or building your own gets you access to this data.

Examples of this type of data include entries in the BGP table, OSPF adjacencies, active neighbors, interface statistics, specific counters and resets, and even counters from application-specific integrated circuits (ASICs) themselves on newer platforms.

 

  • A key point: Lab guide with Ansible Core

We have Ansible installed and a managed host already prepared in the following. The managed host needs to have SSH enabled and a user with admin privileges. Ansible finds managed hosts by looking at the inventory file. The inventory file is also a great place to pass variables that can be used to remove site-specific information; this is set under the host var section below.

Remember that Ansible requires Python, and below, we are running Python version 3.0.3 and Jinja version 3.0.3, which is used for templating. You can pass information to ansible managed hosts with playbooks and ad hoc commands. Below, I’m using an ad hoc command, calling the command module by default, and testing with a ping.

 

Ansible configuration
Diagram: Ansible Configuration

 

Benefits of Network Configuration Automation:

1. Time and Resource Efficiency: Organizations can free up their IT staff to focus on more strategic initiatives by automating repetitive and time-consuming network configuration tasks. This results in increased productivity and efficiency across the organization.

2. Enhanced Accuracy and Consistency: Manual configuration processes are prone to human error, leading to misconfigurations and network downtime. Network configuration automation eliminates these risks by ensuring consistency and accuracy in network configurations, reducing the chances of costly errors.

3. Rapid Network Deployment: Network administrators can quickly deploy network configurations across multiple devices simultaneously with automation tools. This accelerates network deployment and enables organizations to respond faster to changing business needs.

4. Improved Security and Compliance: Network configuration automation enhances security by enforcing standardized configurations and ensuring compliance with industry regulations. Automated security protocols can be applied consistently across the network, reducing vulnerabilities and enhancing overall network protection.

5. Simplified Network Management: Automation tools provide a centralized platform for managing network configurations, making it easier to monitor, troubleshoot, and maintain network devices. This simplifies network management and reduces the complexity associated with manual configuration processes.

Implementing Network Configuration Automation:

To implement network configuration automation, organizations need to consider the following steps:

1. Assess Network Requirements: Understand the specific network requirements, including device types, network protocols, and security policies.

2. Select an Automation Tool: Evaluate different automation tools available in the market and choose the one that aligns with the organization’s needs and network infrastructure.

3. Create Configuration Templates: Develop standardized configuration templates that can be easily applied to network devices. These templates should include best practices, security policies, and network-specific configurations.

4. Test and Validate: Before deploying automated configurations, thoroughly test and validate them in a controlled environment to ensure their effectiveness and compatibility with the existing network infrastructure.

5. Monitor and Maintain: Regularly monitor and maintain the automated network configurations to identify and resolve any issues or security vulnerabilities that may arise.

 

The Need to Automate Network Configuration

There are always hundreds, if not thousands, of outdated rules even though the application service is not required. Another example is unused VLANs left configured on access ports, posing a security risk. The problem lies in the process: how we change and provision the network is not tied to the application. It is not automated. Inconsistent configurations tend to grow as human interaction is required to tidy things up. People move on and change roles.

You cannot guarantee the person creating a firewall rule will be the engineer deleting the rule once the corresponding applications are decommissioned or changed. And if you don’t have a rigorous change control process, deprecated configurations will be idle on active nodes.

 

  • A key point: The use of Ansible variables in an Ansible architecture.

For configuration management, you could opt for Red Hat Ansible. The Ansible architecture consists of modules with tasks on the target hosts listed in the inventory. Various plugins are available for additional context and Ansible variables for flexible playbook development. Ansible Core is the CLI-based version of automation, and Ansible Tower is the platform.

The recommended approach for enterprise-wide security would be a platform-based approach to the Ansible architecture. Using a platform approach using Ansible variables creates a very flexible automation journey where you can have one playbook with Ansible variables, removing any site-specific information running against several different inventories that could relate to your other function, Dev, Staging, and Production.

Network Automation

The network is critical for business continuity, resulting in real uptime pressure. The operational uptime is directly tied to the success of the business. This results in a manual. The culture that manifests is manual and slow. The actual bottleneck is our manual culture for network provision and operation.

 

Virtualization – Beginning the change

Virtualization vendors are changing the manual approach. For example, if we look at essential MAC address learning and its process with traditional switches. The source MAC address of an incoming Ethernet frame is examined, and if the source MAC address is known, it doesn’t need to do anything, but if it’s not known, it will add that MAC to its table and make a note of the port the frame entered. The switch has a port for MAC mapping. The table is continually maintained, and MAC addresses are added and removed via timers and flushing.

 

The virtual switch

The virtual switch operates differently. Whenever a VM spins up, and a VNIC attaches to the virtual switch, the Hypervisor programs everything it needs to know to forward that traffic into its process on the virtual switch. There is no MAC learning. When you spin down the VM, the hypervisor does not need to wait for a timer.

It knows the source is no longer there; as a result, it no longer needs to have that state. Less state in a network is a good thing. The critical point is that the provision of the application/ virtual machine is tightly coupled with the provisioning of network resources. Tightly coupling applications to network resources/provisioning offers less “Garbage Collection.”

 

Box mentality  

When the contents of HLD / LLD are completed, and you are now moving to the configuration stage, the current implementation-specific details are done per box. The commands are defined on individual boxes and are vendor-specific. This works functionally, and it’s how the Internet was built, but it lacks agility and proper configuration management. Many repetitive tasks with a box mentality destroy your ability to scale.

Businesses are mainly concerned with agility and continuity, but you cannot have these two things with manual provisions. You must look at your network as a system, not as individual boxes. When you look at applications and their scaling, the current network-style implementation method does not scale and keeps in line with the apps. A move to network configuration automation and automatic interaction is the solution.

 

Configuration management

Network Configuration Automation and Automate Network Configuration

We must move out of a manual approach and into an automated system. Focus initially on low-hanging fruit and easy wins. What takes engineers the longest to do? Do VLAN and Subnet allocation sheets ring a bell? We should size according to demand and not care about the type of VLAN or the Internal subnet allocation. Microsoft Azure cloud is a perfect example.

They do not care about the type of private address they assign to internal systems. They automate the IP allocation and gateway assignment so you can communicate locally. Designing optimum networks to last and scale is not good enough anymore. The network must evolve and be programmed to keep up with app placement. The configuration approach needs to change, and we should move to proper configuration management and automation.

A widespread tool of choice is Ansible. As previously mentioned, we have Ansible Tower as a platform, and for CLI-based devices, we have Ansible Core—both support variable substitution with Ansible variables.

 

SDN: A companion to network automation?

One benefit of Software Defined Networking (SDN) is that it lets you view your network holistically, with a central viewpoint. Network configuration automation is not SDN, and SDN is not network automation. They work side by side and complement each other. SDN allows you to be abstract and prevents those not needing to see the detail from not seeing it.

The application owners do not care about VLANs. Application designers should also not care about local IP allocations if they have designed the application correctly. Centralization is also a goal for SDN. Centralization with SDN is different from control-plane centralization. Central SDN controller devices should not fully control the control plane.

SDN companies have learned this and now allow network nodes to handle some or part control plane operations.  

 

  • Programming network: Automate network configuration

You don’t need to be a programmer, but you should start to think like one. Learning to program will make you better equipped to deal with things. Programming networks is a diagonal step to what you are doing now, offering an environment to run code and ways to test code before you run it out.

The current CLI is the most dangerous approach to device configuration; you can even lock yourself out of a device. Programming adds a safety net. It’s more of a mental shift. Stop jumping to the CLI and THINK FIRST. Break the task down and create workflows. Workloads are then mapped to an automation platform.

 

  • A key point: TCL and EXPECT

TCL ( Tool Command Language ) is a scripting language that was created in 1988 at UC Berkeley. It aims to tie together Shell scripts and Unix commands. EXPECT is a TCL extension written by Don Libes. It automates Telnet, SSH, and Serial sessions to perform many repetitive tasks.

EXPECT’s main drawback is that it is not secure and is synchronous only. If you log onto a device, you display login credentials in the EXPECT scripts and cannot encrypt that data in the code. In addition, it operates sequentially, meaning you send a command and wait for the output; it does not send send send and wait to receive; it’s a send and waits, sends and wait for mythology.

 

  • A key point: SNMP has failed | NETCONF begins

SNMP is used for fault handling, monitoring equipment, and retrieving performance data, but very few use SNMP to set configurations. More often, there is no 1:1 translation between a CLI configuration operation and an SNMP “SET.” It’s hard to get this 1-2-1 correlation. As a result, not many people use SNMP for device configuration and management of structures.

CLI scripting was the primary approach to automating configuration changes to the network before NETCONF. Unfortunately, CLI scripting has several limitations, including a lack of transaction management, no structured error management, and the ever-changing structure and syntax of commands, making scripts fragile and costly to maintain. Even the use of autocorrelation scripts won’t be able to fix it.

People make mistakes, and ultimately, people are bad at stuff. It’s the nature of the beast. Human error plays a significant role in network outages, and if a person is not logging in doing CLI, they are less likely to make a costly mistake. Human interaction with the network is a major cause of network outages.

 

NETCONF & Tail-F

NETCONF ( network control protocol ) is an XML-based data encoding for configuration and protocol messages. It offers secure transport and is Asynchronous, so it’s not sequential like TCL and EXPECT. Asynchronous makes NETCONF more efficient. It allows the separation of the configuration from the non-configuration items.

Backup and restore are complex with SNMP, as you have no idea what fields are used to restore. Also, because of the binary nature of SNMP, it isn’t easy to compare configurations from one device to another. NETCONF is much better at doing this.

 

A final note: Transaction-based approach

It offers a transaction-based approach. A transaction is a set of configuration changes, not a sequence. SNMP for configuration requires everything to be in the correct sequence/order. But with a transaction, you throw in everything, and the device figures out how to roll it out.

What matters is that operators can write service-level applications that activate service-level changes and don’t have to make the application aware of all the sequence of changes that must be completed before the network can serve application responses and requests. Check out an exciting company called Tail-F (now part of Cisco) which offers a family of NETCONF-enabled products.

 

Conclusion:

Network configuration automation is revolutionizing how businesses manage their networks. It offers many benefits, including time and resource efficiency, enhanced accuracy, rapid network deployment, improved security, and simplified network management. By embracing this technology, organizations can streamline network operations, reduce human error, and stay ahead in the dynamic and ever-evolving digital landscape.

 

Automation Network Configuration