Cloud Computing & OpenStack Foundations
The key to cloud computing is the decoupling of virtual resources from physical. The ability to abstract processors, memory etc from the underlying hardware enables on-demand/elastic provisioning and increased efficiency. This abstraction process has driven the cloud and led to a variety of popular cloud flavors such as IaaS – Infrastructure-as-as-Service, PaaS – Platform-as-as-Service and SaaS – Software-as-as-service. The fundamentals have changed and the emerging way of consuming I.T ( compute, network, storage ) is in the cloud; the new “OS” for the data centre.
Obviously, the cloud cannot operate automatically and needs a management suite to control and deploy service-oriented infrastructures. The remainder of the post discusses one of the most well-known management suites called OpenStack.
These platforms rely on a new networking architecture known as software-defined networking. Traditional networking relies on manual administration and it’s culture is based on a manual approach. Networking gear is managed box by box and administrators maintain singular physical network hardware and connectivity. SDN, on the other hand, abstracts the network. The switch infrastructure may still contain physical switch components but managed like one switch. The data plane is operated as an entire entity rather than a set of loosely coupled connected devices. SDN approach is often regarded as a prerequisite and necessary foundation for scalable cloud computing.
OpenStack is a software platform for the orchestration and automation of data centre environments. It provides API’s enabling users to create virtual machines, network topologies and scale applications to business requirements. It does not just let you control your cloud, you may make it available to customers for individual self-service and management.
It’s a collection of projects (each has a specific mission) with the goal of creating a shared cloud infrastructure maintained by a community. It enables any organization type to build their public or private cloud stack. A key differentiator from OpenStack and other platforms is that it’s open source, run by an independent community continually updating and reviewing publicly accessible information. The key to its adoption is that customers do not have the fear of vendor lock-in. The pluggable framework is supported by multiple vendors, allowing customers to move away from the continuous path of yearly software licences renewal costs. There is real momentum behind it.
The lead up to OpenStack and the cloud computing started with Amazon Web Service (AWS) in 2006. They offered a public IaaS and virtual instances with an API. There was no SLA or data guarantee so it was mainly used by research academies. Amazon continued to build the first private IaaS in 2008, called Eucalyptus.
Historically, OpenStack was founded by NASA and Rackspace. NASA was creating a project called Nebula, which was used for compute. Rackspace was involved in a storage project ( object storage platform ) called Cloud Files. Both were written in Python. These two projects led to a community of collaborating developers working on open projects and components. There are plenty of vendors behind it and from across the entire I.T stack. For servers we have Dell and HP, Storage consists of NetApp and SolidFire, Networking has Cisco and Software with VMware and IBM.
Initially, OpenStack started off with three major services: NOVA compute service, SWIFT storage service, and GLANCE virtual disk image service. Soon after many additional services, such as network connectivity as a service was added. The initial networking suffered simple implementations, providing only basic networking via Linux Layer 2 VLANs and IPtables. Now, with Neutron networking you can achieve a variety of advanced topologies and rich network policies. A lot of the networking is based on tunneling ( GRE or VXLAN ). Tunnels are used within the hypervisor so it fits nice with multi-tenancy. Tunnels are created between the host over Layer 3 network within the hypervisor. As a result, tenancy VMs can spin up where they want and communicate over the tunnel.
What is an API?
Application programming interface ( API ) is the engine under the clouds hood. It is the messenger that takes requests and tells the systems what you want to do and then returns the response back to you. Ultimately creating connectivity.
Each core project (compute, network etc) will expose one or more HTTP/RESTful interfaces for use of public or manage access. This is known as a Northbound REST API. Northbound API faces some kind of programming interface. It conceptualizes lower-level detail functions. Southbound faces the forwarding plane and allows components to communicate with a lower-level component. A southbound protocol could be OpenFlow or NETCONF. Basically, northbound, and southbound are software directions from the reference point of the network operating systems.
We now have East-West interface. At the time of writing, this protocol is not fully standardised but eventually it will be used to communicate between federations of controllers for state synchronization and high availability.
A) OpenStack Compute -Nova is comparable to AWS EC2. Used to provision instance for applications.
B) OpenStack Storage -Swift is comparable to AWS S3. Provides object storage functions for application objects.
C) OpenStack Storage -Cinder is comparable to AWS Elastic Block Storage. Provides persistent block storage functions for stateless instances.
D) OpenStack Orchestration -Heat is comparable to AWS Cloud formation. Orchestrates deployment of cloud services
E) OpenStack Networking -Neutron is comparable to AWS VPC and ELB. Creates networks, topologies, ports, and routers.
There are others such as Identity, Image Service, Trove, Ceilometer, Sahara.
Each component has an API that can be called from either CURL, Python, or the CLI. CURL is command line tool that lets you send HTTP requests and receive responses. Python is a widely used programming language and within the OpenStack ecosystem python automates scripts used to create and manage resources in your OpenStack cloud. Command Line Interface (CLI) are command line tools that can also be used to access and send requests to API’s.
Architecture & Deployment
OpenStack has a very modular design and the diagram below display key OpenStack components. Logically, it can be divided into three groups, a) Control, b) Network c) Compute. All of the components use a database or a message bus. The database can either be MySQL, MariaDB, or PostgreSQL. The message bus can be RabbitMQ, Qpid, and ActiveMQ. For small or DevOps deployments, the messaging and database could run on the same control node, but they could be separated for redundancy.
The cloud controller on the left consists of numerous components, which are often disaggregated to separate nodes. It is the logical interface to the cloud and provide the API service.
The network controller includes the networking service known as Neutron. It offers an API for orchestrating network connectivity. Extension plugins are available to provision additional network services such as VPNs, NAT, security firewalls and load balancing. Generally, separate from the cloud controller as traffic may flow through it. The compute nodes are the instances. This is where the application instances are deployed.
Vagrant is a useful tool for setting up a Dev OpenStack environments for automating and building virtual machines ( with OpenStack ). It’s a wrapper around a virtualization platform so you are not running the virtualization in Vagrant. The vagrant VM gives you a pure environment to work with as it isolates dependencies from other VM / applications. Nothing can interfere with the VM, offering full testing scope.
A good place to start is with Devstack. It’s the best tool for setting up small single node non-production/testing installs.