Suggested Reading: Introduction to OpenStack and a Summary of Getting Started Guides
Let 's start with a quick introduction to IaaS and its infrastructure before diving into this leading open source solution: OpenStack.
Both IaaS and Cloud Infrastructure
Cloud computing architectures often focus on a set of common resources that are virtualized and exposed to users on demand. These resources include computing resources with different functions, persistent storage resources, and configurable networking resources that group these resources together and conditionally expose them to the Internet.
The infrastructure for an IaaS implementation (see Figure 1) follows this model with additional elements such as metering (to account for usage for billing purposes). The physical infrastructure is abstracted from applications and users through virtualization layers implemented by various technologies, including hypervisors (for platform virtualization), virtual networking, and storage.
Figure 1. High-level view of IaaS
While OpenStack is one of the most popular open source cloud solutions available today, it certainly wasn't the first. In fact, OpenStack is a synthesis of two legacy solutions developed in the public and private domains.
An earlier open source IaaS solution is the Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems (Eucalyptus), developed by a research project at the University of California, Santa Barbara. Other solutions include OpenNebula (an open source cloud computing toolbox) and Nimbus (another open source toolbox for IaaS clouds). OpenStack integrates with parts of NASA's Nebula platform and the Rackspace Cloud Files project (cloud storage).
Newcomers to Cloud Computing: OpenStack
OpenStack is a newcomer to the IaaS space, first released in late 2010. Although this solution is less mature and less than two years old, OpenStack is now one of the most widely used cloud stacks. OpenStack is not a single solution, but a growing suite of open source solutions (including core as well as newly incubated projects) that together form a robust and mature IaaS stack.
As shown in Figure 2, OpenStack is built from several core technologies (more than shown here, but those shown here represent key aspects). On the left is the Horizon dashboard, showing a user interface that users and administrators can use to manage OpenStack services. Nova provides a scalable computing platform that supports the configuration and management of large numbers of servers and virtual machines (VMs; in a hypervisor-agnostic way). Swift implements a massively scalable object storage system with internal redundancy. At the bottom are Quantum and Melange, which implement network connectivity as a service. Finally, the Glance project implements a repository for virtual disk images (image as a service).
Figure 2. Core and add-ons of an
OpenStack solution Core and add-ons of an OpenStack solution As shown in
Figure 2, OpenStack is a collection of projects that collectively provide a complete IaaS solution. Table 1 presents these items and the aspects in which they work.
Table 1. OpenStack projects and components
Other important aspects include Keystone, which implements the identity services (management of access to compute servers, images within Glance, and Swift object stores) that are critical to enterprise private clouds.
OpenStack Architecture
OpenStack is represented by three core open source projects (shown in Figure 2): Nova (compute), Swift (object storage), and Glance (VM repository). Nova (or OpenStack Compute), provides management of VM instances across a network of servers. Its application programming interface (API) provides a computational orchestration method for attempts to be ignorant of physical hardware and hypervisors. Note that Nova provides not only an OpenStack API for management, but also an Amazon EC2-compatible API for those who are used to the interface. Nova supports proprietary hypervisors used by organizations, but more importantly, it supports hypervisors like Xen and Kernel Virtual Machine (KVM), as well as OS virtualization like Linux® Container. For development purposes, an emulation solution like QEMU can also be used.
The Swift (or OpenStack Object Storage) project provides a scalable, redundant storage cluster with standard servers configured with regular hard drives. Swift does not represent a file system, it implements a more traditional object storage system that can be used for long-term storage of primarily static data (a key usage model is static VM images). Swift doesn't have a centralized controller, which improves overall scalability. It manages replication across the cluster internally (without the need for a redundant array of independent disks) to improve reliability.
Glance (or OpenStack Image Service) provides a repository for virtual disk images that Nova can use (this option is stored within Swift). Glance provides an API to register disk images, as well as discovery and delivery through a simple Representational State Transfer (REST) interface. Glance is largely agnostic to virtual disk image formats and supports a variety of standards including VDI (VirtualBox), VHD (Microsoft® Hyper-V®), QCOW2 (QEMU/KVM), VMDK/OVF (VMware), and raw formats . Glance also provides disk image checksums, version control (and other metadata), and virtual disk verification and audit/debug logs.
The core OpenStack projects (Nova, Swift, and Glance) are developed in Python, and they are all available under the Apache License.
Demonstrating OpenStack
Installing OpenStack can be time-consuming due to the large number of separate projects that need to be installed and configured to work with each other (see Resources for more information on a full installation). But for the curious reader, there are other options that greatly simplify getting OpenStack up and running.
Readers of my previous articles know that I'm a VM image junkie with a passion for the simplicity of using Linux-based software. A VM allows you to easily create a new instance to try out or demonstrate the software. This VM is a self-contained Linux instance (sometimes called a virtual appliance) that can be preinstalled and preconfigured for use with the necessary software. Software configured in this way greatly simplifies its use, allowing you to experiment with software that would otherwise be difficult or time-consuming to master. See Resources for installation options that meet your specific hardware and underlying operating system needs.
For this demonstration, I decided to use the latest Ubuntu release (12.04) and the OpenStack Essex release. The ISO version of Essex can be installed using the uksysadmin installer (see Resources). After a successful installation of OpenStack Essex on Ubuntu Precise, an external web browser can be used to log into the OpenStack Dashboard. Figure 3 shows the System Panel Images tab interface with Guest VM images in two container formats.
Figure 3. OpenStack Dashboard view of the available guest image
This image can be used to create a demo instance, as shown in Figure 4, the creation has begun. The instance is now ready to use.
Figure 4. OpenStack Dashboard view of the compute instance
\
Now, with the compute image running on OpenStack, I can access it using its IP address (172.16.1.1) through a simple Secure Shell (SSH) session (see Listing 1, user input in bold).
Listing 1. Accessing an OpenStack compute instance via SSH
$ ssh -i Downloads/demo.pem [email protected] The authenticity of host '172.16.1.1 (172.16.1.1)' can't be established. RSA key fingerprint is df:0e:d0:32:f8:6d:74:49:ea:60:99:82:f1:07:5d:3b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.16.1.1' (RSA) to the list of known hosts. Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-virtual x86_64) * Documentation: https://help.ubuntu.com/ System information disabled due to load higher than 1.0 0 packages can be updated. 0 updates are security updates. Get cloud support with Ubuntu Advantage Cloud Guest http://www.ubuntu.com/business/services/cloud The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. To run a command as administrator (user "root"), use "sudo <command>". See "man sudo_root" for details. ubuntu@demo1:~$ ubuntu@demo1:~$ hostname demo1 ubuntu@demo1:~$ ps PID TTY TIME CMD 835 pts/0 00:00:06 bash 948 pts/0 00:00:00 ps ubuntu@demo1:~$
With all these layers running, it can be hard to imagine what's going on. Figure 5 shows the entire stack, hopefully this will help clarify. In this demo, a Mac running Mac OS X provides the base platform. VirtualBox runs on Mac OS X and provides the OpenStack execution platform (running on Ubuntu Linux). Note that VirtualBox is a type-2 hypervisor. At the OpenStack Linux layer, QEMU is used as a guest hypervisor, which is ideal from a hardware perspective, but lacks the performance required for a real production setup.
Figure 5. OpenStack demo stack running on normal hardware
Without support for nested virtualization (effectively running one hypervisor on top of another), I rely on QEMU as my guest running on OpenStack System management program. This allows me to run a guest VM on a guest hypervisor that runs on a type-2 hypervisor. While this setup may be slow, it fully demonstrates an IaaS stack running on a common computer system. Note that some AMD processors today offer an efficient way to support nested virtualization.
While using QEMU is not ideal for performance, it is extremely compatible with KVM (Linux as a hypervisor), making it easy to migrate between the two hypervisors (except for VM images that were compatible before the two). What makes QEMU ideal in this case is that it can execute on hardware that does not provide virtualization support. Note that my platform is virtualization-capable in this example, but since I'm running on VirtualBox (a hypervisor in its own right), the lack of nested virtualization forced me to use a non-virtualized An extended guest hypervisor. In either case, I use libvirt to manage the VMs (start, stop, monitor, etc.), so migrating to KVM on a hardware capable of virtualization is as easy as two lines in the OpenStack configuration file You can modify it.
Other Ways to Use OpenStack
Even if you don't own a cluster, there are other options for enjoying the benefits of OpenStack. Rackspace, one of the creators of OpenStack, offers Linux for the cloud it has high hopes for. Rackspace's OpenStack Cloud Platform provides the flexibility and scalability of OpenStack over public cloud infrastructure.
To simplify OpenStack installations for private clouds, many companies have focused on simplifying the use of OpenStack within private clusters. For example, Piston Cloud Computing provides Piston Enterprise OS, an OpenStack-based private cloud operating system. Mirantis provides enterprises with professional services for building OpenStack infrastructure.
The future of OpenStack
OpenStack will continue to integrate new capabilities, improving the definition of an IaaS solution. There are many other projects available under the OpenStack umbrella, but many are still in the incubation process. The identity service provided by the Keystone project unifies authentication across OpenStack components while integrating with existing authentication systems. There are also community projects with different purposes, such as load balancing as a service (Atlas-LB); cloud installation and maintenance system (Crowbar); cloud configurable and scalable relational database (RedDwarf); REST-based API for cloud orchestration (Heat); and a cloud management tool (Clanavi) that covers monitoring, billing, and more. There are countless other projects in development both inside and outside the OpenStack project, and this list is growing every day as long as OpenStack builds momentum.
OpenStack is not without competition, and as old projects continue to grow, new ones emerge. For example, CloudStack (first released in 2009) has several product installations but lacks the level of open source contributor support that OpenStack has.
Conclusion
Following Linux's evolution as a general-purpose operating system that fits all usage models, OpenStack is also moving toward representing a cloud operating system. Instead of managing a limited set of core and local resources, OpenStack manages a vast network of servers containing compute and storage resources, and the virtual network glue that connects them together.
Since it was first released in late 2010 (Austin), the OpenStack project has released four more versions, the latest in April 2012 (Essex). With each release, OpenStack continues to push new and improved capabilities, raising the bar for other IaaS solutions to compete. It's no surprise that OpenStack is now the standard within the cloud stack under the Apache umbrella.