OpenStack-basic theory

1. Cloud computing overview

1. Concept

狭义的云计算是指IT基础设施的交付和使用模式
广义的云计算是指服务的交付和使用模式

2. Cloud resources

Network resources
Storage resources
Server resources

3. Cloud computing service model

IaaS (Infrastructure as a Service)

提供底层IT基础设施服务，包括处理能力、存储空间、网络资源等
一般面向对象是IT管理人员

PaaS (Platform as a Service)

把安装好开发环境的 系统平台作为一种服务通过互联网提供给用户
一般面向对象是开发人员

SaaS (Software as a Service)

直接通过互联网为用户提供软件和应用程序的服务
一般面向的对象是普通用户

Two, OpenStack overview

一款云计算模型中的一个项目
NASA和Rackspace合作研发并发起
以Apache许可证授权的自由软件和开放式源代码项目
旨在为公共及私有云的建设与管理提供软件的开源项目
覆盖了网络、虚拟化、操作系统、服务器等各个方面
OpenStack服务

service	project name	description
Compute (Compute Service)	New	Responsible for the management of the instance life cycle and the unit of computing resources. Shield the Hypervisor, support multiple virtualization technologies (Red Hat defaults to KVM), and support horizontal expansion
Network (Network Service)	Neutron	Responsible for the management of the virtual network and create the network topology for the instance. It is a tenant-oriented network management, you can define your own network, and each tenant does not affect each other
Identity (identity authentication service)	Keystone	Similar to LDAP service, it authenticates and authorizes users, tenants, roles, and services, and supports multiple authentication mechanisms
Dashboard (Control Panel Service)	Horizon	Provide a WEB management interface to interact with the underlying services of OpenStack
Image Service	Glance	Provide registration and management of virtual machine image templates, copy the prepared operating system as an image template, and use it directly when creating a virtual machine, and support multi-format images
Block Storage (block storage service)	Cinder	Responsible for providing durable fast storage devices for running instances, which can be easily expanded, paid on demand, and support a variety of back-end storage
Object Storage (Object Storage Service)	Swift	Provides elastic storage of the basic cloud for OpenStack, and supports clusters without a single point of failure
Telemetry (metering service)	Ceilometer	A centralized source for measuring, monitoring and controlling data resources,

1. Introduction to important components

OpenStack authentication service (Keystone)

• Keystone provides authentication and access policy services for all OpenStack components. It relies on its own REST (Identity API based) system to work, mainly for (but not limited to) Swift, Glance, Nova, etc. for authentication and authorization. In fact, authorization authenticates the legitimacy of the request from the source of the action message. The following figure shows the identity authentication service process:
  

Keystone采用两种授权方式，一种基于用户名/密码，另一种基于令牌（Token）。

除此之外，Keystone提供以下三种服务：
令牌服务：含有授权用户的授权信息
目录服务：含有用户合法操作的可用服务列表
策略服务：利用Keystone具体指定用户或群组某些访问权限

Keystone authentication service notes:

• Service entrance: Like Nova, Swift and Glance, each OpenStack service has a designated port and exclusive URL, which we call endpoints.
• Location: In a data center, a location specifies a physical location. In a typical cloud architecture, if not all services access a distributed data center or server, it is also called a location.
• User: Keystone authorized user
  　　PS: On behalf of an individual, OpenStack authorizes services to them in the form of users. The user has credentials and may be assigned to one or more tenants. After verification, a specific token is provided for each individual tenant.
• Service: In general, any component connected or managed through Keystone is called a service. For example, we can call Glance a service of Keystone.
• Role: In order to maintain security restrictions, in terms of operations that a specific user can perform in the cloud, the role associated with the user is very important.
  　　PS: A role is a set of usage permissions applied to a tenant to allow a specified user to access or use a specific operation. A role is a logical grouping of usage permissions, which allows general permissions to be simply grouped and bound to users related to a specified tenant.
• Rental room: A rental room refers to a project with all service entrances and specific member roles.
  　　PS: A rental room is mapped to a Nova "project-id". In object storage, a rental room can have multiple containers. According to different installation methods, a rental room can represent a customer, account, organization or project.

OpenStack computing facility-Nova

• Nova is a flexible controller for OpenStack computing. The various actions required for the life of an OpenStack cloud instance will be processed and supported by Nova, which means that Nova will appear as a management platform and is responsible for managing the computing resources, networks, authorization and measurement of the entire cloud. Although Nova itself does not provide any virtual capabilities, it will use the libvirt API to interact with the host of the virtual machine. Nova provides external processing interfaces through Web service APIs, and these interfaces are compatible with Amazon's Web service interfaces.
  Functions and features:

实例生命周期管理
计算资源管理
网络与授权管理
基于REST的API
异步连续通信
支持各种宿主：Xen、XenServer/XCP、KVM、UML、VMware vSphere及Hyper-V

Nova elastic cloud (OpenStack computing component) includes the following main parts:

• API Server（nova-api）

• Message queue (rabbit-mq server)

• Computing workstation (nova-compute)

• Network Controller (nova-network)

• Volume management (nova-volume)

• Detailed explanation of the scheduler (nova-scheduler)
  :

• 1. API server (nova-api)
  API server provides an interface for cloud facilities to interact with the outside world. It is the only channel for outside users to manage the cloud. Through the use of web services to call various EC2 APIs, the API server then sends the request to the target facility in the cloud for processing through the message queue. As an alternative to EC2-api, users can also use OpenStack's native API, which we call "OpenStack API".

• 2. Message queue (Rabbit MQ Server)
  OpenStack uses message queue for communication based on AMQP (Advanced Message Queue Protocol). Nova makes an asynchronous call to the request response, and when the request is received, it immediately triggers a callback. Due to the use of asynchronous communication, no user actions will be placed in a waiting state for a long time. For example, the process of starting an instance or uploading a mirror is time-consuming, and the API call will wait for the return result without affecting other operations. Here, asynchronous communication plays a big role, making the entire system more efficient.

• 3. The scheduler (nova-scheduler) The
  scheduler is responsible for delivering nova-API calls to the target. The scheduler runs as a daemon named "nova-schedule", and appropriately selects the computing server from the available resource pool according to the scheduling algorithm. There are many factors that can affect the scheduling results, such as load, memory, distance of child nodes, CPU architecture, and so on. The powerful thing is that the nova scheduler uses a pluggable architecture.

At present, the nova scheduler uses several basic scheduling algorithms:

随机化：主机随机选择可用节点；
可用化：与随机相似，只是随机选择的范围被指定；
简单化：应用这种方式，主机选择负载最小者来运行实例。负载数据可以从别处获得，如负载均衡服务器。

• 4. Computing workstation (nova-compute)
  The main task of computing workstation is to manage the entire life cycle of an instance. They receive requests and execute them through the message queue, thereby performing various operations on the instance. In a typical actual production environment, many computing workstations will be set up. According to the scheduling algorithm, an instance can be deployed on any computing workstation available.

• 5. The network controller (nova-network) The
  network controller handles the network configuration of the host, such as IP address allocation, configuration project VLAN, setting security groups, and configuring the network for computing nodes.

• 6. Volume workstation (nova-volume)
  Volume workstation management is based on LVM instance volumes. It can create, delete, and attach volumes for an instance, and it can also detach volumes from an instance. Why is volume management so important? Because it provides a means to maintain persistent storage of an instance, for example, when an instance is terminated, if the root partition is non-persistent, any changes to it will be lost. However, if you separate the volume from an instance, or attach a volume to this instance, the data will still be stored even if the instance is shut down. This data can be re-accessed by attaching the volume to the original instance or other instances.

OpenStack image server-Glance

• The OpenStack image server is a system for discovering, registering, and retrieving virtual machine images. We can store images in any of the following storage:

本地文件系统（默认）
S3直接存储
S3对象存储（作为S3访问的中间渠道）
OpenStack对象存储等等。

Functions and features:

提供镜像相关服务。

Glance components:

• Glance-API:
  　 Mainly responsible for receiving Restful requests in response to mirror management commands, analyzing the message request information and distributing the commands it carries (such as adding, deleting, updating, etc.). The default binding port is 9292.
• Glance-Registry:
  　　Mainly responsible for receiving Restful requests in response to mirror metadata commands. Analyze the message request information and distribute the commands it carries (such as obtaining metadata, updating metadata, etc.). The default bound port is 9191.

OpenStack storage facility-Swift

• Swift provides a distributed and continuous virtual object storage for OpenStack, which is similar to Amazon Web Service's S3 simple storage service. Swift has the ability to store hundreds of objects across nodes. Swift has built-in redundancy and failover management, and can also handle archiving and media streams, especially for the measurement of large data (gigabytes) and large capacity (multiple objects).
  swift functions and features:

海量对象存储
大文件（对象）存储
数据冗余管理
归档能力-----处理大数据集
为虚拟机和云应用提供数据容器
处理流媒体
对象安全存储
备份与归档
良好的可伸缩性

Swift components

Swift账户
Swift容器
Swift对象
Swift代理
Swift RING

Swift proxy server

• Users interact with the proxy server through Swift-API. The proxy server is the guard that receives external requests. It detects the location of legal entities and routes their requests.
• In addition, the proxy server also handles the repeated routing request of the failover entity when the entity fails and transfers.

Swift object server

• Object server is a kind of binary storage, which is responsible for processing storage, retrieval and deletion of object data in local storage. Objects are typical binary files stored in the file system, with metadata (xattr) with extended file attributes.

• Note: The xattr format is supported by ext3/4, XFS, Btrfs, JFS and ReiserFS in Linux, but there is no valid test to prove that it can also run well under XFS, JFS, ReiserFS, Reiser4 and ZFS. However, XFS is considered the best option currently.
  Swift container server

• The container server will list all objects in a container, and the default object list will be stored as a SQLite file (Translator's Note: It can also be modified to MySQL, MySQL is used as an example during installation). The container server also counts the number of objects contained in the container and the storage space consumption of the container.
  Swift account server

• The account server is similar to the container server in that it will list the objects in the container.
  Ring (index ring)

• The Ring container records the location information of the physical storage objects in Swift. It is a virtual mapping of the entity name of the real physical storage location, similar to an index service that finds and locates the real physical location of entities in different clusters. The so-called entities here refer to accounts, containers, and objects, all of which have their own different Rings.

OpenStack management web interface----Horizon

• Horizon is a web control panel used to manage and control OpenStack services. It can manage instances, images, create key pairs, add volumes to instances, and operate Swift containers. In addition, users can also use the terminal (console) or VNC to directly access the instance in the control panel.
  In short, Horizon has the following characteristics:

实例管理：创建、终止实例，查看终端日志，VNC连接，添加卷等
访问与安全管理：创建安全群组，管理密匙对，设置浮动IP等
偏好设定：对虚拟硬件模板可以进行不同偏好设定
镜像管理：编辑或删除镜像
查看服务目录
管理用户、配额及项目用途
用户管理：创建用户等
卷管理：创建卷和快照
对象存储处理：创建、删除容器和对象
为项目下载环境变量

2. Openstack network structure diagram

The entire OpenStack is composed of control nodes, computing nodes, network nodes, and storage nodes. (These four nodes can also be installed on one machine and deployed on a single machine)
Among them: the
control node is responsible for the control of the remaining nodes, including virtual machine establishment, migration, network allocation, storage allocation, etc. The
computing node is responsible for the virtual machine operation
network node Responsible for the communication between the external network and the internal network, the
storage node is responsible for the additional storage management of the virtual machine, etc.

Control node architecture:

The control node includes the following services

• Management support services
• Basic management services
• Extended management services
  1. Management support services include two services, MySQL and Qpid
• MySQL: The database is used as a place for storing data generated by basic/extended services
• Qpid: Message broker (also known as message middleware) provides a unified message communication service between various other services
  . 2. Basic management services include five services: Keystone, Glance, Nova, Neutron, and Horizon
• Keystone: Authentication management service, which provides authentication information/token management, creation, modification, etc. of all other components, using MySQL as a unified database
• Glance: Image management service, which provides management of images that can be provided during virtual machine deployment, including image import, format, and production of corresponding templates
• Nova: Computing management service, which provides Nova management of computing nodes and uses Nova-API for communication
• Neutron: Network management service, which provides network topology management of network nodes, and also provides Neutron management panel in Horizon
• Horizon: Console service, which provides management of all services of all nodes in the form of Web. This service is usually called DashBoard
  . 3. The extended management service includes five services: Cinder, Swift, Trove, Heat, and Centimeter
• Cinder: Provides Cinder-related management of storage nodes, as well as Cinder's management panel in Horizon
• Swift: Provides Swift-related management of storage nodes, as well as Swift's management panel in Horizon
• Trove: Provide Trove related to manage database nodes, and provide Trove management panel in Horizon
• Heat: Provides basic operations such as initialization of resources in the cloud environment, dependency processing, and deployment based on templates, as well as advanced features such as automatic shrinkage and load balancing.
• Centimeter: Provides monitoring of physical resources and virtual resources, records these data, analyzes the data, and triggers corresponding actions under certain conditions.
  Control nodes generally only need one network port for communication/management of each node

Network node architecture

The network node only contains Neutron services

• Neutron: Responsible for managing the communication between the private network segment and the public network segment, as well as managing the communication/topology between the virtual machine network, managing the fire protection on the virtual machine, etc. The
  network node contains three network ports.
• eth0: used to communicate with the control node
• eth1: used for communication with computing/storage nodes other than the control node
• eth2: used for communication between an external virtual machine and the corresponding network

Compute node architecture

The computing node includes three services: Nova, Neutron, and Telemeter
1. Basic services

• Nova: Provides virtual machine creation, operation, migration, snapshots and other services around virtual machines, and provides APIs to interface with control nodes, and the control node issues tasks.
  Neutron: provides communication services between computing nodes and network nodes

2. Extended service

• Telmeter: Provides a monitoring agent for computing nodes, and feeds back the status of virtual machines to the control node. It is an agent service of Centimeter

The compute node contains at least two network ports

• eth0: Communicate with the control node, and the controlled nodes are uniformly deployed
• eth1: communicate with network nodes and storage nodes

Storage node architecture

Storage nodes include Cinder, Swift and other services

• Cinder: Block storage service, which provides corresponding block storage. Simply put, it is a virtual disk that can be mounted on the corresponding virtual machine without being affected by file system and other factors. For virtual machines, this operation is For example, a new hard disk can be added, which can complete any operation on the disk, including mounting, unmounting, formatting, converting the file system, etc., most of which are used for space expansion when the virtual machine space is insufficient, etc.
• Swift: Object storage service, which provides corresponding object storage. Simply put, it is a virtual disk space where files can be stored, and only files can be stored in this space. Formatting and file system conversion are not allowed. Most of them are used Cloud disk/file
  storage node contains at least two network interfaces
• eth0: Communicate with the control node, accept the task of the control node, and deploy the controlled node uniformly
• eth1: Communicate with computing/network nodes to complete various tasks issued by the control node