Article directory
Kubernetes(K8s)
一、Openstack&VM
1. Understand virtualization
1.1 , what is virtualization
In computers, virtualization (English: Virtualization) is a resource management technology that abstracts and converts various physical resources of the computer, such as servers, networks, memory, and storage, and presents them after conversion, breaking the gap between physical structures. Indivisible barriers allow users to apply these resources in a better way than the original configuration. The new virtual portion of these resources is independent of how existing resources are set up, geographically or physically configured. Commonly referred to as virtualized resources include computing power and data storage.
Virtualization technology is a set of solutions. A complete situation requires CPU, motherboard chipset, BIOS and software support, such as VMM software or some operating systems themselves. Even if only the CPU supports virtualization technology, in the case of VMM software, it will have better performance than a system that does not support virtualization technology at all.
In the actual production environment, virtualization technology is mainly used to solve the overcapacity of high-performance physical hardware and the reorganization and reuse of old hardware with low capacity, and transparentize the underlying physical hardware, so as to maximize the utilization of physical hardware and fully utilize resources. use
There are many types of virtualization technologies, such as: software virtualization, hardware virtualization, memory virtualization, network virtualization (vip), desktop virtualization, service virtualization, virtual machines, and so on.
1.2. Virtualization classification
(1) Fully virtualized architecture
The virtual machine monitor (hypervisor) is similar to user applications running on the host OS, such as VMware's workstation, which provides virtual hardware for virtualization products.
(2) OS layer virtualization architecture
(3) Hardware layer virtualization
The virtualization of the hardware layer has high performance and isolation, because the hypervisor runs directly on the hardware, which is beneficial to control the OS of the VM to access hardware resources. Products using this solution include VMware ESXi and Xen server
Hypervisor is an intermediate software layer running between the physical server and the operating system, which allows multiple operating systems and applications to share a set of basic physical hardware, so it can also be regarded as a "meta" operating system in a virtual environment. It can coordinate access to all physical devices and virtual machines on the server, also called a virtual machine monitor (Virtual Machine Monitor, VMM).
Hypervisor is the core of all virtualization technologies. When the server starts and executes the hypervisor, it allocates the appropriate amount of memory, CPU, network, and disk to each virtual machine, and loads the guest operating systems of all virtual machines. host machine
Hypervisor is the core of all virtualization technologies. The software and hardware architecture and management are more efficient and flexible, and the performance of hardware can be better utilized. Common products include: VMware, KVM, Xen, etc.
2、OpenStack与KVM、VMWare
2.1、OpenStack
OpenStack: Open Source Management Project OpenStack is an open source project designed to provide software for building and managing public and private clouds. It is not a piece of software, but a combination of several main components to complete some specific tasks. OpenStack consists of the following five relatively independent components:
· OpenStack Compute (Nova) is a set of controllers for virtual machine computing or starting virtual machine instances using groups;
· OpenStack image service (Glance) is a set of virtual machine image search and retrieval system to realize virtual machine image management;
· OpenStack object storage (Swift) is a set of object-based storage systems used in large-scale scalable systems through built-in redundancy and fault tolerance mechanisms, similar to Amazon S3;
· OpenStack Keystone for user identity services and resource management and
· OpenStack Horizon, a Django-based dashboard interface, is a graphical management front end. This open source project, originally jointly developed by NASA and Rackspace in late 2010, aims to create a cloud computing platform that is easy to deploy, rich in functions, and easy to expand. The primary task of the OpenStack project is to simplify the cloud deployment process and bring good scalability to it, trying to become the operating system of the data center, that is, the cloud operating system.
Openstack project hierarchy:
· The first layer is the infrastructure layer. This layer mainly includes Nova, Glance and Keystone. If we want to get the most basic infrastructure services, we must install and deploy these three projects.
The second layer is the extended infrastructure layer. This layer allows us to get more advanced services related to infrastructure, mainly including Cinder, Swift, Neutron, Designate, and Ironic, among which Cinder provides block storage and Swift provides object storage. , Neutron provides network services, Designate provides DNS services, and Ironic provides bare metal services.
· The third layer is an optional enhancement feature that helps users provide some more advanced functions, mainly including Ceilometer, Horizon and Barbican, where Ceilometer provides monitoring and metering services, Horizon provides user interface, and Barbican provides key management services.
· The fourth layer is mainly consumption-oriented services. The so-called consumption-type services mainly mean that the services of the fourth layer need to use the services of the first three layers to work.
The fourth layer mainly includes Heat, Magnum, Sahara, Solum, and Murano. Heat mainly provides orchestration services, Magnum mainly provides container services, and Sahara mainly provides big data services. We can easily deploy Hadoop and Spark clusters through Sahara. Solum mainly provides application development services, and can provide some functions similar to CI/CD. Muarno mainly provides application directory services, similar to the App Store, that is, users can publish some commonly used applications for other users to use. On the far right is Kolla. The main function of Kolla is to containerize all OpenStack services to facilitate OpenStack installation, deployment and upgrade.
2.2、KVM
KVM (Kernel-based Virtual Machine) Kernel-based virtual machine KVM is a Hypervisor integrated into the Linux kernel. It is a full virtualization solution for Linux with X86 architecture and hardware supporting virtualization technology (Intel VT or AMD-V). It is a small module of Linux, which uses Linux to do a lot of things, such as task scheduling, memory management and hardware device interaction. The biggest advantage of KVM is that it is integrated with the Linux kernel, so it is very fast.
2.3、VMWare
VMWare (Virtual Machine ware) VMWare (Virtual Machine ware) is a "Virtual PC" virtual machine management software. Its products can make you run two or more Windows, DOS, LINUX systems on one machine at the same time.
Compared with the "multi-boot" system, VMWare adopts a completely different concept. A multi-boot system can only run one system at a time, and the machine needs to be restarted when the system is switched. VMWare is truly "simultaneously" running, and multiple operating systems are switched on the platform of the main system just like standard Windows applications. Moreover, you can perform virtual partitioning and configuration for each operating system without affecting the data on the real hard disk. You can even connect several virtual machines to form a LAN through the network card, which is extremely convenient. The performance of installing the VMware operating system is much lower than that of the system directly installed on the hard disk, so it is more suitable for learning and testing.
2. Container & orchestration technology
1. History of container development
1.1、Chroot
The concept of container technology can be traced back to UNIX Chroot in 1979. This feature changes the Root directory and other subdirectories to a new location in the file system, and only accepts access to specific processes. It is designed to provide an isolated set of disk space for each process. It was added to the BSD in 1982.
Chroot only provides the function of virtualizing the process file directory, and cannot prevent the process from maliciously accessing the system. This problem is solved in FreeBSDGails container technology
1.2、FreeBSD Jails
FreeBSD Jails is similar to Chroot, but it includes a process sandbox mechanism to isolate resources such as file systems, users, and networks. In this way, it can provide a corresponding IP address for each Jail, customized software installation package, and even configuration scheme. Jails technology provides a simple security isolation mechanism for the FreeBSD system. Its shortcoming is that this simple isolation also affects the flexibility of applications in Jails to access system resources.
1.3、Solaris Zones
The Solaris Zone technology creates a virtual layer for applications, allowing applications to run in isolated zones and achieve effective resource management. Each Zone has its own file system, process space, firewall, network configuration and so on.
Solaris Zone technology really introduces the concept of container resource management. When the application is deployed, configure certain resources for the Zone. During operation, the resource limit can be dynamically modified according to the load of the Zone and it will take effect in real time. When other Zones do not need resources, the resources will be automatically switched to the Zone that requires resources. , this switching is immediate and does not require manual intervention, maximizing resource utilization, and can also isolate certain resources for a single Zone if necessary.
1.4、LXC
LXC refers to Linux Containers, and its functions are realized through Cgroups and Linux Namespaces. It is also the first complete Linux container management implementation. Before the appearance of LXC, there were already Linux-Vserver, OpenVZ and FreeVPS on Linux. While these technologies are mature, these solutions have yet to integrate their container support into the mainstream Linux kernel. Compared with other container technologies, LXC can run on the original Linux kernel without any additional patches. At present, the LXC project is sponsored and hosted by Canonical Co., Ltd.
1.5、Docker
The Docker project was originally created by a platform-as-a-service vendor called DotCloud, which later changed its name to Docker. Docker used LXC in its infancy and then replaced it with its own Libcontainer library. Unlike other container platforms, Docker introduces a whole ecosystem related to container management. These include an efficient layered container image model, a global and local container registry, a streamlined REST API, a command-line interface, and more.
Another container technology with the same target function as Docker is Rocket developed by CoreOS. Rocket is based on the App Container specification and makes it a more open standard.
2. Docker container
2.1 History of Docker
In 2010, a few young people engaged in IT established a company called "dotCloud" in San Francisco, USA.
This company mainly provides PaaS-based cloud computing technology services. Specifically, container technology related to LXC. LXC is Linux container virtualization technology (Linux container). Later, dotCloud simplified and standardized its own container technology and named it - Docker.
After the birth of Docker technology, it did not attract the attention of the industry. And dotCloud, as a small start-up company, is also struggling under the fierce competition. Just when they were about to lose their hold, the idea of "open source" popped up in their minds.
What is "open source"? Open source means open source code. That is to open the source code of the original internally confidential program to everyone, and then let everyone participate and contribute code and opinions.
Open Source, open source
Some software is open source from the beginning. There are also some software that cannot be mixed, and the creators do not want to give up, so they choose to open source. If you can't support yourself, just eat "hundreds of rice".
In March 2013, 28-year-old Solomon Hykes, one of the founders of dotCloud and the father of Docker, officially decided to open source the Docker project.
If you don't open it, it's nothing, it's amazing when you open it.
More and more IT engineers discovered the advantages of Docker, and then flocked to join the Docker open source community.
Docker's popularity has risen rapidly, and the speed is jaw-dropping.
In the month of open source, Docker version 0.1 was released. Every month thereafter, Docker releases a version. By June 9, 2014, Docker version 1.0 was officially released.
At this time, Docker has become the most popular open source technology in the industry, not one of them. Even giants like Google, Microsoft, Amazon, and VMware all favor it and express their full support.
After Docker became popular, dotCloud simply changed the company name to Docker Inc.
Why are Docker and container technology so popular? To put it bluntly, it is because it is "light".
2.2. Principle of Docker
A container is a lightweight virtualization technology because it lacks a hypervisor layer compared with a virtual machine. First look at the picture below, which briefly describes the startup process of a container.
注: hypervisor：一种运行在物理服务器和操作系统之间的中间层软件，可以允许多个操作系统和应用共享一套基础物理硬件。可以将hypervisor看做是虚拟环境中的“元”操作系统，可以协调访问服务器上的所有物理设备和虚拟机，所以又称为虚拟机监视器（virtual machine monitor）。
hypervisor是所有虚拟化技术的核心，非中断的支持多工作负载迁移是hypervisor的基本功能。当服务器启动并执行hypervisor时，会给每一台虚拟机分配适量的内存，cpu，网络和磁盘资源，并且加载所有虚拟机的客户操作系统。
At the bottom is a disk, and the image of the container is stored on the disk. The upper layer is a container engine, which can be docker or other container engines. The engine sends a request downward, such as creating a container, and at this time it runs the container image on the disk as a process on the host.
For containers, the most important thing is how to ensure that the resources used by this process are isolated and restricted, which are guaranteed by the two technologies of cgroup and namespace on the Linux kernel
2.3、NameSpace
Namespace is used for resource isolation. There are seven namespaces in the Linux kernel, and the first six are used in docker. The seventh cgroup namespace is not used in docker itself, but the cgroup namespace is implemented in the runC implementation.
1）第一个是 mout namespace。mout namespace 就是保证容器看到的文件系统的视图，是容器镜像提供的一个文件系统，也就是说它看不见宿主机上的其他文件，除了通过 -v 参数 bound 的那种模式，是可以把宿主机上面的一些目录和文件，让它在容器里面可见的；
2）第二个是 uts namespace，这个 namespace 主要是隔离了 hostname 和 domain；
3）第三个是 pid namespace，这个 namespace 是保证了容器的 init 进程是以 1 号进程来启动的；
4）第四个是网络 namespace，除了容器用 host 网络这种模式之外，其他所有的网络模式都有一个自己的 network namespace 的文件；
5）第五个是 user namespace，这个 namespace 是控制用户 UID 和 GID 在容器内部和宿主机上的一个映射，不过这个 namespace 用的比较少；
6）第六个是 IPC namespace，这个 namespace 是控制了进程兼通信的一些东西，比方说信号量；
7）第七个是 cgroup namespace，上图右边有两张示意图，分别是表示开启和关闭 cgroup namespace。用 cgroup namespace 带来的一个好处是容器中看到的 cgroup 视图是以根的形式来呈现的，这样的话就和宿主机上面进程看到的 cgroup namespace 的一个视图方式是相同的；另外一个好处是让容器内部使用 cgroup 会变得更安全。
3、D&K&O
3.1、Docker&KVM
VM uses Hypervisor virtualization technology to simulate hardware resources such as CPU and memory, so that a Guest OS can be established on the host machine, which is often said to install a virtual machine.
Each Guest OS has an independent kernel, such as Ubuntu, CentOS or even Windows, etc. Under such a Guest OS, each application is independent of each other, and VM can provide a better isolation effect. However, this isolation effect requires a certain price, because part of the computing resources need to be handed over to virtualization, which makes it difficult to make full use of existing computing resources, and each Guest OS requires a large amount of disk space, such as Windows The installation of the operating system requires 10~30G disk space, and Ubuntu also needs 5~6G, and this method is very slow to start. It is precisely because of the shortcomings of virtual machine technology that spawned container technology.
The container is for the process, so there is no need for a Guest OS, only an independent file system is required to provide the required file collection. All file isolation is at the process level, so startup times are faster than VMs and require less disk space than VMs. Of course, process-level isolation is not as good as expected, and the isolation effect is much worse than that of VM.
Generally speaking, compared with VMs, containers have their own advantages and disadvantages, so container technology is also developing in the direction of strong isolation.
Docker provides a container for programs to run while ensuring that these containers are isolated from each other. A virtual machine has a similar function, but it creates a complete operating system stack through the hypervisor. Unlike the way of virtual machines, Docker relies on the LXC (Linux Containers) technology that comes with Linux. LXC takes advantage of Linux's ability to isolate memory, CPU, and network for processes. The Docker image does not need to start a new operating system, so it provides a lightweight way to package and run programs. And Docker can directly access the hardware, making its I/O operations much faster than virtual machines.
doubt:
Docker can run directly on the physical server, which raises the question: Is it necessary to use OpenStack if Docker is already used?
Performance test comparison chart of Docker and KVM. As expected, the difference in the time to start KVM and Docker containers is very significant, and the difference in memory and CPU utilization is very large, as shown in the table below.
The huge performance difference between the two parties leads to the fact that under the same workload, KVM requires more CPU and memory resources, resulting in higher costs.
3.2、KVM&openstack
openstack is a cloud management platform, which itself does not provide virtualization functions. The real virtualization capabilities are provided by the underlying hypervisor (such as KVM, Qemu, Xen, etc.). The so-called management platform is just for convenience. If there is no openstack, you can also use virsh and virt-manager to realize the operation of creating a virtual machine, but the method of typing the command line requires a certain learning cost, which is not very friendly to ordinary users.
KVM is the lowest-level hypervisor, which is used to simulate the operation of the CPU. However, a user can complete the operation of the virtual machine on KVM and needs the support of network and peripheral I/O, so we borrowed from qemu to make certain modifications. Form qemu-kvm. But openstack will not directly control qemu-kvm, and will use a libvirt library to indirectly control qemu-kvm. The status of qemu-kvm is like the underlying driver.
OpenStack: Open Source Management Project
OpenStack is an open source project designed to provide software for building and managing public and private clouds. It is not a piece of software, but a combination of several main components to complete some specific work. OpenStack consists of the following five relatively independent components:
-
OpenStack Compute (Nova) is a set of controllers for virtual machine computing or using groups to start virtual machine instances;
-
OpenStack image service (Glance) is a set of virtual machine image search and retrieval system to realize virtual machine image management;
-
OpenStack object storage (Swift) is a set of object-based storage systems for large-scale scalable systems through built-in redundancy and fault tolerance mechanisms, similar to Amazon S3;
-
OpenStack Keystone for user identity services and resource management and
-
OpenStack Horizon, a Django-based dashboard interface, is a graphical management front-end.
This open source project, originally jointly developed by NASA and Rackspace in late 2010, aims to create a cloud computing platform that is easy to deploy, rich in functions, and easy to expand. The primary task of the OpenStack project is to simplify the cloud deployment process and bring good scalability to it, trying to become the operating system of the data center, that is, the cloud operating system.
KVM: Open Virtualization Technology
KVM (Kernel-based Virtual Machine) is an open source system virtualization module that requires hardware support, such as Intel VT technology or AMD V technology. It is a complete virtualization based on hardware and is completely built into Linux.
In 2008, Red Hat acquired Qumranet to obtain KVM technology and promoted it as part of the virtualization strategy. When RHEL6 was released in 2011, it supported KVM as the only hypervisor. KVM focuses on high performance, scalability, high security, and low cost.
Fate with Linux
A Linux, dubbed the cloud era by some enthusiastic supporters, is an open source operating system for public and private clouds. One is a part of the Linux kernel that converts Linux into a Type-1 hypervisor that can enjoy the existing Linux kernel process scheduling, memory management, and device support without any changes.
OpenStack is hot. Like Linux, it aims to build a kernel around which all software vendors work. Many sub-projects of OpenStack provide agile management of various resources (such as computing power, storage, and network) in the cloud computing platform. In addition, OpenStack also provides support for virtualization technology.
KVM is integrated in each major distribution version of Linux and is managed by Linux's own scheduler. KVM focuses on being the best virtual machine monitor, and it is the best choice for enterprises using Linux. In addition, it also supports the Windows platform, so it is also the best choice for heterogeneous environments.
Both OpenStack and KVM are developing rapidly
OpenStack is a big project with many supporters. Today, more than 180 companies and more than 400 developers have actively contributed to this project, and its ecosystem is even larger, with more than 5,600 people and 850 institutions. In September of this year, the OpenStack Foundation was formally established. Platinum members include Red Hat, IBM, and HP, and gold members include Cisco, Dell, and Intel.
OpenStack is basically a software project with nearly 550,000 lines of code. Break down into core projects, incubation projects, and supporting projects and related projects. In addition to the five major components mentioned above, Quantum related to the virtual network is listed as a core project for the first time.
KVM is an open virtualization technology that stands out. It is jointly developed by a large and active open community, including Red Hat, IBM, SUSE and others. In 2011, IBM, Red Hat, Intel and HP established the Open Virtualization Alliance (OVA) to help build the KVM ecosystem and increase the adoption rate of KVM. Today, OVA has more than 250 member companies, among which IBM has more than 60 programmers working exclusively in the KVM open source community.
3.3、Docker&openstack
There is a good complementary relationship between OpenStack and Docker. The emergence of Docker can make full use of resources at the IaaS layer, because Docker is lighter than virtual machines,
The utilization rate of resources will be more fully;
The cloud platform provides a complete data center management solution. As for which hypervisor or container to use, it is only a small part of the cloud platform. A cloud platform like OpenStack includes multi-tenant security, isolation, management, monitoring, storage, networking and other parts. The management of cloud data centers requires a lot of service support, but this has nothing to do with whether Docker or KVM is used.
Docker is not a full-featured VM, it has many serious flaws, such as security, Windows support, so it cannot completely replace KVM. Now the Docker community has been making up for these shortcomings, of course, this will bring a certain performance loss.
4. Container Orchestration
4.1. Changes in application deployment
4.2. Container Management
How to manage so many containers
• How to scale horizontally
• The container is down, how to recover
• Does not affect business after container update
• How to monitor containers
• How to schedule newly created containers
• Data security issues
4.3. Cloud Native
Cloud native is the best path or best practice. In more detail, cloud native designates an optimal path for users that is low mental burden, agile, and can maximize the use of cloud capabilities and value in a scalable and reproducible manner.
The technical scope of cloud native includes the following aspects:
-
The first part is the cloud application definition and development process. This includes application definition and mirroring, configuring CI/CD, messaging and streaming, and databases.
-
The second part is the orchestration and management process of cloud applications. This is also a part of Kubernetes' focus, including application orchestration and scheduling, service discovery governance, remote invocation, API gateway, and Service Mesh.
-
The third part is monitoring and observability. This part emphasizes how to monitor, log collect, and trace applications on the cloud, and how to implement destructive testing on the cloud, which is the concept of chaos engineering.
-
The fourth part is the underlying technology of cloud native, such as container runtime, cloud native storage technology, cloud native network technology, etc.
-
The fifth part is the cloud-native tool set. In addition to these core technical points, there are many supporting ecological or peripheral tools that need to be used, such as process automation and configuration management, container mirror warehouse, cloud-native security technology, and cloud passwords. management etc.
-
Finally, there is Serverless. Serverless is a special form of PaaS, which defines a more "extremely abstract" application writing method, including concepts such as FaaS and BaaS. Whether it is FaaS or BaaS, its most typical feature is Pay as you go, so Serverless billing is also an important knowledge and concept.
4.4、Swarm
At present, the three mainstream container platforms Swarm, Mesos and Kubernetes have different container scheduling systems;
Swarm is characterized by direct scheduling of Docker containers and provides an API consistent with the standard Docker API.
Docker is installed on each server and the HTTP-based DockerAPI is enabled. There is a SwarmManager manager in this cluster to manage container resources in the cluster. The management object of the manager is not at the server level but at the cluster level. That is to say, through the Manager, we can only issue instructions to the cluster in general and cannot specify what to do on a specific server (this is also the root of Swarm). As for the specific implementation of management, the Manager exposes an HTTP interface, through which external users can manage the cluster.
4.5、Mesos
Mesos uses a relatively independent scheduling system for different running frameworks, and its framework provides native support for Docker containers. Mesos is not responsible for scheduling but for delegating authorization. After all, many frameworks have already implemented complex scheduling.
4.6、Kubernetes
Kubernetes uses concepts such as Pod and Label to combine containers into logical units that depend on each other. Related containers are combined into Pods and then deployed and scheduled together to form a Service. This is the main difference between Kubernetes, Swarm, and Mesos.
Kubernetes (k8s) is an open-source platform for automating container operations, including deployment, scheduling, and node-cluster scaling. If you have ever used Docker container technology to deploy containers, you can think of Docker as a low-level component used internally by Kubernetes. Kubernetes supports not only Docker, but also Rocket, another container technology.
With Kubernetes you can:
-
Automate container deployment and replication
-
Expand or shrink container size at any time
-
Organize containers into groups and provide load balancing between containers
-
Easily upgrade to new versions of application containers
-
Provide container resiliency, replace the container if it fails, etc...
3. Kubernetes
1. Borg system
Borg. Google's Borg system runs more than hundreds of thousands of tasks, from thousands of different applications, across multiple clusters, each cluster (cell) has tens of thousands of machines. It achieves high utilization through administrative control, efficient task packing, oversubscription, and process-level performance isolation. It supports high-availability applications with runtime capabilities, scheduling policies that minimize failure recovery time, and reduce the probability of associated failures. The following is the system architecture diagram of Borg. Among them, Scheduler is responsible for task scheduling.
2. Basic introduction of k8s
Just when the Docker container technology was in full swing, everyone found that if you want to apply Docker to specific business implementations, there are difficulties—arrangement, management, scheduling and other aspects are not easy. Therefore, people urgently need a management system for more advanced and flexible management of Docker and containers. At this time, K8S appeared.
*K8S * is a container-based cluster management platform. Its full name is kubernetes**
3. Main functions of k8s
Kubernetes is a tool used by docker containers for orchestration and management. It builds a container scheduling service based on Docker, providing resource scheduling, balanced disaster recovery, service registration, dynamic scaling and other functional suites. Kubernetes provides functions such as application deployment, maintenance, and expansion mechanisms. Using Kubernetes can easily manage cross-machine running containerized applications. Its main functions are as follows:
-
Data volume: Data is shared between containers in the Pod, and data volumes can be used.
-
Application health check: The service in the container may be blocked and unable to process requests. Monitoring and checking policies can be set to ensure the robustness of the application.
-
Replicating application instances: The controller maintains the number of Pod replicas, ensuring that a Pod or a group of Pods of the same type is always available.
-
Elastic Scaling: Automatically scales the number of Pod copies according to the set indicator (CPU utilization).
-
Service discovery: Use environment variables or DNS service plug-ins to ensure that the program in the container discovers the access address of the Pod entry.
-
Load balancing: A group of Pod copies are assigned a private cluster IP address, and load balancing forwards requests to backend containers. Other Pods in the cluster can access the application through this ClusterIP.
-
Rolling update: The update service is not interrupted, and one Pod is updated at a time instead of deleting the entire service at the same time.
-
Service Orchestration: Deploying services through file descriptions makes application deployment more efficient.
-
Resource monitoring: The Node node component integrates the cAdvisor resource collection tool, which can summarize the resource data of the entire cluster node through Heapster, then store it in the InfluxDB time series database, and then display it by Grafana.
-
Provide authentication and authorization: support attribute access control (ABAC), role access control (RBAC) authentication and authorization strategies.
4. k8s cluster
This cluster mainly consists of two parts:
-
A Master node (master node)
-
A group of Node nodes (computing nodes)
It is clear at a glance: the Master node is mainly responsible for management and control. The Node node is a workload node, which contains specific containers.
5. Master node
The Master node includes API Server, Scheduler, Controller manager, etcd. API Server is the external interface of the entire system, which is called by clients and other components, which is equivalent to a "business hall". The Scheduler is responsible for scheduling the resources within the cluster, which is equivalent to a "scheduling room". Controller manager is responsible for managing the controller, which is equivalent to the "big manager".
6. node node
Node nodes include Docker, kubelet, kube-proxy, Fluentd, kube-dns (optional), and Pod
7、k8s master
7.1、api server
Kubernetes API Server: Kubernetes API, the unified entrance of the cluster, the coordinator of each component, provides interface services with HTTP API, and all object resource additions, deletions, changes, queries, and monitoring operations are handed over to APIServer for processing and then submitted to Etcd for storage.
7.2、managerController
Kubernetes Controller: 处理集群中常规后台任务，一个资源对应一个控制器，而ControllerManager就是负责管理这些控制器的各个资源控制器对于如下：
1）Replication Controller: 管理维护Replication Controller，关联Replication Controller和Pod，保证Replication Controller定义的副本数量与实际运行Pod数量一致。
2）Node Controller: 管理维护Node，定期检查Node的健康状态，标识出(失效|未失效)的Node节点。
3）Namespace Controller: 管理维护Namespace，定期清理无效的Namespace，包括Namesapce下的API对象，比如Pod、Service等。
4）Service Controller: 管理维护Service，提供负载以及服务代理。
5）EndPoints Controller: 管理维护Endpoints，关联Service和Pod，创建Endpoints为Service的后端，当Pod发生变化时，实时更新Endpoints (即Pod Ip + Container Port)。
6）Service Account Controller: 管理维护Service Account，为每个Namespace创建默认的Service Account，同时为Service Account创建Service Account Secret。
7）Persistent Volume Controller: 管理维护Persistent Volume和Persistent Volume Claim，为新的Persistent Volume Claim分配Persistent Volume进行绑定，为释放的Persistent Volume执行清理回收。
8）Daemon Set Controller: 管理维护Daemon Set，负责创建Daemon Pod，保证指定的Node上正常的运行Daemon Pod。
9）Deployment Controller: 管理维护Deployment，关联Deployment和Replication Controller，保证运行指定数量的Pod。当Deployment更新时，控制实现Replication Controller和 Pod的更新。
10）Job Controller: 管理维护Job，为Jod创建一次性任务Pod，保证完成Job指定完成的任务数目
11）Pod Autoscaler Controller: 实现Pod的自动伸缩，定时获取监控数据，进行策略匹配，当满足条件时执行Pod的伸缩动作
7.3、etcd
etcd is a distributed and reliable key-value storage system, which is used to store key data in distributed systems, and this definition is very important.
etcd is a third-party service, distributed key-value storage system. Used to maintain cluster status, such as Pod, Service and other object information
etcd is a highly available distributed key-value (key-value) database. etcd internally uses the raft protocol as the consensus algorithm, and etcd is implemented based on the Go language. Etcd is a very important component in the Kubernetes cluster, which is used to save all network configurations and object status information of the cluster. There are two services in the entire kubernetes system that need to use etcd for collaboration and storage configuration, which are:
1) The network plug-in flannel, and other network plug-ins also need to use etcd to store network configuration information
2) kubernetes itself, including the state and meta-information configuration of various objects
7.4、scheduler
Select a Node node for the newly created Pod according to the scheduling algorithm. The scheduler assumes the important function of linking the previous and the next in the whole system. The linking means that it is responsible for receiving the controller manager to create a new Pod, and arranges a target Node for it to settle down. The linking means that after the placement work is completed, the kubelet service on the target Node The process takes over the subsequent work.
That is to say, the role of the scheduler is to select the most suitable Node from the Node list for each Pod on the Pod list to be scheduled through the scheduling algorithm.
8、k8s node
8.1、kubelet
Kubelet is the Agent of the Master on the Node node. Each node will start the kubelet process to handle the tasks sent by the Master node to the node, and manage the life cycle of the running container on the machine, such as creating containers, mounting data volumes on Pods, Download secrets, get container and node status, etc. The kubelet turns each Pod into a set of containers.
1. Kubelet listens to four ports by default, which are 10250, 10255, 10248, and 4194
-
10250 (kubelet API): The port through which the kubelet server communicates with the apiserver. It periodically requests the apiserver to obtain the tasks it should handle. Through this port, you can access and obtain node resources and status.
-
10248 (health check port): By visiting this port, you can judge whether the kubelet is working normally, and specify the listening address and port through the kubelet startup parameters --healthz-port and --healthz-bind-address.
-
4194 (cAdvisor monitoring): through this port, the kublet can obtain the environment information of the node and the status of the container running on the node. Visit http://localhost:4194 to see the management interface of cAdvisor. Through the startup parameters of kubelet,
--cadvisor-portyou can Specifies the port to start on. -
10255 (readonly API): Provides pod and node information, the interface is exposed in a read-only form, and access to this port does not require authentication and authentication.
8.2、be a proxy
Implement the Pod network proxy on the Node node, maintain network rules and four-layer load balancing work, kube-proxy is similar to a reverse proxy in essence. We can regard the kube-proxy running on each node as a transparent proxy of the service and concurrently LB.
kube-proxy monitors the service and Endpoint information in apiserver, configures iptables rules, and forwards requests directly to pods through iptables
8.3、docker
The engine that runs the container.
8.4、pod
Pod is the smallest deployment unit. A Pod consists of one or more containers. The containers in the Pod share storage and network and run on the same Docker host.
1) Pod basic structure
The role of Pause:
Let's see that there will be many pause containers on the node node, which correspond to the pod one by one.
Each Pod runs a special container called Pause, and other containers are business containers. These business containers share the network stack and volume mounted volume of the Pause container, so communication and data exchange between them are more efficient. , we can make full use of this feature at design time to put a group of closely related service processes into the same Pod. Containers in the same Pod can communicate with each other only through localhost.
The pause container in kubernetes mainly provides the following functions for each business container:
PID Namespace: Different applications in a Pod can see the process IDs of other applications.
Network Namespace: Multiple containers in a Pod can access the same IP and port range.
IPC Namespace: Multiple containers in a Pod can communicate using SystemV IPC or POSIX message queues.
UTS namespace: Multiple containers in the Pod share a hostname; Volumes (shared storage volume):
Individual containers in a Pod can access Volumes defined at the Pod level.
2) Controller pods
-
ReplicationController & ReplicaSet & Deployment
ReplicationController
It is used to ensure that the number of replicas of the container application is always kept at the user-defined number of replicas, that is, if a container exits abnormally, a new Pod will be automatically created to replace it, and if there are abnormally many containers, it will be automatically recycled.
In the new version of Kubernetes, it is recommended to use ReplicaSet to replace ReplicationController
ReplicaSet
There is no essential difference between ReplicaSet and ReplicationController, but the name is different, and ReplicaSet supports a collection of selectors
Although ReplicaSet can be used independently, it is generally recommended to use Deployment to automatically manage ReplicaSet, so that there is no need to worry about incompatibility with other mechanisms (for example, ReplicaSet does not support rolling-update but Deployment supports it)
Deployment provides a declarative definition method for Pod and ReplicaSet, which is used to replace the previous ReplicationController to manage applications conveniently.
Typical application scenarios:
(1), define Deployment to create Pod and ReplicaSet
(2), rolling upgrade and rollback application
(3), expansion and cable capacity
(4), suspend and continue Deployment
Deployment can not only be updated by rolling, but also can be rolled back. How to find out that the service is unavailable after upgrading to the V2 version, and you can roll back to the V1 version.
HPA(HorizontalPodAutoScale)
Horizontal Pod Autoscaling is only applicable to Deployment and ReplicaSet. In the V1 version, it only supports expansion based on the CPU utilization of the Pod. In the vlalpha version, it supports expansion and contraction based on memory and user-defined metrics.
- StatefullSet
StatefullSet is to solve the problem of stateful services (corresponding to Deployments and ReplicaSets are designed for stateless services), and its application scenarios include:
(1) Stable persistent storage, that is, the same persistent data that can still be accessed after the Pod is rescheduled, based on PVC
(2) A stable network flag, and its PodName and HostName remain unchanged after the Pod is rescheduled, based on the Headless Service (that is, the Service without Cluster IP).
(3) Orderly deployment and orderly expansion, that is, Pods are ordered, and when deploying or expanding, they must be carried out in sequence according to the defined order (that is, from 0 to N-1, all previous Pods must be Both are in the Running and Ready states), based on init containers.
(4) Orderly shrink, orderly delete (ie from N-1 to 0)
解析：
有状态：需要实时的进行数据更新及存储，把某个服务抽离出去，再加入回来就没有办法正常工作，这样的服务就是有状态服务。
例如： mysql，Redis....
无状态：docker主要面对的是无状态服务，所谓的无状态服务就是没有对应的存储数据需要实时的保留，或者是把服务摘除后经过一段时间运行后再把服务放回去依然能够正常运行，就是无状态服务。
例如： Apache、lvs
-
DaemonSet
DaemonSet ensures that all (or some [nodes are tainted (can be imagined as a label), if the pod is not defined to tolerate this taint, then the pod will not be assigned to this node by the scheduler]) a copy of the Pod runs on the Node. When a Node joins the cluster, a Pod is also added for them. These Pods are also recycled when a Node is removed from the cluster. Deleting a DaemonSet will delete all Pods it created, using some typical usages of DaemonSet:
(1) Run the cluster storage daemon, for example, run glustered and ceph on each Node
(2) Run the log collection Daemon on each Node, for example: fluentd, logstash.
(3) Run the monitoring Daemon on each Node, for example: Prometheus Node Exporter
Job is responsible for batch tasks, that is, tasks that are executed only once, and it guarantees that one or more Pods of the batch task end successfully
Cron Job management is based on time Job, namely:
- run only once at a given point in time
- run periodically at a given point in time
9. Other plug-ins
CoreDNS: It can create a domain name IP correspondence analysis for the SVC in the cluster
DASHBOARD: Provide a B/S structure access system for the K8S cluster.
INGRESS CONTROLLER: Officially, only four-tier proxy can be realized, while INGRESS can realize seven-tier proxy
. FEDERATION: Provide a Multi-K8S unified management function across the cluster center
PROMETHEUS: Provide K8S cluster monitoring capability
ELK: Provide K8S cluster log unified analysis intervention platform
-
DaemonSet
DaemonSet ensures that all (or some [nodes are tainted (can be imagined as a label), if the pod is not defined to tolerate this taint, then the pod will not be assigned to this node by the scheduler]) a copy of the Pod runs on the Node. When a Node joins the cluster, a Pod is also added for them. These Pods are also recycled when a Node is removed from the cluster. Deleting a DaemonSet will delete all Pods it created, using some typical usages of DaemonSet:
(1) Run the cluster storage daemon, for example, run glustered and ceph on each Node
(2) Run the log collection Daemon on each Node, for example: fluentd, logstash.
(3) Run the monitoring Daemon on each Node, for example: Prometheus Node Exporter
Job is responsible for batch tasks, that is, tasks that are executed only once, and it guarantees that one or more Pods of the batch task end successfully
Cron Job management is based on time Job, namely:
- run only once at a given point in time
- run periodically at a given point in time