The title picture was taken in downtown San Francisco: Twin Peaks in the sea of clouds
Not long ago, we published how to deploy a multi-node Fabric 1.0 cluster (clickable), mainly describing the steps for manual deployment. In this series, we will discuss how to automate the deployment of Fabric v1.0 on the most popular Kubernetes container platform today, so as to realize functions such as management and monitoring of the distributed blockchain platform.
Readers who follow this official account may find that the author mainly studies blockchain and cloud-native applications. The two seemingly unrelated fields are perfectly combined in this article, which confirms a sentence that the author often shares: There is always a time to meet teachers in technology!
During the writing of this article, Jia Yixing, an engineer in our team, gave suggestions and corrections to the Kubernetes part, and I would like to express my gratitude.
[Note: To download the PDF version of this article and the source code of this article, you can follow this official account: Henry Notes, and send a message "blockchain as a service " or " baas " in the background . 】
Overview
Looking forward, looking forward, Hyperledger Fabric 1.0 is officially here, and the community users are cheering for it: finally an enterprise-level blockchain application platform. However, after being excited and returning to calm, people often find that building a Fabric platform is a very difficult process. Not only must you have the theoretical foundations of blockchain such as cryptography, distributed computing, and consensus algorithms, but you must also be familiar with engineering technologies that are not commonly used by enterprise users such as containers, Golang/Node.js. s reason. Lowering the threshold for use and improving ease of use will be an important task to promote enterprise blockchain applications in the future.
Our previous article described the steps for installing and deploying a multi-node Fabric cluster, aiming to explain the basic operating principles of Fabric, so the deployment process is a manual (cao) manual (gen) installation method. In actual development and testing, automated deployment is needed to improve efficiency. This article introduces how to use the container platform Kubernetes (K8s) to automatically deploy Fabric 1.0 to realize the basis of Blockchain as a Service (BaaS).
It should be pointed out that BaaS is currently mostly used for development and testing, that is, deploying multiple blockchain nodes on the same BaaS platform, each node representing a different organization. This is obviously a centralized deployment method and can only be used for development and testing purposes. The deployment of the real environment requires multiple BaaS distributed in the network to work together, which is another work to be improved.
We chose to deploy Fabric on K8s for several reasons. First of all, the components of Fabric are packaged by containers, which is easy to deploy on container platforms such as K8s, and use the platform to achieve high availability, monitoring management, automated operation and maintenance and other purposes.
Secondly, Fabric is a distributed system. According to the specific requirements of the application, the number of components in the cluster will be different, which needs to be configured and adjusted flexibly. K8s is a container platform oriented to the microservice architecture, which is easy to expand and can well meet the requirements of Fabric.
In addition, K8s has the ability of multi-tenancy, which can run multiple isolated Fabric instances on the same platform, which is convenient for development and testing. For example, one instance is used for development and another instance is used for testing.
There is a sub-project in Hyperledger called Cello, whose purpose is to provide BaaS for Hyperledger. It is understood that the deployment of Fabric on Docker and Swarm has been supported, and the support for K8s is still under development. Because the characteristics of platforms such as K8s are not considered in the design of Fabric, some workarounds are required to deploy Fabric on K8s, which will be mentioned in the relevant parts later.
Overall structure
2.1 Infrastructure
1) Network part
A Kubernetes cluster consists of multiple nodes. In order for the containers on the cluster to communicate normally, an overlay network needs to be created and all the containers on the cluster are connected to this network.
Figure 2- 1
As shown in Figure 2-1, the host network is marked by a blue line, and the nodes have cmd clients, Kubernetes masters and workers, and NFS servers. Among them, the cmd client is used as the command line client for operating K8S and Fabric clusters. The NFS server is used to share various configuration files of Fabric and K8s among various nodes, and can also be replaced by other shared storage supported by K8s.
Overlay networks, such as Flannel, can be easily created through Kubernetes' cluster add-ons. As shown by the red line in Figure 2-1 (some details are omitted to illustrate the role of Flannel), Kubernetes adds all pods to the Flannel network, so the containers in the pod can communicate with each other.
The address segment of the Flannel network can be specified in the add-on configuration file, and the IP address of kube_dns can also be modified through configuration, but the IP address must be in the specified address segment. As shown in Figure 2-1, the network of Flannel is 10.0.0.1/16, and the IP address of kube_dns is 10.0.0.10.
In the Fabric design, chaincode currently runs as a Docker container on the host where the peer container is located. The peer container needs to call the interface of the Docker engine to build and create the chaincode container. The calling interface is through this connection:
unix:///var/run/docker.sock
This method actually has security risks, which will not be discussed too much here. The correct posture should be to call the dedicated running environment of chaincode, such as starting a new Docker Host and calling it remotely using the TCP interface.
The container created by docker.sock is outside the Kubernetes system. Although it is still on the Flannel network, it cannot obtain the IP address of the peer node. This is because the Docker engine that created the container uses the host's default DNS resolution to resolve the peer's domain name, so it cannot be found.
In order to solve the problem of resolving domain names, it is necessary to add relevant parameters to the DOCKER_OPTS of each worker. Taking Figure 2-1 as an example, the IP of kube_dns is 10.0.0.10, and the IP address of the host network DNS is assumed to be 192.168.0.1. The container of chaincode can be resolved to the peer node. The modification steps are as follows:
1. Edit /etc/default/docker and add the following parameters to DOCKER_OPTS to set the DNS used by Kubernetes (important!):
"--dns=10.0.0.10 --dns=192.168.0.1 --dns-search \
default.svc.cluster.local--dns-search svc.cluster.local \
--dns-opt ndots:2 --dns-opt timeout:2 --dns-opt attempts:2"
2. Run the following command to restart Docker (Note: the command may be different in different Linux environments):
systemctl daemon-reload
systemctl restart docker
systemctl restart docker.service
[Note: To download the PDF version of this article and the source code of this article, you can follow this official account: Henry Notes, and send a message "blockchain as a service " or " baas " in the background. 】
2) Shared storage
K8s and Fabric clusters require more configuration files. To facilitate management, these files can be stored uniformly through the NFS server, as shown in Figure 2-1.
The cmd client can generate the crypto-config directory through the cryptogen tool, which is used to store the configuration files of the nodes in the Fabric cluster. For example, the msp used by peer0.org1 is stored in the following directory:
crypto-config/PeerOrganization/org1/peers/peer0/msp
The cmd client only needs to mount the NFS shared directory to the local /opt/share/, and then write the crypto-config to this directory, so that each node can access the configuration file.
In Kubernetes, the files on NFS are mounted to the container through PV and PVC. In addition to creating the corresponding PV and PVC, it is also necessary to mount the correct path in the node's configuration file. If the shared directory of the NFS server is /opt/share, you can specify the mount point of peer.org1 when creating the PV:
/opt/share/crypto-config/PeerOrganization/org1/
Then create a PVC corresponding to that PV, so that the PVC can be used to mount this directory in the node's configuration file. The node needs to add the corresponding path after the mount point according to its own ID to ensure that the mounted configuration file is correct. For example, peer0.org1 should add peers/peer0/msp after the path, and the full path of its mount directory is as follows :
/opt/share/crypto-config/PeerOrganization/org1/peers/peer0/msp
2.2 Component division
In Kubernetes, Namespace is a very important concept, which is used to divide different virtual clusters, and the organization in Fabric is based on the division of certificate issuing agencies. The namespace of K8S corresponds to the organization of Fabric, which not only makes the organizations independent from each other, but also Taking full advantage of the DNS service of K8S, each organization can be distinguished by domain name. Using namespace to separate the components of each organization, it can also implement network policy (network policy) to isolate different organizations and realize the ability of multi-tenancy (not covered in this article).
Figure 2- 2
As shown in Figure 2-2, assuming that there are multiple peer organizations and orderer organizations in the Fabric network, the following describes how to divide and correspond in Kubernetes:
a. If the domain name of the peer organization of the Nth Fabric is orgN, its corresponding namespace on Kubernetes is set to orgN, and there are multiple pods under this namespace. The types of pods are as follows:
1) Peer Pod: Including Fabric peer, couchDB (optional), representing each organization's peer node.
2) CA Server Pod： Fabric CA Server。
3) CLI Pod: (Optional) An environment that provides command-line tools for manipulating the organization's nodes, channels, or chaincode.
b. If the domain name of the orderer organization of the Nth Fabric is orgordererN, its corresponding namespace on Kubernetes is orgordererN, and there is only one Pod under this namespace, which is used to run the orderer node.
c. Kafka namespace has nothing to do with Fabric's organization. It is only used to run and manage Zookeeper and Kafka containers to implement consensus algorithms.
2.3 Communication between Pods
Each Pod in Kubernetes has an independent IP address. However, it will bring a lot of trouble to communicate directly between Pods through IP:port. Therefore, it is necessary to bind a service to each Pod, so as to use the service name to access.
The naming method of the service should follow the following principles to reveal the Pod information bound to it:
1) The namespace of service and pod should be the same.
2) The name of the service should be the same as the id of the container in the Pod.
For example, the peer0 node belonging to org1 in Fabric runs with a Pod whose namespace is org1 and whose name is peer0 in K8S. The full name of the service bound to this Pod should be peer0.org1 . Where peer0 is the name of the service, and org1 is the namespace of the service. Such a mapping relationship is very close to the host domain name.
(To be continued)
Readers are welcome to continue to like and leave a message after the article, and tell us that you like this kind of article.
[Note: To download the PDF version of this article and the source code of this article, you can follow this official account: Henry Notes, and send a message "blockchain as a service " or " baas " in the background . 】
Related article: Hyperledger Fabric 1.0 Multi-Node Cluster Deployment
Introduction to "Blockchain Technology Guide"
For more information on Hyperledger, as well as the technical details of blockchain, including Bitcoin, Ethereum, public chains, consortium chains, side chains, Lightning Network, etc., please refer to the author and Dr. Zou Jun co-authored " Blockchain Technology Guide , Machinery Industry Press:
Jingdong purchase link:
http://item.jd.com/12007317.html
Readers are welcome to continue to like, leave a message and exchange after the article. Henry's notes mainly include technical articles on blockchain and cloud computing. Welcome to pay attention: