Hangzhou Kaikola Education Technology Co., Ltd. is an online education company dedicated to providing learning tutoring for primary and middle school students. At present, the company's back-end service infrastructure mainly relies on Alibaba Cloud native, including computing, network, storage and Kubernetes services.
Technology selection background
2020 is a year for the company to grow and develop. The entire company team has expanded from a few hundred people to several thousand people now. When it is used intensively, there will basically be several thousand people operating in the background at the same time. Some internal back-office operating systems are built with PHP, and performance and business have gradually been unable to meet the company’s demand planning. In addition, the company’s development department has split the microservices now, and the main external service is Dubbo in java language. Clusters and back-end systems need to seamlessly connect to the Java Dubbo service, so PHP has gradually been unable to meet the needs of the company.
At that time, I also researched the Dubbo project of PHP. Since the project was basically unupdated and maintained, I passed it down. Later I was interested in the simple and high-performance go language, and then I paid attention to the Dubbo-go project. After a period of research, I found out Dubbo Go meets our business needs and the community is very active. Later, we decided to choose Dubbo-go as the back-end pc business framework.
Some students may also ask why not use gRPC, which has better cross-language support, because the initial RPC service clusters of many companies are built based on the Dubbo ecosystem. If the cost of changing the framework is too large, it will basically not be considered. Although gRPC The degree of cross-language support is better, but many things need to be made by yourself, such as service registration, service discovery, log monitoring, etc.
At that time, when I decided to choose Dubbo-go, there were some objections in the development department. Why not directly transfer to java. If you transfer to java, there will be no cross-language communication problem. The problem of transferring to java is that the entry cost is high, and it is for the entire company. As far as the technology stack is concerned, maintaining the diversity of languages can more calmly deal with future business changes. Go itself is a high-performance language that is not weaker than Java and is very suitable for microservice architecture.
The challenge
After determining the framework selection, the first task I received was to build a set of scaffolding that can quickly create business projects, and develop an RPC proxy service based on the HTTP protocol. The deployment needs to be connected to the company’s containerized deployment platform. They all started from scratch, and there is basically no information that can be used for reference on the Internet.
The first is to plan the structure of the Dubbo-go project and determine the project directory structure. After referring to Dubbo-go Demo and other Go projects, the project directory structure is finally determined. The following directory structure can be used as a reference.
In order to be consistent with the Java Service Registry, Dubbo-go uses the following components in project selection:
- Use zookeeper as the registry
- nacos as the configuration center
- Database orm uses gorm
- The message queue uses RocketMQ
In order to increase the efficiency of development, we can streamline the configuration before the provider service is initialized. Only the most basic configuration can be similar to the following. For the coding of the provider service, refer to the Dubbo-go demo.
The following is the main method code for service startup:
Dubbo-go RPC service gateway design
Generally using Dubbo, the provider side needs to expose interfaces and methods, and the consumer side needs to be very clear about the interface and method definitions used by the service, as well as the types of input parameters and other information. It also needs to be based on the API provided by the provider side, and both ends can Normal communication call.
However, the usage scenario of the gateway does not care about the detailed definition of the interface to be called. The gateway only pays attention to the method to be called, the parameters passed, and the ability to receive the return result. The basis for implementing the gateway proxy is the generalization of Dubbo/Dubbo-go. Invoking characteristics.
The following is the official demo given by Dubbo-go. After the generalized service is loaded, it takes 3 seconds to complete the call. However, in actual use, the service must not be loaded in real time to wait for 3 seconds, so the cache needs to be loaded when the gateway application starts. Good need for generalized services.
After studying the generalization of Dubbo-go and calling the demo, it is found that it is feasible to design a dubbo-go gateway with this feature. The difficulty is that we need to obtain and cache every parameter that requires the gateway proxy RPC service method and the path of the service. In this way, the generalized call service can be initialized before the call. The configuration of a service is as follows.
Because it is a gateway proxy made in go language, it is not possible to obtain the Java RPC service configuration through the Java jar package. If it is manually maintained, the workload is too large and error-prone, which is obviously unacceptable. After a period of understanding, Java services can achieve configuration acquisition through annotations. When the Java side adds annotations to the method and starts the service, the configuration information will be sent to MQ through messages, and the gateway consumes these messages to achieve the acquisition of Java RPC services. Configuration.
Dubbo Go's RPC service does not support annotations, so after thinking about it, I wrote a small tool that scans the code. I added corresponding annotations before each RPC service method, and obtained the configuration of the RPC service by scanning the annotations. After the configuration is obtained, the RPC service configuration is generated in the project directory, and the configuration is read and sent to MQ when the application is started.
After the gateway proxy is implemented, more functions can be implemented on the basis of the gateway, such as token verification, whitelisting, current limiting, fuse, and log monitoring. The effect of gateway proxy request is as follows:
Containerization department
The company’s internal containerized deployment environment is Alibaba Cloud’s K8s. Deploying to the K8s platform only needs to provide an image file. Since Dubbo-go is compiled as a binary file, it does not require any additional third-party libraries and can be stable in a Docker environment. run. There is a docker image file as shown in the figure below. You can use any linux distribution such as centos as the base image.
LABEL maintainer="<[email protected]>"
LABEL version="1.0"
LABEL description="KKL-GO-NKO-BASE"`
ARG envType=stable
#设置环境变量
ENV envType ${envType}
#编译打包好的压缩包
ADD ./target/nko-base-${envType}.tar.gz /app/
WORKDIR /app
EXPOSE 20000After the image is written, it is provided to the publishing platform. The publishing platform machine starts the image and decompresses the packaged file, and executes the Dubbo-Go program.
Container entrypoint set to [bash, -c, tar -zxf nko-base-stable.tar.gz && SERVER_ENV=kubernetes && sh ./nko-base/bin/load.sh start -group=stable]Since there are usually multiple deployment environments from development and testing to production, we need to modify the compilation script in the dubbo-go samples demo to support multi-environment packaging.
In addition, the default registered IP of Dubbo-go is the virtual IP of the K8s pod, and the networks between different K8s clusters cannot communicate with each other, so if you need to call across clusters, you need to modify the default registered IP, and change the default registered pod IP + port to The IP of the physical machine of Kubernetes plus the corresponding port. Kubernetes will write the IP of the physical machine plus the corresponding port environment variable in the pod. The application can obtain the IP and port of the physical machine by reading the environment variable. If you need to implement this function, you need to modify Dubbo. -go registration logic. For example, taking the zookeeper registration center as an example, we can modify the registered IP and port by extending the registerTempZookeeperNode method of registery/zookeeper/registry.go. The code is as shown in the figure below. The official Dubbo-go will support customization in the form of configuration in the later version. The function of registering IP and port.
func (r *zkRegistry) registerTempZookeeperNode(root string, node string) error {
...
regIp = os.Getenv(constant2.RegistryEnvIP) //实体机的ip
regPort = os.Getenv(constant2.RegistryEnvPort) //实体机的端口
urlNode, _ := common.NewURL(node)
role, _ := strconv.Atoi(urlNode.GetParam(constant.ROLE_KEY, ""))
if role == common.PROVIDER && regIp != "" && regPort != "" {
urlNode.Ip = regIp
urlNode.Port = regPort
node = url.QueryEscape(urlNode.String())
}
zkPath, err = r.client.RegisterTemp(root, node)
...
}Final words
If you encounter some problems while using dubbo-go, you can submit an issue or enter the community DingTalk group for communication.
Personal advice: If you are ready to enter Dubbo-go, it is best to solve these problems yourself and give official feedback to PR. Only when you encounter problems and solve problems can you grow and gain something, right?
About the author: Zeng Fanwei (github @jack15083), a front-line programmer with 9 years of service-side business development experience. He has worked as a back-end development engineer for many companies such as Tencent Reading. He is currently working in Hangzhou Kaikola Education Technology Co., Ltd. Go language service infrastructure and middleware and part of the business development work.
This article is the original content of Alibaba Cloud and may not be reproduced without permission.