This article comes from an interview with Xinming He from Open Source China, first published in Open Source China
Go is currently not the mainstream in Ctrip, but it has gathered a group of loyal fans. Relying on Ctrip's internal Golang community, it has continued to grow and develop. It has already had a large-scale project practice, and its development momentum is very rapid.
Recently, based on the early projects of the open source community dubbo-go, Ctrip refactored and developed the dubbo-go v1.0.0 version, which is easier to extend and more complete, and open sourced, re-contributing back to the community.
If you need more interview materials from major companies, you can also click to enter directly and get it for free! Password: CSDN
background
On May 21st, after more than a year of incubation, Apache Dubbo graduated from the Apache Software Foundation and became a top-level Apache project.
Dubbo is a high-performance RPC framework open sourced by Ali in 2011, and it has considerable influence in the Java ecosystem. At first, I experienced a gloomy period of "stop maintenance" that was criticized by the outside world. Later, in 2017, Dubbo's prodigal son returned and officially announced that it would re-focus on maintenance.
The relaunched Dubbo aims to reactivate the community, win back the trust of developers, and gradually build Dubbo into an international and modern project. It has been one and a half years since the announcement of the restart.
In this process, Dubbo released multiple versions and gradually transformed from an RPC framework to a microservice ecosystem. In early 2018, Dubbo entered the Apache Software Foundation incubator and began to develop the community based on the Apache approach.
A year later, Dubbo released the first milestone version 2.7.0 since the restart of maintenance in the Apache incubator, adding features such as asynchronous support that the community is calling for, and the separation of registry and configuration centers.
During this period, the development of Dubbo 3.0 was also put on the agenda. In mid-April this year, the official announcement of the progress of Dubbo 3.0. The new features of this version include support for the asynchronousization of the Filter chain, responsive programming, and cloud native/Service Mesh direction The exploration and integration with Ali inside and outside.
Then Dubbo graduated. What's the news about Dubbo after graduation?
The ecology is still developing. The Dubbo community released Dubbo Roadmap 2019 a few days ago, and plans to release the official version of Dubbo 3.0 in February 2020. Interested students can check it out in detail.
Recently, it was officially announced that the Go language had joined the Dubbo ecosystem, and the dubbo-go project was released.
Prior to this, Dubbo's cross-language scalability has been implemented. The supported languages include PHP, Node.js and Python. At the same time, it also implements REST call support based on the standard Java REST API-JAX-RS 2.0. The details are as follows:
- PHP: php-for-apache-dubbo, by Lexin, providing client and server
- Node.js: dubbo2.js, by kilometer.com, providing client
- Node.js: egg-dubbo-rpc, by Ant Financial's egg team, providing client and server
- Python: py-client-for-apache-dubbo, by kilometer.com, providing client
Now that dubbo-go is added, Go developers can finally taste the taste of Dubbo.
It is reported that the dubbo-go project will complete the migration to the Apache Software Foundation this week, as a sub-project of the top-level Apache Dubbo project. By then, the new address of the dubbo-go project will also become: https://github.com/apache /dubbo-go.
If you need more interview materials from major companies, you can also click to enter directly and get it for free! Password: CSDN
Open Source China Interview Record
Regarding the R&D background and specific technical details of the dobbo-go project, Open Source China interviewed the co-sponsor of the project, He Xinming, Ctrip Basic Central and Taiwan R&D Department.
Question: What is dubbo-go, what is its positioning, and why this project?
Answer: dubbo-go is a complete Go language implementation of Dubbo.
We know that Dubbo itself is based on Java, and many companies also focus on Java development and use Dubbo as RPC or microservice development framework.
Recently, the Go language ecology has developed rapidly. Due to its language advantages, some of our departments have begun to try to use Go to develop some new projects. There will be problems that need to be solved:
- How to realize the intercommunication between Go project and Java & Dubbo project?
- The Go project itself also has demands for RPC and microservice development frameworks. How to solve them?
Based on these two issues, the Ctrip team refactored and developed a dubbo-go v1.0.0 version that is easier to extend and more complete based on the early project of dubbo-go, and contributed it back to the community.
Its primary purpose is to solve the intercommunication problem between the Go project and the Java & Dubbo project, and it also provides a choice of RPC and microservice development frameworks for the Go project.
dubbo-go provides client and server sides. Currently, the dubbo-go community is one of the most active communities in the Dubbo ecosystem. The subsequent positioning needs to meet the official requirements of Dubbo and the needs of community users.
Question: We know that Dubbo has very high achievements in the Java ecosystem, and the current Go ecosystem itself also has some well-known microservice frameworks. Does dubbo-go have the ability to compete with other frameworks in the Go ecosystem?
Answer: Our greatest ability is to serve as the Go language version of Dubbo, which opens up the gap between the two languages, makes Dubbo closer to the cloud native, provides developers with the greatest flexibility, and significantly reduces the number of companies' existing services going to the cloud. In the cloud-native era, enterprises have one more choice.
Question: Are the features of Go reflected in dubbo-go? (For example, how did Go's high concurrency transform from Java-based Dubbo to dubbo-go?)
Answer: My understanding of the Go language is that first of all, the learning cost is relatively small. Compared with the learning cost of Java, the Go language is easier to learn and use.
Secondly, at the language level, Go has the advantages of simple, efficient, and lightweight coroutines in the high-concurrency processing of its CSP programming model. Compared with Java programs based on JVM, Go programs based on runtime have the ability to start instantaneously. Attracting many developers, I won't elaborate on it here.
Finally, it is the advantage of being a cloud native language. With the emergence of excellent projects such as Docker, k8s and Istio, the underlying cloud native is basically unified by the Go language. I believe that the days for enterprises to develop in the cloud native mode are not far away. I think the ecology of the Go language should get better and better, and more and more people will use it.
Introducing Java-based Dubbo into Go, as mentioned earlier, the advantage brought by dubbo-go is that it can quickly integrate into the cloud-native field. To talk about the characteristics of the Go language, you can refer to the design of the asynchronous network I/O model in dubbo-go. This part reflects the advantages of the Go language lightweight coroutine.
Here are also the deficiencies of the Go language:
- Compared to Java, Go is still a very young language, and there is no template library available, so the community has paid a high development cost in writing and maintaining the Hessian 2 protocol library;
- Compared with Java's try/catch error handling method, Go's error handling capability is weaker;
- The overall ecology is still not as good as Java, for example, there is no powerful network I/O library like Netty.
Why do you mention this, because Dubbo itself uses the official Java libraries of Netty and Hessian 2 protocol, and dubbo-go did not have these when it was started, which made dubbo-go very difficult all the way, but the community eventually Overcome them, and additionally contributed to the open source Getty and Hessian2 projects.
Special thanks to Yu Yu, the early organizer of the dubbo-go community. The early version of the project was developed in 2016 with the support of his leader Hu Changcheng and his colleague Liu Weisan. His contributions to the Hessian2 and Getty projects are also the latest version of dubbo-go. Lay a solid foundation.
Question: Not long ago, Dubbo announced that it would emphasize Service Mesh in 3.0 after it was announced. This is language-agnostic. Is it necessary for dubbo-go to join the ecosystem at this time?
Answer: Service Mesh is indeed a general direction for the future development of microservices, but at this stage, large domestic companies have not seen very successful cases. Many small and medium-sized companies have not yet completed their own microservices or even started.
At present, the dubbo-go community gives priority to solving the problems encountered in the implementation of this type of enterprise microservice technology, focusing on supplementing related functions, optimizing overall performance and solving bugs. As for the future, I believe that with the exploration of Dubbo Mesh in the field of Service Mesh, dubbo-go will definitely follow up and play an important role.
Q: What is the update relationship between dubbo-go and Dubbo? Is it a synchronous update feature or some innovation of its own?
Answer: The latest version we are releasing is v1.0.0. We will specify the compatible Dubbo version every time we release a new version. Therefore, dubbo-go needs to be compatible with the functions corresponding to the Dubbo version number, and some Dubbo features will be updated synchronously.
Q: What are the interesting features of the newly released version?
Answer: The currently released version v1.0.0 supports the following functions:
- Role: Consumer(√), Provider(√)
- Transmission protocol: HTTP(√), TCP(√)
- Serialization protocol: JsonRPC v2(√), Hessian v2(√)
- Registration Center: ZooKeeper(√)
- Cluster strategy: Failover (√)
- Load balancing: Random(√)
- Filter: Echo Health Check(√)
- extension mechanism
The dubbo-go v1.0.0 version is mainly maintained by me and my colleague Fang Yincheng on Ctrip. Community members Zhou Ziqing and Gao Singer participated in the contribution. This version follows Dubbo's code layered decoupling design. The main functions of Dubbo 2.6.x will gradually be implemented in dubbo-go, including Dubbo's SPI-based code expansion mechanism. Dubbo-go also has a corresponding extension mechanism corresponding to it.
In the future, we will gradually introduce more implementations of current scalable modules, such as adding more Loadbalance load balancing, Cluster Strategy cluster strategy implementation (currently these tasks are actively claimed by community partners, and hope that more Go language enthusiasts can Join the community to contribute); Another example is k8s, which is very popular in the cloud native field. We will also synchronize Dubbo's roadmap and follow up with k8s as the support of the registration center. Currently, community member Zhang Haibin is responsible for the follow-up.
Of course, the majority of developers can also implement new implementations of these module interfaces and expand them through extensions to fulfill their special needs without modifying the source code. At the same time, we welcome developers to contribute useful extensions to the community.
This version solves a major problem: a solution that interoperates with the Dubbo Java version. We extracted this part of the Hessian2 project, which originated from the early contributions of the community Yu Yu, and is now maintained by the community member Wang Ge, with Zhou Ziqing and Gao Singh participating in the contribution.
At present, the project has completed compatible support for most types of Java. You can also integrate the project separately into your own project. Its open source protocol is Apache-2.0.
Another important thing is the TCP asynchronous network I/O library currently used by dubbo-go. This library is also based on the Getty project written by Yu Yu in the early days. It is currently maintained by the community’s brother Wang and Fang Yincheng. It is also Apache-2.0 Open source agreement. In the next version, we will further optimize the network I/O and thread dispatch of dubbo-go and Getty.
In addition, we plan to support several other important functions of Dubbo in the next step, such as:
- routing rule routing rule (dubbo v2.6.x)
- dynamic configuration dynamic configuration center (dubbo v2.7.x)
- Metrics and monitoring (dubbo v2.7.x)
- trace link monitoring (dubbo ecos)
Question: What is the current application status of the project?
Answer: Dubbo-go has now begun to be used by some companies to try to integrate Go language applications into the company's existing Java & Dubbo technology stack, and build new Go language distributed applications and other scenarios. For example, the internal Go of Zhongtong Express calls the Java Dubbo service; as the service framework of Ctrip's Go language application, and the interoperability of Go and Java applications.
The specific application situation can be viewed: https://github.com/dubbo/go-for-apache-dubbo/issues/2
Q: What is the next evolution direction?
Answer: After dubbo-go moved to the Apache Software Foundation as a sub-project of Apache Dubbo, the first and most important thing is the further optimization of performance. Although the performance can meet the production-level requirements of the application, we feel that Go has not yet been used. The advantages of language, there is still a relatively large room for optimization. For example, the Getty mentioned above, the next version will make some optimizations for the network I/O model and thread dispatch of the dubbo-go application Getty.
In addition, it contains some important functions mentioned above that we need to complete in the near future to maximize compatibility with Dubbo in terms of functional integrity. Regarding the future development of dubbo-go, it will also evolve to the online roadmap of Dubbo 2.7.x.
Q: Speaking of performance, what is the current performance situation?
Answer: We have done a dubbo-go-benchmark project. When the CPU model is Intel® Xeon® CPU E5-2609 0 @2.40GHz and the number of CPU cores is 4*8, the hardware level will send 1k and return 1k of data. With 100 concurrent requests, 100w total requests, qps can reach about 12,000.
The CPU performance is replaced by a higher configuration such as Intel Core i9 2.9GHz, and the qps can reach about 20,000.
We will continue to optimize the performance of the Hessian2 library and the Getty library later to save resources for the majority of users.
If you need more interview materials from major companies, you can also click to enter directly and get it for free! Password: CSDN
If you like this article, please forward it and like it.
Remember to follow me!