On October 18, Tencent open sourced the RPC development framework - tRPC, which is claimed to have the characteristics of "multi-language, high performance". The first batch of open source supports Go/Cpp programming languages. As we all know, there are already many open source RPC frameworks, such as gRPC, Thrift, Dubbo, and bRPC. Could it be that none of them can meet Tencent's needs? Is Tencent reinventing the wheel? Do we really need so many RPC frameworks?
To this end, Open Source China conducted an interview with Tencent’s tRPC team to answer some of the doubts in the minds of netizens.
1. Some netizens believe that there are already many open source RPC frameworks, and Tencent is reinventing the wheel by launching tRPC. What do you think of this view?
There are indeed enough existing frameworks, but for Tencent, one more set of frameworks cannot just be one more set. The core is to unify the existing frameworks.
In the past, at Tencent, the RPC framework was selected by the business itself, resulting in a wide variety of frameworks in use, including open source and self-developed, reaching dozens. They use different communication protocols and use different name services, resulting in unsmooth interoperability between services and hindering business development. At the same time, the maintenance and use costs of many RPC frameworks are also very high, and there is an urgent need to converge the existing frameworks. Whether to choose an existing framework or to develop a new framework for convergence, we also thought deeply about this issue before developing tRPC, conducted full discussions internally, and finally decided to use self-developed tRPC. Because Tencent's business forms are diverse and have relatively high requirements for performance, development language support, and architecture openness, open source or internal RPC frameworks cannot fully meet Tencent's business needs.
2. Tencent open sourced the RPC framework Tars in 2017. Is there any connection between today's tRPC and Tars? Why start a new one and develop a new one?
tRPC and Tars are two completely independent frameworks. However, tRPC also borrowed part of the design from Tars at the beginning of its design. Some of the core developers and designers of tRPC were also the core developers and designers of Tars. The reason for starting a new business is mainly because Tars cannot undertake the goal of unifying the company's internal framework stock. The main reason is that its own structure is relatively closed. tRPC adopts a plug-in design and has a very open architecture. It can be easily integrated into existing service management platforms, which is more conducive to inventory convergence.
3. When did the tRPC project start? Is there a story behind it worth sharing?
The tRPC project started in 2019, and it has been more than 4 years now. At that time, when the company had the largest number of existing frameworks, there were dozens of them, which seriously affected R&D efficiency and hindered further business development. There are indeed many stories that have happened in tRPC from its establishment to its development. For example, at the beginning of its establishment, there were different opinions within the company as to whether we should start a new business and build tRPC to converge the existing framework. There was a post on this issue in our internal forum, and there was a heated discussion throughout the company, with hundreds of comments and tens of thousands of PVs. No explanation.
After such a large-scale internal discussion, it was finally decided to develop tRPC in-house. The development of tRPC has received extensive support from the company’s internal technical staff. In order to make tRPC become a comprehensive RPC framework and be able to shoulder the important task of unifying the stock, we have adopted an internal community model for research and development. Many technical colleagues in the company who are good at framework development have actively participated. Hundreds of people have contributed code to tRPC. many. The design and development process also saw the collision of various ideas. For example, the overall architectural form of tRPC plug-in was determined through several closed-door meetings with dozens of people and in fierce arguments.
4. Why open source tRPC? What do you hope open source will bring to the project?
There are two reasons for open source tRPC: 1. It is needed when the company's internal business expands externally. For example, if a game goes overseas, the business needs to be privatized and deployed by an external enterprise. At this time, because tRPC is used for business development, the code needs to be delivered. 2. I hope to do more technology sharing and exchanges with the outside world through open source, and use the power of the community to further build tRPC better.
5. The official mentioned that tRPC supports multiple communication protocols. Can you tell us specifically which communication protocols are supported? Can protocol versatility and high performance be achieved at the same time?
The tRPC framework supports the tRPC protocol by default, and also supports the industry's HTTP(s)/gRPC/bRPC/Tars/Thrift protocols, as well as various internal communication protocols within the company. Currently, only the HTTP(s)/gRPC protocol is open sourced, and other protocols will be gradually open sourced in the future. .
As for whether the protocol has both versatility and high performance, it depends more on the business scenario and requirements. If you want versatility, you can choose the HTTP or gRPC protocol. If you want high performance, you can choose the tRPC protocol. Because the design and implementation of the protocol itself will have a relatively large impact on performance.
6. Compared with other open source frameworks, tRPC appeared relatively late. From the perspective of the evolution history of the RPC framework, is there any essential difference between tRPC and other open source RPC frameworks?
The evolution of the RPC framework has indeed become very mature. It is difficult to say that there are any essential differences between the open source RPC frameworks in the industry. They are more in line with the demands of their respective business development. The main purpose of the emergence of tRPC is to converge the company's existing frameworks. It has certain late-mover advantages. It can absorb the advantages of existing existing frameworks and avoid shortcomings. It builds an open architecture based on plug-in ideas on the basis of high performance. This makes it more applicable to Tencent’s diverse and complex business scenarios.
7. Officially mentioning project planning, there are two main points. One is to open source more programming languages: Java, Python, and Node. The other is to enrich the ecology and open source more plug-ins and components related to cloud native. I would like to ask what are the reasons for considering this as a key development direction in the future?
Mainly, we hope that the framework can be used more widely and efficiently, and that more development language support can cover more scenarios. For example, many companies now use the Java language, and tRPC can only become a candidate if it supports Java. Another example is that most AI scenarios now use Python, so tRPC can only be used if it supports Python.
Enriching the ecosystem also hopes that tRPC can help users build their own microservice systems more quickly. At present, everyone likes to use cloud-native related components to build their own microservice systems. If tRPC can enrich the cloud-native plug-in ecosystem, then users using tRPC to connect with these cloud-native components will be more efficient and faster.
8. Tencent has tRPC, Baidu has bRPC, Alibaba has Dubbo, and Byte has Volo. Why do every major manufacturer have to develop its own RPC framework?
Why do major manufacturers develop their own RPC framework? Personally, I think it is mainly determined by the business form. Although the RPC frameworks look similar, when they are actually applied to business, there are many differences depending on the form of the business. If you use an open source framework, it is very likely that it will cost a lot to resolve these differences, and it will take a longer period of time, or it may not even be resolved. For example, when we used tRPC in game scenarios, we found that the general RPC design could not meet the needs of stateful business scenarios such as games. We also based on the tRPC plug-in architecture and worked with the game team to replace the underlying communication components. Only to meet the needs of the game scene. If an open source framework is adopted, this problem may be difficult to solve.
Alibaba Cloud suffered a serious failure and all products were affected (restored). Tumblr cooled down the Russian operating system Aurora OS 5.0. New UI unveiled Delphi 12 & C++ Builder 12, RAD Studio 12. Many Internet companies urgently recruit Hongmeng programmers. UNIX time is about to enter the 1.7 billion era (already entered). Meituan recruits troops and plans to develop the Hongmeng system App. Amazon develops a Linux-based operating system to get rid of Android's dependence on .NET 8 on Linux. The independent size is reduced by 50%. FFmpeg 6.1 "Heaviside" is released