Article Directory
big model craze
The "top class" in this year's Internet track is none other than large models. The "China Artificial Intelligence Large-scale Model Map Research Report" released by the New Generation Artificial Intelligence Development Research Center of the Ministry of Science and Technology at the end of May shows that 79 large-scale models with a scale of more than 1 billion parameters in my country have been released, almost entering the "Hundred Model War".
Baidu's Wenxin Yiyan, Ali's Tongyi Qianwen, Xunfei Xinghuo large model, Zhipu AI's ChatGLM, etc. have been released. Since then, Meituan, Baichuan Intelligent, Yunzhisheng, Meitu, Tencent... domestic technology companies that have newly joined the large-scale model track have emerged one after another, and an "arms race" around large-scale models has become increasingly fierce.
large model landing
After ChatGPT set off the AI boom, Microsoft has become one of the most important companies in this wave. Not only because it is the major shareholder of OpenAI, or because it launched New Bing with AI blessing.
More importantly: as the world's largest operating system service provider, the world's largest office software developer, and the world's second largest cloud service provider, Microsoft even proposed that "all its products will be integrated with large-scale model components, fully embracing The large model landed. "
At the Zhongguancun Forum 2023, Li Yanhong took the title of "Large Models Change the World" and also proposed that "Baidu will be the first company to redo all products, not integration, not access, but redoing, refactoring... "
It is unceremoniously predicted that all future services will be oriented to or rely on "big models" to provide product services.
So how should services for the future "big model" be designed or refactored?
service design or refactoring
In order to support the "big model" call, the service needs to be relocated and become a "base". The base here can be understood as the foothold of the "big model": the throughput of target data.
The powerful "big model" redefines human-computer interaction. Analyze the needs of users in a short period of time, and provide targeted services according to the needs. The current interaction mode of manually triggering the static interface of the App by the user is broken. It becomes the triggering of single or multiple target service interfaces after the "big model" AI analysis appeal, and finally summarizes and tailors the response data of each service. Carry out service function output.
For example: in the map scene,
guest A: Help me plan the Beijing travel route for the 11th October...
Map: Beijing Attractions -> Weather for the 11th -> Attraction Ratings -> Arrangement of the appropriate browsing order among attractions -> …
Based on the characteristics of this interaction, combined with various technical concepts such as distributed and microservices in cloud native, we can refactor, upgrade or redesign services.
Future Service Architecture
Microservice
In order to support the interaction mode of the "big model" in the future, it can meet various arbitrary service assembly and assembly. We need to encapsulate services at the smallest granularity, which also continues the core idea of microservices.
Stratification
It should be noted here that the current interaction mode still exists. We want to use the minimum cost to merge and support the two interaction modes. Then it is necessary to introduce a "layered" design idea to abstract and differentiate different interaction modes into different logical layers.
Here is a pattern, as follows:
Large Model Application Architecture
The architectural pattern is divided into entry layer, large model result call layer, protocol layer, business cohesion layer, data access layer, and microservice call layer.
Architecture Design Diagram
As shown in the above figure, each logical layer:
- entry level
- Complete the middleware registration task and provide basic capability support for subsequent service functions. Include
- Interface token authentication [Sign plus salt mode],
- Service exception capture [Panic Recover middleware: capture service exceptions to prevent the main program from panic],
- Monitoring service registration [Prometheus indicator collection],
- Log middleware [initialize log function, print access log Access_log],
- Mesh service registration [Proxyless Service Mesh for traffic fusing and current limiting, preventing call avalanche...]
- Complete the middleware registration task and provide basic capability support for subsequent service functions. Include
- Large model call layer
- Provide the "base" capability for the large model, and provide the API call capability of the corresponding service based on the output results of the large model. Contains two service granularity protocols, composite and single protocol
- protocol layer
- Contains two types of composite protocol and single protocol, providing content data output for business and large model calls.
- Single protocol, API interface for service minimum granularity encapsulation
- Composite protocol, encapsulated API interface after assembly for multiple services
- Contains two types of composite protocol and single protocol, providing content data output for business and large model calls.
- business cohesion layer
- It is the service aggregation layer corresponding to the composite protocol. Perform serial/parallel orchestration of multiple services at this layer, and provide service aggregation data externally
- data clipping layer
- Above the service call layer, it is an independent encapsulation of the request and response data of each service
- Microservice call layer
- Complete service calls based on multiple communication protocols
- In addition, they are Util and Tool parts
- Throughout the service, provide public capability and capability support related to observability and stability
Architecture Demo implementation
//篇幅有限，见后续博文
summary
Under the current situation of increasingly fierce competition and global complexity and change, enterprises and teams will become the final winners only if they grasp the opportunities and make arrangements in advance, and have absolute agile competitiveness!