Open source address
https://github.com/XiaoMi/mone/tree/master/miapi-all/mi-api Welcome friends who are interested in cloud native technology/R&D efficiency to join (Fork, Star) us.
background
At present, the software technology field is rich in ecology, and various products and technologies are booming. The current mainstream technology architecture in the Internet industry is generally based on collaborative research and development of microservices. In the collaboration system of microservice R&D, the automation and intelligence capabilities of microservice interface documents, testing, Mock and other related collaboration links are a key link in determining the efficiency of R&D delivery. MiApi is used to improve and fill the vacancies and deficiencies in this link. .
What is MiAPI
The first one-stop platform on the market that integrates multiple types of RPC interface automated management, automated document generation, interface testing, mocking, team documentation and other functional modules. It has redefined the development and delivery process of microservice interfaces and is committed to providing the best interface governance services and improving development efficiency to a greater extent. And has obtained the relevant software copyright "Mi-Api One-stop Microservice Interface Maintenance Platform" and related technology patent "Annotation-based RPC interface document automatic generation technology".
problems we face
The reason why we decided to set up a project to do this is because our business research and development actually has several pain points in the delivery of service interfaces:
Extremely high interface docking costs
The ability to generate documentation for microservice interface development for multiple protocols (such as Dubbo and Grpc) is lacking. Manually writing documents extremely consumes developers' manpower, time, and energy. Versions are difficult to control, modifications are difficult to trace, and interface changes/collaboration costs are extremely high.
Mutually blocking debugging processes
There is a lack of testing/debugging tools in the development process of microservice interfaces. The provider and the user cannot mock data in advance, blocking each other's development progress.
Lack of continuous delivery capabilities
Interface/project document management is chaotic and lacks good aggregation, continuous delivery and long-term maintenance capabilities.
However, several mainstream interface management software currently on the market, such as YApi, Swagger, Postman, etc., are almost unable to take into account the pain points we face. For multi-protocol, the support for automatic generation is relatively weak.
Based on the above pain points, we decided to independently design and develop a complete, more automated, process-based, and intelligent solution. The following article will introduce the project in detail, as well as how we solved and designed some pain points.
Core processes and capabilities
core process
In order to automate more than 90% of the entire service interface delivery process, we have designed an online interface definition delivery process based on annotations. That is, during the definition phase of the service interface, developers will add interfaces that need to provide external capabilities according to their own needs. Basic annotations indicate that after completing the interface definition and running the service, you can directly search and select your own service in the platform, load and save it. Since most of the current mainstream microservice projects are based on the Spring/Springboot framework under Java, this project plan is mainly aimed at spring framework projects. Of course, we also support adding maintenance interfaces manually.
During the saving process, the platform will generate basic examples of the interface, mock data, etc. based on the basic information of the interface obtained through automatic analysis and the option descriptions customized by the developers. At the same time, after loading and saving, you can directly use the generated request case to debug the corresponding interface, or mock the interface.
The main process is shown in the figure:
Core flow chart
core competencies
For the entire platform, we focus on providing the following core capabilities:
Multi-protocol service interface support
The platform provides support for the configuration and maintenance of RPC protocol type interfaces such as Http/Https , Dubbo , and Grpc . Including but not limited to the automatic generation of interface documents for various protocol types, automatic generation and customization of interface mock data, remote debugging of interfaces, etc.
For services that have added dependencies and annotations in the project, you can directly search and select to load them in the platform (the loading process will automatically generate document data).
Platform usage examples
One-click testing and mocking
For the service interfaces under the above supported protocol types, the platform supports direct calling and debugging using use case data generated by the platform. At the same time, the platform will also generate corresponding mock data and the address for calling the mock for the maintained interface.
Share quickly
项目中维护的接口可以通过自定义聚合集合的方式生成接口集合链接分享给调用方，同时也支持直接生成 pdf格式文件进行转发。
技术与架构
上述介绍了接入平台的核心流程与平台提供的核心功能，以下将对平台的整体设计以及一些核心能力的技术方案做详细说明。
整体架构
技术架构图
服务接口信息的加载生成
对于不同协议类型的接口，我们关心的接口信息可能不完全一致。对Http来说，我们可能需要关注该接口的url路径、请求方式、出入参数等。而对于Dubbo接口来说，我们可能更关注该接口的服务名、方法名、出入参类型、服务版本、分组等。总体而言，对于大多数服务接口，接口的入口、出入参这些信息都是我们需要关心的，而这些数据的获取则需要对业务项目从接口层、方法层、参数层进行解析。
因此，我们针对不同的协议类型的服务项目提供了相应的解析包，在服务运行后，通过可选的开关选项，决定是否对项目进行扫描解析，若启用，则在业务项目 spring 容器整体初始化完成之后，触发扫描器（扫描器的具体设计实现将在后续进行介绍），对项目服务、接口、参数等进行扫描解析，并缓存解析数据。
项目数据扫描解析完成后，组件将把缓存的数据推送至主体平台，平台根据一定规则存储这些接口数据，从而业务研发人员即可在平台中搜索相应服务数据，根据自身需要选择、加载、生成指定的服务、接口文档。
接口加载流程图
维护接口的调试与Mock
除了生成服务的接口文档之外，在开发过程中，研发人员还需要对指定的接口进行 mock 或调用测试。
关于调试，在接口文档的生成过程中，平台将根据解析到的接口基本数据生成一些请求示例、请求参数用例等，这些数据在文档中都会体现。研发人员可以在不进行任何额外配置与参数编写的情况下，通过平台的测试引擎，对不同协议类型接口直接进行调用，对于不同协议类型的接口调用底层实现方式有所区别，具体如下图所示：
测试的流程图
对于 mock 来说，不同协议接口的mock也有所区别。例如 http/https 协议的接口，平台将直接生成该接口的 mockUrl，调用方直接调用该 url 即可获取该接口的 mock 数据。而对于 dubbo 服务接口来说，调用方一般基于服务方提供的 api 包进行调用，因此需要进行特殊处理拦截转发 dubbo 调用请求，获取平台生成的 mock 数据，整体如下图所示：
接口mock流程图
因此，为了实现上述描述的功能，平台整体将由提供给业务项目引用的依赖包、交互平台、Mock服务器几个组件构成。
核心组件
业务依赖包
为了获取上述我们所需的业务项目接口的基本数据，我们需要业务方引入我们提供的一套依赖包，针对不同协议接口的服务我们提供了针对性的依赖包，例如对提供 Http 接口、Dubbo接口的项目，我们分别设计提供了适配 Http、适配 Dubbo 的依赖包。这些包将用于不同协议接口数据的解析、数据推送、服务注册等等。
适配不同协议的依赖包在具体解析等实现上有所区别，但整体结构基本一致，所有包都由注解模块（annos）、核心模块（core）、缓存模块（cache）构成。
其中，注解模块（annos）提供了包括启用文档生成的开关注解、服务接口层级的定位注解、方法层级的定位描述注解以及字段层级的描述注解以上4个核心注解。具体注解的作用在技术方案的注解定义中将详细介绍。
核心模块（core）主要提供了扫描器（scanner）以及相关实现工具类。该模块是整个MiApi的核心，它用于扫描解析业务项目接口，获取、解析服务、接口的基本信息如方法名、出入参等数据。具体实现同样在技术方案中的扫描器模块加以说明。
缓存模块（cache）则用于暂存扫描器（scanner）扫描解析到的基本数据，用于后期的数据推送等。
主体平台
主体平台是用户交互的入口平台项目，它提供了一些基本的包括团队管理、接口管理、文档管理、接口数据生成、多协议接口测试等模块功能。当然，平台最重要的能力是承接来自各地、各方业务项目的数据推送，包括业务的心跳注册（用于实时在线数据，技术方案中将做介绍）、接口基本数据推送，并基于接收到的这些基本数据生成例如请求用例、mock数据等等，并将生成好的数据以一定规范组织聚合成对用户友好的文档页面内容。
Mock服务器
Mock服务器是较为独立的一个模块，它的主要功能是接收存储平台基于接口数据生成的、或者研发人员自定义的mock数据，以及承接来自各方的mock请求，根据请求的路径、参数、以及一定的规则进行匹配最终返回指定接口的mock数据。对于不同协议接口的 mock 调用逻辑不完全一致，具体将在技术方案中的Mock数据生成与调用拦截中进行介绍。
总结
本文介绍了关于 MiApi 项目的研发背景、平台提供的主要流程与能力以及一些核心能力的设计思考，关于本项目更多的技术细节与实现我们将在后续文章中陆续进行分享。