On July 7, 2023, under the guidance of the Organizing Committee Office of the World Artificial Intelligence Conference, the Data Elements and Privacy Computing Forum co-hosted by the Lingyu Open Source Community, Ant Group and Machine Heart was held at the Shanghai World Expo Convention Center. At the forum, Wang Lei, General Manager of Ant Group's Privacy Computing Department and head of the Yinyu community, released the Yinyu 1.0 version, and gave an overall introduction to the framework expansion and upgrade of the Yinyu version 1.0. The lingo 1.0 version not only further expands the scope of open source, but also optimizes and expands the overall architecture. The core content involves product layer, resource layer, interconnection and other sectors, and the overall effect covers performance optimization, usability improvement, and interconnection form Rich.
Figure: Argument 1.0 Architecture Diagram
Product Layer: Smooth Learning Curve Improves Ease of Use
The Lingyu open platform has been opened to 50+ institutions for experience in the past year. This time, the lingo 1.0 version brings a new MVP (Minimum Viable Product) deployment experience package : **A lightweight functional experience tool for beginners in privacy computing, with built-in nodes and data resources, mutual authorization between nodes, installation You can experience the main common functions such as data processing, data analysis, model development, and model evaluation. **Compared with the Hidden Words open platform that focuses more on production scenarios, the Hidden Words MVP deployment experience package lowers the threshold of use in various ways, paving the way for the formal use of business production. (Download address: https://www.secretflow.org.cn/docs/quickstart/mvp-platform)
1. What problems can the MVP deployment experience package solve?
As an industrial-grade highly available privacy computing framework, Lingyu is often praised for its performance and stability. However, for many potential users and novice users of privacy computing, the basic requirement is to quickly perceive functions, understand the complete process, and obtain intuitive effects to make judgments or decisions.
Lingyu MVP deployment experience package transforms user experience feedback into mature product capabilities. From the perspective of beginners in privacy computing, the preparation steps are embedded in the installation and deployment process as much as possible to improve the efficiency of direct face-to-face between users and privacy computing functions. In addition, we have also broken the original application review and resource support restrictions, so that innovative technologies can benefit a wider range of user experience without threshold.
2. The specific advantages of the argot MVP deployment experience package
- Advantage 1: Simplify complexity and reduce stuck points in preparation steps
In the original experience process, users need to configure node resources and data resources by themselves. In order to solve the problem of long preparation links and many stuck points, the Argument MVP deployment experience package encapsulates these preparation steps in the package and provides it to users in the form of a "one-click installation package". Its installation process also covers the pre-procedure preparations for joint projects, so users can start experiencing privacy computing features more quickly.
- Advantage 2: template configuration reduces the difficulty of getting started with complex components
For users who experience it for the first time, the Lingyu MVP deployment experience package provides a scenario-based training flow template option, and the configuration of each component has been completed. These templates can be run automatically until the result, helping users understand the principles of components and reducing the configuration difficulty in custom training streams.
- Advantage 3: The function experience and function explanation of the novice training camp are carried out simultaneously
The Lingyu MVP deployment experience package combines functions and tutorials into one, and adds an interactive novice guide, allowing users to complete hands-on practice while learning, further smoothing the learning curve of privacy computing.
The Resource Layer: Addressing Difficult Dimensions in Interagency Computing Missions
Lingo version 1.0 is officially open source **Kuscia privacy computing task orchestration framework: **Kuscia can solve integration problems such as port merging and API access when using Lingo, and supports different modes such as interconnection or built-in deployment of third-party systems and third-party System interoperability. (github address: https://github.com/secretflow/kuscia)
1. What is the infrastructure difference? What challenges will arise in interagency computing missions?
Privacy computing involves many cross-agency scenarios, and the various participants in the joint project are different in many aspects such as data storage, data transmission, computing resources, and security controls. These can be collectively referred to as differences in infrastructure.
Different participants have differences in operating environment and network links. Network links refer to the communication addresses, communication protocols, message encryption, request authentication, etc. between nodes when the parties build privacy computing applications. The operating environment may be divided into physical machines, virtual machines, etc.
Cross-institutional computing tasks involve the resource coordination of multiple institutions. In the process of task scheduling, it is necessary to coordinate and manage the resource allocation of various agencies to ensure that tasks can be completed on time, and it is necessary to use secure communication protocols and mechanisms to protect data from tampering or theft during transmission. If the effective management and scientific scheduling of resources cannot be guaranteed, a series of problems such as inefficient computing tasks, waste or redundancy of computing resources, unstable applications and even task failures will be caused.
2. How does Kuscia solve these problems?
In the deployment, it basically solves the requirements of lightweight (lightweight nodes support 2C4G at least), and focuses on the diversified port adaptation requirements of business organizations when accessing. It not only supports single ports for organizations, but also supports multi-task port consolidation. In addition, the privacy computing deployment issues that business organizations are currently focusing on include networking models . Kuscia supports a decentralized P2P model of equal cooperation, a centralized model that is easier to manage and control, and a hybrid model in which the two coexist due to multi-party cooperation.
In task scheduling, in addition to basic standard configurations such as DAG task scheduling, Kuscia focuses on supporting multi-task concurrency . Since privacy computing tasks are usually computationally intensive, simultaneous execution by multiple organizations and multi-task concurrency will cause competition for computing resources. Kuscia ensures the reasonable allocation of resources by means of resource isolation between tasks and controllable task priorities.
In the connection with external systems, Kuscia has realized the interconnection and intercommunication of the task scheduling layer , and supports the UnionPay black box interconnection and intercommunication protocol. Kuscia provides a unified operating interface for privacy computing applications, and users can directly call a variety of existing privacy computing engines such as FATE. For example, business party A uses the federated learning capability of a certain framework in its business platform, but due to business expansion and upgrading, a single capability can no longer meet the demand. Using the adage Kuscia, business party A can realize the integrated call and capability of the original framework's computing power Expand, do both.
Algorithm layer and scheduling layer full-stack interconnection
At the end of 2022, led by the Privacy Computing Alliance of China Academy of Information and Communications Technology, the industry's first white-box interconnection open protocol "Privacy Computing Cross-Platform Open Algorithm Protocol Part I: ECDH-PSI" designed by Ant Group was officially released, marking that privacy computing has entered a deeper level. The level of intercommunication has entered a new situation of algorithm intercommunication.
The SS-LR protocol has also been standardized in Lingo version 1.0. SS-LR interconnection involves standardization of transport layer, standardization of cryptographic protocol, standardization of security operator interface and standardization of application algorithm, almost involving all aspects of the argot algorithm engine. Based on this, the interconnection and intercommunication infrastructure of the Cryptography Engine has begun to take shape, and other interoperability algorithm protocols will be launched faster and more in the future to form a complete set of open algorithm (white box) protocol clusters.
At the same time, Kuscia also supports the latest interconnection and interoperability protocol in the interconnection and interoperability requirements of the Beijing Financial Industry Alliance in terms of task scheduling, enabling algorithm containers to be scheduled and executed across platforms and ecosystems. At present, Kuscia has been coordinated with the Insight platform. Not only the algorithm container of Insight can run on the Kuscia platform, but also the algorithm container of Lingyu can also run on the Insight platform.
Through the interconnection of the algorithm engine and the full stack of the Kuscia scheduling platform, Lingyu is committed to building a more transparent and open world. Yinyu is willing to join hands with all walks of life to empower the global interconnected encrypted data element circulation network and promote the development of the privacy computing industry.
More Highlights of Cryptic Version 1.0
In addition to the three points mentioned above, the lingo 1.0 version is also capable of updating in terms of algorithm layer, device layer, cryptographic capabilities, and ease of use. (Your ⭐️ is the greatest encouragement to the Yinyu open source community_ : https://github.com/secretflow/secretflow)
