On June 13, the open source large model sub-forum of the 2023 Open Atom Global Open Source Summit was successfully held. This forum brings together experts in the field of large-scale models in China, including mainstream players in industry and academia, to discuss the latest technologies and applications of large-scale models, and provide industry insiders with up-to-date technical information, in-depth technical analysis and practical applications guide. Guests from Open Atom Open Source Foundation TOC, Shanghai Artificial Intelligence Industry Association, Harbin Institute of Technology (Shenzhen) School of Computer Science, Tsinghua University, Baidu, Ali, Huawei, Tencent, Shenzhen Data Exchange, 4Paradigm and other units shared cutting-edge views.
Sun Wenlong, Chairman of the Open Atom Open Source Foundation
Sun Wenlong said in his speech that as open source has become one of the most effective ways to build a technology ecosystem, promoting the development of large-scale models in the form of open source and openness will help gather global wisdom and build a viable technology ecosystem that can be continuously iteratively updated. Through the open source large model system, all enterprises can share data, share computing power, and jointly build algorithms. At present, the foundation is also preparing to establish a large-scale model working committee, aiming to rely on the foundation's open-source public welfare platform to organize forces from all walks of life to focus on upstream and downstream partners, provide data computing power and algorithm public welfare services in an open-source and open cooperation manner, and accelerate the development of large-scale models The landing of industry applications promotes the development of large-scale model open source ecology.
Tan Zhongyi, Chairman of the Open Atom Open Source Foundation TOC
Tan Zhongyi introduced the importance of open source large models from a macro perspective. He believes that the big model condenses the knowledge of the world and will completely change the generation, dissemination and development of knowledge. Large models need to continue to promote ecological health and continuous evolution through open source collaboration. The goal is to build the core infrastructure of the next-generation digital economy. The key is to use open source data sets and open source training programs in compliance, and generate open source general-purpose large models through domestic computing power training. It is recommended to start with open source datasets, open source dataset cleansing programs, and platforms that provide model and data storage and distribution functions. At present, the foundation TOC has established an open source large model SIG (Special Interest Group, similar working group), and the foundation has also started to build a large model working committee. Colleagues from all walks of life are welcome to participate.
Xu Qi, Deputy Secretary-General of Shanghai Artificial Intelligence Industry Association
Xu Qi explained the opportunities and challenges of large models. He believes that the distributed training of large models tests the comprehensive capabilities of the whole stack and the whole process, such as algorithms, data, frameworks, and resource scheduling, and calls on domestic large models to be open source with Chinese characteristics.
Nie Liqiang, Executive Dean of the School of Computer Science, Harbin Institute of Technology (Shenzhen)
Nie Liqiang introduced the development history and research status of the multimodal large model, and demonstrated the self-developed multimodal dialogue system - Jiutian.
Dong Yuxiao, Assistant Professor, Department of Computer Science, Tsinghua University
Dong Yuxiao introduced the open source attempt from the 100 billion model GLM-130B to ChatGLM, and showed the training of the model and the efforts made in technical iteration.
Zhang Jun, Product Manager of Baidu Flying Paddle Framework
Zhang Jun introduced the large-scale model training capability of the industry-level deep learning open source and open platform Flying Paddle, and introduced Baidu's groundbreaking contribution to the breakthrough of deep learning distributed training technology.
Lin Junyang, Senior Algorithm Expert of Basic Large-scale Model of Alibaba Dharma Institute
Lin Junyang introduced the model capabilities of Ali's large model Tongyi Qianwen, including natural language and multimodal understanding and generation capabilities, and introduced related open source work. In addition, he also talked about the combination of Tongyi Qianwen and the Mota community. In the future, the combination of the basic large language model and various external open source models will create a more general AI system.
Jin Xiaoxian, Senior Architect of Huawei MindSpore
Jin Xiaoxian shared MindSpore's practices related to AI super-large models in recent years. He started with the development of the current large-scale model and the challenges faced, and focused on the technical practice of MindSpore and the practice in the large-scale model.
Li Baojia, Network Architect of Tencent Data Center
Li Baojia introduced Tencent's self-developed Xingmai high-performance computing network, which uses self-developed switches to build a super-large-scale network architecture, achieving 90% of the network load and a 16% increase in effective computing power under 2K-scale clusters. He also shared how the full-stack operating system can effectively reduce the interruption of business training caused by network reasons, so as to ensure the high reliability and availability of GPU clusters.
Wang Teng, Deputy General Manager of Shenzhen Data Exchange
Wang Teng believes that my country's data element market is in the incubation period and is developing rapidly, but the phenomenon of data islands still exists. The open source ecosystem encourages data sharing and openness, and encourages cooperation and innovation between users and developers. In the field of large models, Shenzhen Data Exchange is willing to become a transportation hub for the supply of open source large model training datasets, and at the same time solve the problem of compliance processing of open source large model training datasets with the help of the DEXCO community of Shenzhen Data Exchange data transaction compliance specialists. In addition, he also introduced the development of the Open Islands open source community and the Open Islands privacy computing FATE framework, which aims to protect data privacy.
Senior System Architect of 4Paradigm Lu Mian
Lu Mian, a senior system architect from 4Paradigm, introduced the underlying domestic computing power requirements behind large model training and services, including GPU virtualization solutions, heterogeneous computing technologies, etc.
This forum fully explores the latest technology and application of large language models, leads the audience to appreciate the practice of large models of major domestic companies and analyzes the domestic computing power scheduling under large models, creating a "technical feast" full of dry goods ", pointing out a clearer direction for the future of open source large models.