Author of Wisdom Things (public account: zhidxcom)
| Edited by Cheng Qian and Li Shuiqing
| Xin Yuan
Zhishi reported on March 24 that today, the GTIC 2023 China AIGC Innovation Summit was successfully held in Beijing!
Just when the curtain of the GPT-4 era was lifted, this summit, with the theme of "AI New Era Creates a New World", is the first domestic high-level innovation summit focusing on generative AI (AIGC), bringing together more than 20 participants Industry-university-research leaders launched rounds of fierce technical confrontation and point of view collisions to help every industry person who is forging ahead in the fog to complete a journey of eliminating the false and seeking the truth.
Too many questions have flooded our brains in the past two months: Will GPT-4 really bring about an explosion of cognitive intelligence? How much is the gap between our AI and ChatGPT? Will the Chinese version of OpenAI be born? Will MaaS (Model as a Service), which is frequently mentioned by major manufacturers, be a definite trend? Will China's AIGC industry be "big manufacturers win all" or "a hundred flowers bloom"... Many questions have been answered in this summit, and most of the answers are different from our intuition without thinking, and even the opinions of experts are diametrically opposite. making these discussions extremely valuable.
From the confrontation of large-scale model products of major manufacturers to the collision of top AI investment ideas, from the early adopters of AIGC’s pioneering products to the “violent aesthetics” competition of computing power companies, from the debate of “big model VS small model” to “Why didn’t ChatGPT first appear on The question of the soul of China... wave after wave of climaxes, the summit site was full of seats, the crowd was full, and the atmosphere of communication was enthusiastic.
On behalf of the sponsor, Gong Lunchang, co-founder and CEO of Zhiyi Technology, said at the summit that the AI industry is ushering in a new period of market opportunities, and the China AIGC Innovation Summit hopes to build a communication platform for academia, industry, and investment circles. This summit mainly includes a main forum of the AIGC summit forum, three thematic forums of large models, China-like ChatGPT, and AIGC application innovation.
At the meeting, academic institutions such as the Chinese Academy of Sciences, industry giants such as Microsoft, Baidu, Kuaishou, Nvidia, Kunlun Wanwei, and SenseTime, Lanzhou Technology, Moxin Artificial Intelligence, UBTECH, Zhipu AI, Luchen Technology, Cloud Entrepreneurial pioneers such as Zhisheng, Zhujian Intelligent, aiXcoder, Movie Book Technology, and Computational Aesthetics (Nolibox), and guest representatives from investment institutions such as Qiming Venture Capital, Genesis Partners Capital, and Blue Run Ventures shared large models and generative methods. AI's cutting-edge innovations, business prospects, computing power evolution, entrepreneurial opportunities and investment strategies.
The following is the essence of the speech at today's AIGC summit.
1. Speech from the Sponsor: With the evolution of large models, the AI industry is ushering in a period of new opportunities
Gong Lunchang, co-founder and CEO of Zhiyi Technology, delivered a speech for the summit on behalf of the sponsor. The emergence and evolution of key algorithms and models such as pre-trained language models and diffusion models have promoted the rapid development of generative AI, and related products have attracted global attention in a very short period of time.
▲Gong Lunchang, co-founder and CEO of Zhiyi Technology
The past week has been a critical period for AI to advance by leaps and bounds. The release of GPT-4 and Wenxinyiyan, and Microsoft's integration of GPT into the office family bucket have led to a leap in production tools. The AI industry is ushering in a new period of market opportunities.
Based on this, the China AIGC Innovation Summit hopes to build a communication platform for the academia, industry, and investment circles, so that everyone can have in-depth exchanges, stimulate ideas, and promote the implementation of cooperation and innovation.
This year is the seventh year since the establishment of Zhiyi Technology. Zhiyi Technology adheres to the two-wheel drive of technology and industry, focuses on cutting-edge technologies represented by digitization and intelligence and their industrial applications, and builds two major business systems: industrial media and enterprise services.
Zhiyi Technology has an industrial media matrix represented by Smart Things, Core Things, and Car Things. It has become an industrial media with unique positioning in China and has high influence and credibility. The core enterprise service system cooperates with experts and scholars from outstanding companies in the industry and the world's top universities to hold a series of talks and lectures for new youth, and cooperates with top domestic and foreign companies to hold customized open courses. Up to now, more than 600 courses have been completed. Good reputation.
2. Large models bring about the rise of cognitive intelligence, and major manufacturers and world-class scientists focus on it
At the summit in the morning, Zhou Ming, founder and CEO of Lanzhou Technology & CCF Vice Chairman of China Computer Federation, explained the new paradigm brought by the large model. Understand the mysteries of the world.
At the same time, Xu Mingqiang, Chief Technology Officer of Microsoft's Omni-channel Business Unit, took everyone to explore AIGC trends and the application of Microsoft's Azure OpenAI service in enterprises; Yuan Foyu, vice president of Baidu Group, who just launched Wenxin Yiyan, came to the scene to discuss how Wenxin Yiyan can change Cloud computing market game rules.
1. Zhou Ming from Lanzhou Technology: The large model brings about the rise of cognitive intelligence, and the nine major aspects are the key points
Zhou Ming, founder and CEO of Lanzhou Technology, vice chairman of CCF of China Computer Federation, and chief scientist of Innovation Works, gave an in-depth interpretation of the new paradigm brought by the large model.
▲Zhou Ming, Founder and CEO of Lanzhou Technology, Vice Chairman of CCF of China Computer Federation, Chief Scientist of Innovation Works
As a technical expert from Microsoft, Zhou Ming said that he was deeply influenced by Bill Gates, the co-founder of Microsoft, and believed that large models are bringing about the rise of cognitive intelligence. Large models, especially ChatGPT, represent a practical era for language understanding, multi-round dialogue, and problem solving. At the same time, large models can effectively solve the problem of NLP task fragmentation, greatly improve research and development efficiency, and mark that NLP has entered the stage of industrial implementation.
At present, AI is going through the 2.0 era from a dedicated model for a single task, to a general model for a wide range of tasks, and then to the AGI era of general artificial intelligence. The era of AI 2.0 will first innovate creative content, office methods, search engines, human-computer interaction interfaces, finance and other fields.
Founded in June 2021, Lanzhou Technology has launched a number of large-scale external product services. At present, it has implemented Mencius large-scale model, AIGC (Intelligent Creation) platform, machine translation platform, financial NLP platform and other technologies and products. Flush, China Fund and other enterprises. Combined with ChatGPT-like technology, Lanzhou Technology has launched a dialogue robot MChat, which can help users complete various tasks in specific scenarios through intelligent dialogue.
Talking about the prospect of the future direction of the industry, Zhou Ming said frankly that the current ChatGPT-like technology is still lacking in reasoning, logic, mathematics and arithmetic, and factual errors. In the future, the nine major issues related to large models deserve special attention, involving reasoning ability, factual correctness, and Chinese processing ability.
2. Microsoft Xu Mingqiang: Cooperate with OpenAI to build a supercomputer
Model parameters are showing an exponential growth trend. Today, doubts about big models are replaced by new ones in as little as 1-2 years. Therefore, Xu Mingqiang, chief technology officer of Microsoft's omni-channel business unit, firmly believes that the model will still grow rapidly, because only 1/10 of the current high-quality corpus is currently used.
▲Xu Mingqiang, Chief Technology Officer of Microsoft Omni-Channel Division
All of this is inseparable from the support of powerful computing power, which also determines the size and parameters of the model that can be trained. Therefore, Microsoft Azure cooperated with OpenAI to build an AI supercomputer designed for large-scale AI training, which has 285,000 CPUs and 10,000 GPUs.
Xu Mingqiang compared the large language model to a sponge, and Wikipedia, medical or scientific papers are water, and stuffing these papers will bring about the emergence of its capabilities.
Finally settled in enterprise applications. In the CPU era, the question considered in enterprise applications is how to transform business problems into computing problems, that is, to convert applications into computing problems through compilers. Now it is transformed into how to convert problems in various industries into a content processing problem.
Enterprise-level ChatGPT application scenarios include customer service, sales marketing, content generation, knowledge management, and decision-making assistance.
3. Zhang Jiajun, Chinese Academy of Sciences: Demystifying the "Zidong Taichu", the multi-modal large model first appeared "multi-specialized and multi-functional"
At the meeting, Zhang Jiajun, researcher & doctoral supervisor of the Institute of Automation, Chinese Academy of Sciences, and vice president of Wuhan Institute of Artificial Intelligence, explained how the "Zidong Taichu" large model understands the technological mysteries of the world.
▲Zhang Jiajun, Researcher & Doctoral Supervisor of Institute of Automation, Chinese Academy of Sciences, Vice President of Wuhan Institute of Artificial Intelligence
Zhang Jiajun said that the effect of deep learning pre-training large models continues to improve. At present, through the "big data + large model" method under the condition of self-supervised learning, the multi-modal large model has initially shown "multi-specialty and multi-functional", and has made rapid progress in small sample learning, natural language question answering, and cross-modal generation. Large models have led to a wave of innovation, but their energy consumption and cost are extremely high, and there is still a big gap in cognitive ability compared with humans.
"Zidong Taichu" is the world's first 100 billion parameter multi-modal large model launched by the team of the Institute of Automation, Chinese Academy of Sciences. Zhang Jiajun said that this model supports multi-task self-supervised learning at the token level, modality level, and sample level. The multimodal weakly correlated data is trained for 128 days on 512 cards, and the unified modeling of modal understanding and modal generation is realized at the same time. "Zidong Taichu" supports cross-modal retrieval and generation examples such as searching for images by text, generating sounds by images, and generating images by sounds. For example, if you input a real image, Zidong Taichu can generate a personalized 3D image.
At present, the team has launched Zidong Taichu Open Service Platform 1.0, Zidong Taichu Luoshen 1.0 AIGC Intelligent Generation Platform, and integrated resources from various industries, universities and research institutes to build an open source and open ecosystem of artificial intelligence, and explore the industrialization path of general artificial intelligence.
4. Yuan Foyu from Baidu: Wenxin’s words will change the rules of the cloud computing market game, and more than 100,000 companies have applied to call
The global AI industry has entered an explosive period, and AI is creating a brand new world. In 2021, Baidu CEO Robin Li once said: "When a computer has the ability to understand human natural language, has the ability to express clearly, and has good logical reasoning ability, it will be very human-like. And the memory and computing power of machines Far stronger than humans. Therefore, AI will definitely revolutionize every industry today.”
▲Yuan Foyu, Vice President of Baidu Group
The key technical support behind Baidu's new-generation large language model and generative AI product Wenxinyiyan is natural language understanding.
Yuan Foyu, vice president of Baidu Group, emphasized, "Among the world's major manufacturers, Baidu is the first to release. Microsoft directly called Open AI, and Google opened the test on March 21. Meta and Amazon have not really released the same type and same type of AI. level products."
Wen Xin Yi Yan has five core competencies, namely literary creation, commercial copywriting, mathematical logic calculation, Chinese comprehension, multi-modal generation, can continue to write "Three-Body" and "Dream of Red Mansions"; generate live copywriting; point out mathematical problems Make mistakes and give the problem-solving process; strong ability to understand Chinese language; multi-modal generation ability of generating pictures, audio and video based on Baidu's self-developed large language model.
Yuan Foyu said that from the performance of Wenxin Yiyan, it has the ability to understand human intentions, and the accuracy, logic, and fluency of answers are gradually approaching human levels.
A week after the Wenxin Yiyan press conference, more than 100,000 companies have applied for the Wenxin Yiyan API call service test.
The IT technology stack in the era of artificial intelligence has a four-layer architecture of application layer, model layer, framework layer, and chip layer. Yuan Foyu said: "Globally, Baidu is the only company with leading products in these four layers."
In addition, the development of large models will bring three major industrial opportunities in new cloud computing, industry model fine-tuning and application development.
3. Will China produce an OpenAI? Investor: No!
Tracing back to the application upwards, Zhang Fuzheng, head of the Kuaishou MMU NLP Center and Audio Center, talked about the exploration and application of AIGC technology in Kuaishou; exploring the computing power downwards, Wang Wei, the founder and CEO of Moxin Artificial Intelligence, traced the computing power of the AIGC era to the bottom How to "evolve"; In addition, Zhou Zhifeng, partner of Qiming Venture Capital, explained the entrepreneurial opportunities and investment strategies in the new wave of AI from the perspective of investment.
1. Kuaishou Zhang Fuzheng: Exploring AIGC applications, digital humans, audio and video are more blooming
Real knowledge comes from practice. At the meeting, Zhang Fuzheng, head of Kuaishou MMU Natural Language Processing Center & Audio Center, shared the exploration and application of Kuaishou in AIGC by demonstrating the latest cases of AI generating digital humans, music, and videos.
▲ Zhang Fuzheng, head of Kuaishou MMU Natural Language Processing Center & Audio Center
Zhang Fuzheng first talked about his understanding of AIGC from the perspective of workflow. Based on the workflow starting point of "creators input the ideas they want to express, and synthesize content output through AI technology", Kuaishou uses platform engines, infrastructure, AI atomic capabilities, AIGC AIGC has been laid out in many aspects such as solutions and landing scenarios.
In the digital human scenario, by simply inputting text, users can generate digital human applications with precise mouth shapes and rich expressions/actions within 5 minutes, which can be used in multiple scenarios such as e-commerce, recruitment, anchors, and training.
In terms of smart music, users can generate lyrics and specific melodies that rhyme and are related to the subject words according to the information such as the subject words input by the user. Applications such as AI singers are also close to the singing level of real people.
In terms of video production, AIGC technology is also of great help to creators. For example, the average consumption of Kuaishou video creation "one-click film" exceeds 450 million, and the average daily consumption of copywriting works exceeds 40 million.
2. Moxin Artificial Intelligence Wang Wei: Sparse computing becomes the optimal solution for large-scale model landing, and Moxin leads the evolution of AI 2.0 computing power
On the demand side, the development of digital civilization has undergone fundamental changes. Generative AI has opened the door to building explosively successful applications, and the demand for computing power in the era of AI 2.0 large models has been completely subverted. Compared with the AI 1.0 era of small models, which focused on the versatility of computing power, the structure of large pre-trained models is unified and more focused on scalability. The growth of computing power and the speed of reasoning have become pain points in the development and application of large models.
Wang Wei, founder and CEO of Moxin Artificial Intelligence, said: "It is difficult to rely solely on hardware to meet the demand for exponential growth in computing power, and it must be integrated with software and hardware. In this direction, sparse computing is recognized as the most promising direction for development and implementation. .” Compared with dense computing, sparse computing can achieve a performance improvement of 1-2 orders of magnitude.
▲Wang Wei, Founder and CEO of Moxin Artificial Intelligence
At the beginning of 2021, Moxin’s first high-sparse AI computing chip, Antoum, was successfully taped out, and then Moxin released the Antoum-based AI computing card series S4, S10, and S30 to support diverse AI application scenarios. Based on Moxin's unique double-sparse algorithm and the concept of soft-hard synergy, Moxin's products can take into account high performance and low power consumption, and the double-sparse algorithm can achieve up to 32 times sparseness in many networks and applications.
Through the actual measurement on the 176B open source large model BLOOM, the ink core S30 computing card can achieve a generation speed of 25 tokens/second when only using the low-to-medium magnification sparse rate, and the generation speed of 4 S30 exceeds 8 A100 Speed, greatly accelerating the inference speed.
Wang Wei said: "The rapid development of large models has brought AI chip start-ups the opportunity to challenge giant players, have a brand-new display stage, and use subversive innovation to bring order-of-magnitude performance breakthroughs."
3. Qiming Venture Capital Zhou Zhifeng: Nearly 60% of generative AI entrepreneurs focus on multi-modal applications, and the Chinese ecology may create more opportunities
Over the past 40 years, with the continuous explosive growth of computing power and data, the form of AI technology has undergone exponential changes. How to stand on the new node to predict the trend and plan in advance?
Zhou Zhifeng, partner of Qiming Venture Partners, said that this wave of AI driven by ultra-large-scale pre-training models has shown transformative generalization capabilities and emerging phenomena from the underlying technology, which has solved many problems faced by entrepreneurship in the AI 1.0 era to a certain extent. , including that AI technology is only a small part hidden in end products, society lacks reasonable expectations for AI technology, lacks a complete application development infrastructure and environment, and lacks a listed company and capital market valuation system, etc. AI has once again become a hot spot for entrepreneurship and investment. Within two years of the release of GPT-3 in 2020, the investment of global venture capital institutions in AI companies has increased by 4 times, and there will be 1.37 billion US dollars in financing in 2022 alone.
▲Zhou Zhifeng, Partner of Qiming Venture Partners
Different from the Baidu founder’s view that “China will basically not produce another OpenAI”, Zhou Zhifeng believes that China and the United States have very different ecological environments for AI base models, and China has many unique opportunities. In addition to the high-tech barriers, high talent density, and high-capital demand for large-scale models, young entrepreneurs, veterans of vertical industries, and big names in the AI industry all have different entrepreneurial opportunities in multiple dimensions of technology and application. The technology team of Qiming Venture Capital summed up a "map" of the new wave of AI ecological architecture and key layout areas. Come for venture capital reference.
According to the exchange statistics between the Qiming Venture Capital technology team and more than 100 companies established after 2020, in the field of generative AI entrepreneurship, 14% of the entrepreneurs focus on the underlying technology, 57% of the entrepreneurs focus on multi-modal applications, 29 % of entrepreneurs focus on language applications. Technology-driven startups that can build their own barriers on AI technology and application-oriented startups that can integrate into industrial workflows and provide high business value are more likely to stand out.
4. Round table dialogue: ChatGPT detonated the technological revolution, "winner takes all" or "let a hundred flowers bloom"?
How does the crazy ChatGPT set off a new round of technological revolution? In the roundtable dialogue in the morning, Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology, Sun Bin, President & COO of Zhujian Intelligent, Huang Dongyan, a voice technology scientist at UBTECH, and Liang Yu, a partner of Genesis Partners Capital, discussed issues related to technology, industry, and investment. The explosive questions pushed the atmosphere of the scene to a climax.
▲The round table dialogue session, from left to right: Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology, Sun Bin, President & COO of Zhujian Smart, Huang Dongyan, voice technology scientist of UBTECH, and Liang Yu, partner of Genesis Partners Capital
ChatGPT was born out of nowhere, and was praised by Nvidia CEO Huang Renxun as "the iPhone moment of artificial intelligence". Sun Bin, President and COO of Zhujian Intelligent, which focuses on the implementation of the natural language understanding track industry, said with emotion that the popularity of this product was indeed within their expectations. In addition, the new paradigm change brought about by large models and violent parameters has become a beginning.
UBTECH is a head player who empowers artificial intelligence to service robots. "The emergence of GPT allows us to see that artificial intelligence has penetrated into all walks of life, and the era of service robots entering thousands of households has come." said Huang Dongyan, a voice technology scientist at UBTECH.
▲Huang Dongyan, voice technology scientist of UBTECH
Previously, Yann LeCun, one of the three giants of deep learning, once said that ChatGPT has no special innovation, but is well combined. Liang Yu, a partner of Genesis Partners Capital who focuses on investing in early-stage technology companies, also expressed a similar view. He believes that from a technical point of view, the system integration and engineering of the Transformer architecture has been steadily innovating, and ChatGPT has not achieved a revolutionary breakthrough. . Huang Dongyan agreed with Liang Yu's point of view, and added that OpenAI discovered the "password given by God" in the process of integration and trial and error, making the dialogue achieve an amazing human-like performance.
Subsequently, Zhang Guoren raised a question that attracted much attention: the launch of OpenAI's GPT-4 and the successive releases of Microsoft-related products have made peers feel a lot of pressure. How long is the leading edge of the Microsoft + OpenAI combination expected to last?
In this regard, Sun Bin believes that any combination of performance tools depends on how people use and use them, and ultimately the output of people should be used as the judging standard. The long-term value of their combination lies in whether it can be used in other industries in the future, so that technology and industry can be perfectly unified. Huang Dongyan said that how far the combination of Microsoft and OpenAI can go depends on its technological innovation capabilities and iterative development speed. Of course, there will be a large number of AI companies that may have "dark horse" technologies in this process.
▲Sun Bin, President & COO of Bamboo Smart
In general, Microsoft's approach is actually a model for the development of this industry. Liang Yu said: "To a certain extent, Microsoft is playing an ecological strategy." Industry applications and user access to the ecosystem can form a "data flywheel", and its rolling speed will become faster and faster.
Settled in the domestic environment, large-scale model training also requires good Chinese language learning corpus to roll as a "flywheel".
From Wang Huiwen, the co-founder of Meituan, to Wang Xiaochuan, the founder of Sogou, to Kai-fu Lee announcing the Project AI 2.0 plan, many bigwigs are joining the AIGC entrepreneurial wave. When Zhang Guoren asked whether this wave of entrepreneurship would be "winner takes all" like entrepreneurship in the Internet era, or "a hundred flowers bloom", several guests agreed with the latter.
Sun Bin said that the situation in the era of large models is different from the era of Internet "big subsidies". AI has formed a relatively fixed pattern of entrepreneurial chains. The new large language model brings new underlying engines, but the accumulated industrial ecology and models will be reused , will continue the characteristics of a hundred flowers blooming. He admitted that big language models may still be dominated by big manufacturers in the industry, but breakthroughs in key points depend on science and technology companies.
Liang Yu also agrees with this point of view. He believes that while major companies are leading the way, opportunities for start-up companies may lie in the application layer, cutting into the vertical field for industrial landing, reducing costs and increasing efficiency. "Start-ups must learn to avoid the 'footsteps of the giant beast'. All great companies emerge from small cracks."
▲Liang Yu, partner of Genesis Partners Capital
Talking about the elements that AIGC should have for entrepreneurship, Liang Yu said that the competition, capital environment, and talents of large-scale entrepreneurship have qualitatively changed compared with the previous two years. The density of capital, the density of entrepreneurs, and the density of entrepreneurs have soared. Strong technical background and industry understanding.
Sun Bin believes that, just like the co-founder of OpenAI started his business 8 years ago with the accumulation of technology, resources, and contacts, entrepreneurs now also need to consider three things: computing power, engineers, and data, so that entrepreneurship will get twice the result with half the effort.
Many speakers at the summit pointed the end of AI development to general artificial intelligence (AGI). How to define general artificial intelligence? What is the ultimate problem it wants to solve?
Huang Dongyan believes that general artificial intelligence and vertical artificial intelligence are actually complementary. Intuitively speaking, big companies will provide a general artificial intelligence platform, while vertical artificial intelligence is to do in-depth development in different industries.
Sun Bin said that looking at the present ten years later, this may be the starting point of general artificial intelligence. Liang Yu also mentioned that this year may be the first year of general artificial intelligence. From today, general artificial intelligence may gradually penetrate into all aspects of people's production and life like water and electricity.
Zhang Guoren said that perhaps many sci-fi movies have portrayed the appearance of general artificial intelligence for us, such as Jarvis in Iron Man and MOSS in Wandering Earth. Although the forms are different, they will reach or exceed the level of intelligence Humanity.
▲ Zhang Guoren, co-founder and editor-in-chief of Zhiyi Technology
So, if we look back at today's wave of AIGC application innovation 10 years later, what will be the significance and impact on the technology industry and human society?
Sun Bin believes that if we look at today from 10 years later, this will be a major change in the way computers interact, making people no longer learn complex languages. Liang Yu also mentioned that this year may be the first year of general artificial intelligence. From today, general artificial intelligence may gradually penetrate into all aspects of people's production and life like water and electricity.
From an ethical point of view, Huang Dongyan said: "How to stimulate the positive effects of technology from the perspective of human nature is very important. In ten years, maybe everyone will have their own intelligent robot, so that people can enjoy better after gaining efficiency. life.” Following that, safety and ethics issues became the concerns of industrial people. Huang Dongyan believes that comprehensive governance is required from laws and regulations, company self-discipline, public awareness, and safety ethics standards.
Zhang Guoren concluded that looking back ten years later, there should be many things that are now taken for granted, but will become unusual at that time. From the current perspective, it is like people are now accustomed to using electronic payment, which is different from ten years ago or The relationship between earlier adoption of cash payment methods.
5. Large model session: What is the gap between domestic and GPT-4? Verticalization and localization overtaking
At the large model forum in the afternoon, Zhang Peng, CEO of Beijing Zhipu Huazhang Technology Co., Ltd., explained the pre-training large model, the foundation of the generative AI era. You, the founder of Luchen Technology and the principal of the Department of Computer Science, National University of Singapore, Yang Yang discussed the challenges and practices of training AI large models at low cost.
Shi Jianping, investment partner of Blue Rush Ventures, believes that pre-training large language models is opening the era of cognitive intelligence. The algorithmic practical level brings insights.
1. Zhipu AI Zhang Peng: Large models also have Moore's Law, exploring new paths for GLM
The first guest speaker of the large model forum is Zhang Peng, CEO of Beijing Zhipu Huazhang Technology Co., Ltd. (abbreviation: Zhipu AI). As a pioneer across industry and academia, he shared the technical path and implementation progress of pre-training large models.
▲ Zhang Peng, CEO of Beijing Zhipu Huazhang Technology Co., Ltd.
Pre-trained large models have become the infrastructure of a new generation of AI applications. Zhang Peng said that there is a Moore's Law in the field of large models: the number of parameters in a single model will increase by 10 times every year. When the model parameters reach hundreds of billions, the large model has passed an important threshold of qualitative change, and ChatGPT is a representative.
"We have been working hard." Zhang Peng said that based on the transformation of the achievements of the Computer Department of Tsinghua University, Zhipu AI launched the first multi-modal large-scale model in 2021. Zhang Peng said that training a 100 billion model faces various challenges, and requires patience and time.
Zhang Peng said that the bilingual 100-billion model GLM-130B jointly developed by Zhipu AI, the evaluation report shows that GLM-130B is close to or equal to GPT-3 175B (davinci) in terms of accuracy and fairness. In addition, since last year, Zhipu AI has successively open-sourced the AI code tool CodeGeeX, and released the ChatGLM-6B open source model that can run on a single card. products, greatly lowering the threshold for using large models.
Finally, Zhang Peng emphasized the concept of Model as a Service (MaaS) at the meeting, advocating the provision of flexible deployment methods ranging from pre-trained large models to APIs to multi-level applications, and from cloud to privatization to all-in-one deployment.
2. You Yang from Luchen Technology: Build a large model training infrastructure Colossal-AI to reduce the cost of implementing AI large model applications
The number of AI model parameters has increased tens of thousands of times in just a few years. In the future, AI may be more intelligent and powerful than the human brain. The challenge facing large model training today is that the training cost is extremely high.
While big data and large models are simultaneously improved, how to create more effective optimization methods, reduce costs and increase efficiency to achieve scalable and efficient computing, and reduce the cost of implementing large AI models has become a key pain point in the industry.
Luchen Technology has built a set of efficient distributed AI large model training infrastructure Colossal-AI. It includes three parts: efficient memory management system, N-dimensional parallel technology and large-scale optimization method.
According to You Yang, the founder of Luchen Technology and the president of the Department of Computer Science, National University of Singapore, young professor You Yang, at present, Colossal-AI has become one of the fastest-growing software in the global basic software market, and has been open sourced on Github: https://github. com/hpcaitech/ColossalAI
▲You Yang, the founder of Luchen Technology and the principal of the Department of Computer Science, National University of Singapore, young professor
The N-dimensional parallel technology developed by Luchen Technology has created a variety of parallel strategies including higher-dimensional tensor parallelism, providing a lower-cost and efficient distributed training solution for large models.
In addition, based on Colossal-AI's heterogeneous scheduling system, users only need to write one line of code to realize dynamic management of GPU memory, CPU memory, and hard disk, increasing the model capacity of the hardware by dozens of times.
For example, the previous GPT-3 training with 175 billion parameters required 128 GPUs, while Colossal-AI only required 64 GPUs, significantly reducing the hardware requirements and cost of large models. With the same equipment, Colossal-AI can help users complete model training more quickly and reduce costs.
3. Blue Rush Venture Capital Shi Jianping: Cognitive intelligence has become the frontier of AI, and "AI-First application" has become a trend
Shi Jianping, an investment partner of LanRun Ventures, pointed out that pre-training large language models has ushered in the era of cognitive intelligence, and cognitive intelligence has become the frontier of the next AI.
▲ Shi Jianping, investment partner of BlueRun Ventures
Machines are gaining unprecedented cognitive capabilities such as language generation and understanding, knowledge reasoning, etc. Behind them are AI OS (operating system), basic models and pre-trained models, and reinforcement learning technology.
How do startups embrace the world of cognitive intelligence? Shi Jianping suggested that not all companies have to make large-scale models, and they can also cut in from the vertical application path, such as fine-tuning open source models/third-party hosted models with their own data; they can also cut in from the platform path, such as for downstream vertical application developers Provide training, fine-tuning, management, service and other platform tools.
He said: "The large pre-trained language model has opened up the era of intelligent computing with cognitive intelligence as the core driving force." The foundation of digital civilization is code, and cognitive intelligence will also redefine the way software is built. "AI-First "Application" will become a trend. For example, the new version of Microsoft's search engine, Bing, has already given excellent examples.
"Now the pace of technology iteration is faster than PPT." Shi Jianping laughed. In this context, how does an enterprise establish its core competitive advantage? In his view, the core will be to use its own data to train fine-tuned artificial intelligence models; at the same time, the integration of the intelligent world and the digital world will bring more imagination.
4. NVIDIA Xu Tianhao: The collaboration of software and hardware makes the effectiveness of computing power exceed 50% when training GPT3 in large-scale clusters
The improvement of hardware computing power not only depends on the improvement of chip technology, but also depends on accurately capturing the needs and trends of AI model algorithm evolution. Xu Tianhao, head of NVIDIA's consumer Internet industry solution architect, said: "Find the key points of computing acceleration, and continue to innovate to meet future business needs."
▲Xu Tianhao, Head of Solution Architect for NVIDIA Consumer Internet Industry
In the era of large models, one card is far from being able to carry the training of a model, and more individuals are needed to form machine nodes that can cooperate with each other.
NVIDIA introduces NVLink, NVSwitch and IB technologies. Among them, the Ampere and Hopper architectures build nodes based on NVSwitch, and implement cluster networking through the IB network, so that these powerful individuals can efficiently cooperate to complete the same thing.
Among them, the underlying hardware is the base. In order for developers to use the hardware and actually solve problems, software collaboration is required. Therefore, NVIDIA has been building SDKs and scene applications in the past to solve problems in various industries, among which NeMo Framework is to solve the problems of large model training and reasoning deployment.
So, how to evaluate how much resources are needed to train GPT-3? Xu Tianhao explained a formula: the time consumed = the FLOPS needed to build a large model/the effective computing power of the hardware. Based on the effective integration of parallel methods and a series of optimizations, NVIDIA's NeMo Framework can make the effective performance of hardware computing power reach more than 50% in the process of training GPT-3.
In addition, in order to accelerate the large-scale deployment of large-scale models in enterprises, NVIDIA NeMo Framework also provides an integrated large-scale model solution based on the integration of FasterTransformer and Triton.
5. Yang Fan of SenseTime: The AI production paradigm has undergone major changes, and artificial intelligence will usher in a more prosperous "Great Navigation Era"
At the meeting, Yang Fan, co-founder of SenseTime and president of the large-scale device business group, explained the path to change and the circle of competence of start-ups that have emerged from the "small model era".
▲Yang Fan, co-founder of SenseTime and president of the large device business group
AI is leading a new generation of technological revolution, from AI-assisted scientific research and generative AI to the recently popular chat robot ChatGPT. Big (computing) power works miracles, and quantitative changes lead to qualitative changes. Yang Fan said that the AI production paradigm is undergoing a major shift—the era of large models is coming.
Yang Fan said that in the era of AI "small models" in the past 10 years, deep learning methods for solving single problems and industrialized small model production tools have gradually matured; in the new era of large models, large models as a service (MaaS) have become a new theme, A new AI paradigm was born around the cost reduction of large-scale model infrastructure, an arms race of computing power data, and real-time user feedback.
Facing this trend, SenseTime, which has been rooted in AI for nearly 10 years, has developed a large device called SenseCore, which enables the ultimate large-scale model development experience from four levels: AI native infrastructure, large-scale model production platform, algorithm model service, and industry application.
Yang Fan said that SenseTime will combine years of industry experience to provide high-efficiency, low-cost, and large-scale next-generation AI infrastructure products and services. The whole stack will accelerate the production and deployment of hundreds of billions of large models, and promote data collection, labeling, and management efficiency. Significantly improved and shortened model iteration cycle. At the same time, SenseTime will also provide large-scale model development support services to ensure the implementation of development results.
6. ChatGPT-like session: the battle between To B and To C, big model VS small model
In the afternoon China-like ChatGPT forum, Li Xiaohan, co-founder and vice president of Unisound, Jian Renxian, founder and CEO of Zhujian Intelligent, and Fang Han, CEO of Kunlun Wanwei, respectively explained their own big language model development process and industry insights.
1. Yunzhisheng Li Xiaohan: There are three major laws in the evolution of AI, and enterprises will move towards general and vertical paths
Li Xiaohan, co-founder and vice president of Unisound, said that the emergence of ChatGPT is the biggest feeling for AI start-ups that AGI may become a reality in the next few years, and they are trying to find ways to integrate into the wave of large-scale models.
▲ Li Xiaohan, co-founder and vice president of Unisound
The evolution of AI is showing three laws, from algorithm-centric to model-centric, from small and beautiful to large projects and the withering of intermediate tasks. Intermediate tasks refer to the intermediate tasks that have no independent goals in the previous AI development will gradually be weakened or disappeared.
Before 2022, both giants and AI start-ups are making large-scale models, and the emergence of ChatGPT has verified the feasibility of this road for the industry. "When the model parameters reach a certain scale, there may be 'emergent' capabilities." Li Xiaohan said, and, in terms of cognitive intelligence, after a certain period of development, machines may surpass humans.
At the same time, the large model makes the development of enterprises present two paths, which are to create a general large model service and a large model for vertical scenarios. The general-purpose large-scale model platform of the big factory will form a service externally, and produce a model flywheel at the "cabbage price". Companies facing different scenarios need a more vertical model and pay more attention to the controllability of data and services.
Unisound will focus on the smart medical industry, launch a large model for a specific industry, and provide customers with an enterprise customized large model on top of the industry. Li Xiaohan said that their vision is to move from the industry version to the enhanced general version.
2. Jian Renxian of Zhujian Intelligence: "Large language model + knowledge + application", the future operating system of general artificial intelligence
Jian Renxian, founder and CEO of Zhujian Intelligence, said that large-scale language models will become the operating system of general artificial intelligence, which will bring "two worlds", one is the closed-source world dominated by OpenAI and Microsoft, and the other is the closed-source world dominated by Deepmind. And the open source world dominated by Google. He believes that the combination of startups and large companies will enable innovation to scale and present a situation of "a hundred flowers blooming".
▲ Jian Renxian, Founder and CEO of Zhujian Smart
He further pointed out that the future software paradigm will be an application driven by natural language technology, consisting of "big language model + knowledge + application". He believes that the model itself has no practical value, but the segmented application based on a large language model has practical value, so he led the development of "Model as Application" (MaaS) rather than "Model as a Service" (MaaS).
Jian Renxian believes that ChatGPT-based applications will bring great value to white-collar workers and bring value to enterprises, such as customer service automation, virtual assistants, knowledge management and employee training, and the four application scenarios will be subverted first.
In order to balance the advantages and disadvantages of large language models and small language models, Jian Renxian proposed a formula: "Small natural language processing model + knowledge model + large model = natural language processing dual engines". He believes that small models can make large models more reliable. controllable, usable and interpretable. At present, Zhujian Intelligence has used its natural language processing expertise to integrate existing scalable products with large-scale language models to provide pre-trained models for various industries.
3. Kunlun Wanwei Fang Han: There are three major gaps in the development of large-scale models between China and foreign countries, and the innovation of domestic business models begins with enterprise services
Large-scale models with hundreds of billions of models are beginning to emerge with real general artificial intelligence, and the iPhone moment of AI has arrived. Fang Han, CEO of Kunlun Wanwei, said that ChatGPT may become a milestone in the evolution of carbon-based life to silicon-based life, and it is the second evolution in human history.
▲Fang Han, CEO of Kunlun Wanwei
However, the current gap in the development of large-scale models between China and foreign countries includes three aspects. First, the annual cost of 50 million to 100 million US dollars is the admission ticket for hundreds of billions of large-scale model training. Second, the data quality of Chinese text is not high. The gap is the engineering technology gap.
Fang Han said that OpenAI's efforts in the direction of GPT have proved that general artificial intelligence is achievable. After GPT-3 has just been released and experienced, he believes that GPT-3 represents a milestone for AIGC and will greatly disrupt the field of content generation.
At the application level, Fang Han said that the AIGC industry will follow the logic of reducing costs for the B-side and increasing efficiency for the C-side. It can be seen that Microsoft's layout on the B side is concentrated on major customers in industries such as finance and energy. The reason is that "the data of these enterprises can be combined with large models such as GPT to generate the next generation of production paradigms." On the C side, taking Microsoft's Copilot as an example, it will bring people productivity improvements.
Therefore, Fang Han predicts that the business model innovation of the domestic AIGC industry will first appear in the field of B-end enterprise services, followed by the field of C-end UGC tools.
7. AIGC application innovation session: AI transforms code generation, video creation, art design...
In the AIGC Application Innovation Forum in the afternoon, Hao Yiyang, CTO of aiXcoder, Fan Shuo, Director & Beijing District President of Movie Book Group, and Huang Shengyu, Co-Founder of Nolibox, respectively discussed the collision between AIGC and code generation, Metaverse, design creativity and other industries fusion.
1. aiXcoder Hao Yiyang: GPT-4 brings new changes in code generation, and will promote model expansion to hundreds of billions in the future
At the meeting, Hao Yiyang, CTO of aiXcoder (Silicon Technology), brought a keynote speech on the theme of "Code Generation in the Era of Large Language Model (LLM)".
He said that GPT-4 has brought about new changes in code generation, supporting operations such as longer sequences, more instruction number fine-tuning, and multi-modal (picture input), showing effects that are more suitable for general use. There are many problems, including the lack of relevant documents, dependent libraries and requirements documents, as well as slow speed and information security threats.
In fact, there is a big difference between the program generation model and the language model. For example, in terms of interaction methods, the ordinary dialogue language model mainly focuses on question and answer and continuation, while the program generation model needs to fill in the blanks, complete, and back up. There are still many things that GPT-4 cannot do in code generation. For example, in terms of real-time performance, GPT-4 is not applicable in some code error correction and code completion scenarios that require real-time feedback; the context sequence is still limited. It is difficult to take into account all the context of medium and large projects; there is a large gap between the complete information of the code project and the text of web crawling.
As a start-up that entered the AI intelligent programming robot track in 2018, aiXcoder launched the first domestic code generation pre-training model product aiXcoder XL in June 2022. The product supports full-featured natural language input to full programming language output.
Looking forward to the aiXcoder roadmap, Hao Yiyang said that aiXcoder will promote the expansion of the model from tens of billions to hundreds of billions, add a large amount of mixed data of natural language processing + code, and specially construct instruction data sets for various scenarios in programming, so as to obtain comprehensive performance Better code editing tools.
▲ aiXcoder (Sixin Technology) CTO Hao Yiyang
2. Fan Shuo of Moviebook Group: Building a digital operating system for content terminals, multimodality will usher in an explosion in the next two years
Generative AI has reached the explosive stage, and the explosion of applications brought by it has made people feel the process of AI leaping from perception to cognitive intelligence, and has brought a lot of market space.
ChatGPT makes the text mode reach the explosive period, but the explosive period of pictures, videos, and sounds has not yet arrived in this era. Fan Shuo, Director of Moviebook Group and President of Beijing District, said that with the emergence of industry needs and the improvement of production efficiency, pictures, videos and even digital twins will become irreversible trends in the future, making it more intuitive and convenient for people to obtain information.
▲Fan Shuo, Director of Filmbook Group and President of Beijing District
The entire process of technological change is the same as the process of human development, from solving repetitive tasks to thinking about logic and creativity.
At present, the text mode has brought people productivity improvements. Fan Shuo said: "2023 to 2025 will become the era of multi-modal explosion." In the future, the content of generative AI construction will not only be papers, codes, It is user-defined and self-interactive production.
In addition, for terminal enterprises, "the model is not directly accessible to many enterprises." Fan Shuo said, so the model needs more applications to connect, more platforms to support, and then data structure and content production Only by standardization and process can it be truly applied to the entire content generation process. Moviebook Group has developed an AI digital operating system.
The development of the large-scale model industry will continue to adapt to some repetitive work in the field of content generation. In the future, its system will achieve standardization capabilities, and at the same time connect with corresponding open engines to build an overall ecology, and continue to radiate different modes such as text, sound, pictures, and video.
3. Computational Aesthetics Huang Shengyu: Grasp the "three elements" of transformation and let AIGC drive design creativity
AI is affecting human art. Huang Shengyu, co-founder of Nolibox, shared how AIGC drives the innovation of design and creativity production mechanism.
▲Huang Shengyu, co-founder of Nolibox
Huang Shengyu said that the integration and evolution of design and computing has gone through three stages: the 1.0 era is machine-aided design represented by Adobe, the 2.0 era is digital design tools represented by tools such as Canva, and the 3.0 era is based on AIGC. Direct design. For this purpose, computational aesthetics has introduced tools such as "Graph Universe" and "Draw Universe".
"Intelligent design is to the design industry, just like autonomous driving is to the transportation industry." Huang Shengyu said, "But the design behavior is complex, generally there is no optimal solution, and the optimization function is often not unique and clear."
There are at least three major challenges to realize the controllable design creativity production driven by AIGC: 1. The language incommensurability between humans and machines when it comes to creativity. 2. Acquisition and analysis of complex design creative needs. 3. It is difficult to iterate and implement design generation creative solutions. In this regard, Huang Shengyu proposed "three elements" for the transformation from design cognition to robot cognition: 1. The quantifiability of design assets. 2. The generalization of design experience. 3. Simulability of design behavior.
Founded in 2020, Computational Aesthetics is a technology company incubated by Tsinghua University. It has launched the world's first commercially available visual design dataset. Its core products include the AIGC productivity tool "Drawing Universe" and the intelligent design engine "Graph Universe". One step plan is to launch the AI design creative platform Yeahpix.
Conclusion: Generative AI detonated the innovation of content production and interaction paradigm, witnessing the arrival of a new era of AI
GTIC 2023 China AIGC Innovation Summit ended successfully, but the innovation of content production and interaction paradigms catalyzed by large models and generative AI has just begun.
In this AIGC event, we saw that ChatGPT was born under the development and accumulation of technology in the era of large models, which excited the industry, academia and investment circles. At the same time, this phenomenon-level product has become a milestone event in the era of large-scale models, showing the potential of intelligent emergence for enterprises that have been cultivating large-scale models for many years.
At the same time, the scale of model parameters is exploding, and the difficulty of 100 billion-level model training can be imagined. At the AI chip and computing power level, there are opportunities for large-scale deployment of generative AI products.
Generative AI has brought subversive application innovations. From text, pictures, codes, videos to creative production and metaverse industries, generative AI has opened up infinite space for creativity and imagination for different industries. More and more ChatGPT-like products are commercialized, and AI is penetrating into all walks of life, accelerating the era of general artificial intelligence.
Looking forward to the future, generative AI is leading us into a new world of AI, and ChatGPT, which is advancing rapidly, is setting off a new round of technological revolution.
GTIC 2023 China AIGC Innovation Summit will become an important platform for industry-university-research circles to conduct in-depth exchanges and thoughts on cutting-edge technology and industrial implementation, and invite AIGC companies in various sub-tracks to become witnesses to the arrival of the new era of AI.
Next, Zhishi will make a more complete report on some speeches and summit forums, please pay attention to the follow-up push of Zhishi.
11
share:
A total of 283 articles were published
come on, say a few words
published
related suggestion
03-27
Tim Cook Comes to Beijing, Apple Leaves China
03-27
03-24
03-24
03-24
more information
-
welcome to flirt
join us
Scan the QR code to add me and throw your resume directly
Business Cooperation
Scan the code and add me to chat directly
market collaboration
Scan the code and add me to chat directly
-
Follow us
Smart stuff
car stuff
Core stuff
Smart Things Open Class
Smart things copyright Beijing ICP No. 16059766
Copyright ©2014-2023
zhidx.com. All Rights Reserved