3rd week of July 2023 Collection of large models
- 2023.7.25
- Copyright statement: This article is the original article of the blogger chszs, and shall not be reproduced without the permission of the blogger.
1. Huawei releases new AI storage products in the era of large models
On July 14, Huawei released a new AI storage product in the era of large models in Shenzhen, providing storage solutions for basic model training, industry model training, and segmented scene model training and reasoning. The new AI storage products launched by Huawei this time include the OceanStor A310 deep learning data lake storage and the FusionCube A3000 training/push super-integrated all-in-one machine . Among them, OceanStor A310 deep learning data lake storage can realize massive data management in the whole process of AI from data collection, preprocessing to model training, and inference application. The FusionCube A3000 training/push super-integrated all-in-one machine is oriented to large-scale model training/reasoning scenarios in the industry. It can provide an all-in-one deployment experience for tens of billions of model applications.
2. Intel releases cost-effective Gaudi2 accelerator card
On July 11, Intel launched the second-generation Gaudi deep learning accelerator Habana Gaudi2 for the Chinese market. In the evaluation of the large language model GPT-3, Gaudi2 also showed its better performance. For customers running deep learning training and inference workloads in China, Gaudi2 is an ideal choice compared to other products on the market for large-scale generative AI and large language models. In addition to outperforming the A100 in performance, Gaudi2 offers about twice the price/performance relative to the A100 on various state-of-the-art models.
On GPT-3 training, Intel used 384 Gaudi 2 accelerators to complete training in 311 minutes; compared to Nvidia's training time on 512 H100 GPUs was 64 minutes. This means that, based on the GPT-3 model, each H100 outperforms Gaudi2 by a factor of 3.6. Price/performance is an important consideration in the relative value of the H100 and Gaudi2, with the Gaudi2 server costing significantly less than the H100.
3. Meta releases Llama 2, the strongest free commercial model
Llama has been arguably the most powerful open source large model in the AI community. However, due to the open source agreement, it has not been free for commercial use. On July 19, 2023, at today's Microsoft Inspire Partner Conference, Meta announced the deepening of cooperation with Microsoft, officially launching a new generation of open source large-scale language model Llama 2, and making the model free for commercial and research use.
Facing the technical fence that OpenAI and Google are working hard to build, Meta seems to want to find another way to enter this large-scale model competition among giants through ecological opening. Meta's high-profile open source Llama 2 this time is undoubtedly GPT-4 and Google's PaLM 2, which have just taken the "technical confidentiality route" in the front.
The Llama 2 model series released by Meta this time contains three parameter variants of 7 billion, 13 billion and 70 billion. According to the official introduction, compared with Llama 1, Llama 2 has 40% more training data, doubles the context length, and adopts a group query attention mechanism. Specifically, the Llama 2 pre-training model is trained on 2 trillion tokens, and the fine-tuned Chat model is trained on 1 million human-labeled data. According to published evaluation results, Llama 2 outperforms other open source language models on a number of external benchmarks including inference, coding, proficiency and knowledge tests.
This move means that most large domestic models will be upgraded to a new generation.
- Paper address: https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/
- Project address: https://github.com/facebookresearch/llama
4. LG releases EXAONE 2.0, a large multimodal language model
On July 20th, LG officially released the multi-modal large language model EXAONE 2.0, which supports Korean and English, and can be used in the development of new materials and new drugs. According to reports, EXAONE 2.0 has learned "about 45 million professional documents such as patents and papers obtained through partnerships, and 350 million images." In order to solve the high cost problem of super-large AI, EXAONE 2.0 has a lightweight design while processing various information such as large-scale language models (LLM), images, and languages. Currently, EXAONE 2.0 only provides services for the B2B (business-to-business) field.
Comments: The speculation is based on the open source large model project + self-training.
5. Stanford research finds that GPT-4 is "dumb"
Recently, an arXiv preprint paper from Stanford and UC Berkeley gave quantitative experimental results on this problem and published relevant evaluation and response data. Not long after the publication of the paper, this research has attracted widespread attention and discussion, and many netizens agree with the results described in the paper.
Specifically, after studying the results generated by the March and June 2023 versions of GPT-3.5 and GPT-4 through four tasks, the researchers found that these two LLMs did become worse on some indicators, In particular, the ability of GPT-4 to solve mathematical problems can be said to have fallen in an avalanche - the accuracy of the March version was 97.6%, and only 2.4% was left in June. The researchers also speculated about the reasons for these changes. It should be noted that newer versions of LLM do not always produce better results. In fact, even though GPT-4 performed better overall, the June version made mistakes on questions that the March version got right.
There is currently no definitive conclusion on this issue.
6. IDC recently released the "AI Large Model Technical Capability Assessment Report, 2023"
On July 20, the international data company IDC recently released the "AI Large-scale Model Technical Capability Assessment Report, 2023", evaluating the overall strength of China's large-scale models from multiple dimensions such as algorithms, ecology, and service capabilities. Among them, the large models of Baidu and Ali scored relatively high.
7. Many countries have issued laws and regulations to supervise AIGC technology
- China: Recently, seven departments including the Cyberspace Administration of China jointly issued the "Interim Measures for the Administration of Generative Artificial Intelligence Services" (hereinafter referred to as the "Measures"), which will come into force on August 15, 2023. The announcement of the "Measures" provides an important legal guarantee for the healthy development of China's generative artificial intelligence services.
- European Union: On June 14 local time, the European Parliament, the main legislative body of the European Union, passed a draft law called the "Artificial Intelligence Act" (AI Act), which will impose new restrictions on the most dangerous uses of the technology. Restrictions - Voting to ban real-time remote biometric technology, the EU ban means that faces cannot be scanned in real time in public; at the same time, companies such as OpenAI and Google must conduct risk assessments and disclose more data used to create programs. The EU's progress in regulating generative AI could have a huge impact on the field, which is estimated to be worth more than $1.3 trillion over the next 10 years. Violations of EU regulations can result in a company facing fines of up to 6% of annual revenue.
- United States: In late June, the U.S. Department of Commerce announced that its National Institute of Standards and Technology (NIST) would launch a government task force to develop guidelines to address the risks posed by generative artificial Opportunities presented by new technologies.
- On July 21 local time, U.S. President Biden met with the heads of seven leading technology companies in the field of artificial intelligence in the United States. The seven companies are Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI. These companies have voluntarily pledged to the White House to watermark AI-generated content.
- Australia: Recently, the Australian government stated that it will close existing legal loopholes and adopt "safeguard measures" for new forms of artificial intelligence technology. Federal Minister for Industry, Innovation and Science Ed Husic has released recommendations from the Australian National Science and Technology Council, as well as a discussion paper on artificial intelligence, which aims to consider targeted regulatory measures.
- United Kingdom: In the United Kingdom, several national regulators are tasked with drafting rules covering artificial intelligence, including the Financial Conduct Authority, according to Reuters. Currently, the agency is consulting with the Alan Turing Institute and other legal and academic institutions to improve its understanding of AI technology.