At the beginning of the new year, the topic of large models remains unabated. The amazing capabilities demonstrated by ChatGPT have pushed the research and application of large models to a climax, and people have heatedly discussed what the launch of this advanced "species" means.
The author of this article, Rob Toews, released the top ten predictions for AI development in 2023. On the whole, most of the predictions are inseparable from the keyword "big model", and specific analysis is also reasonable. Of course, the development of Wen Shengtu, humanoid robots and other fields also plays a decisive role. 2023, let us wait and see.
Link:
https://mp.weixin.qq.com/s/E_v7k_VlbHA8of8smlqikQ
2. The evolution of the heart of the machine / Understanding the AI-driven software 2.0 intelligence revolution
This article will take you through a magnificent development history in the field of artificial intelligence, from four dimensions: the academic progress promoted by key figures, the emergence of algorithms and ideas, the progress of companies and products, and the iterative impact of brain science on neural networks. A deep understanding of "the evolution of the heart of the machine". Forget about those fancy image production applications, let's learn something close to the essence of AI.
Link:
https://mp.weixin.qq.com/s/5s1hLaXnWVPSuElkGMhXxw
3. The Lone Warrior of AGI, legendary engineer John Carmack: Surprised not to see someone like me
AGI is the holy grail of artificial intelligence. The pursuit of AGI by Dallas' most famous technological wizard is like a once-in-a-century mission to the moon. In the race for AGI, there is also a separate "groupthink" competition from scientists, academics and big tech companies, who are also actively seeking solutions.
Last August, Carmack announced that his AGI startup, Keen Technologies, had raised $20 million in a new funding round from several high-profile investors. In December 2022, Carmack resigned from the leadership position of Meta's virtual reality department to devote himself to AGI research and development.
Link:
https://mp.weixin.qq.com/s/MMfWc6ss8w8QgnC_-cUnwg
4. The success of Sam Altman, the head of OpenAI
Today, Sam Altman, who has become the CEO of OpenAI, is a well-deserved technology leader in the world. In the year after he left YC president, Sam Altman, who had communicated with countless entrepreneurs and technical talents, published a blog summarizing the 13 characteristics he needs to be successful. If you are eager to succeed, or at least want to Be great yourself, this blog will inspire your personal growth. You're lucky if you happen to watch it early in your career.
Link:
https://mp.weixin.qq.com/s/AHEbDPSCUEvRrdq9zn5YmQ
5. ChatGPT, and cleverly designed Infra
The author mainly has four points of view: ChatGPT is not a black technology, but a product of continuous open scientific research; ChatGPT is a victory for engineering and products; ChatGPT will not make people unemployed, but will bring more opportunities; Infrastructure will be this game Win the battle, but design Infra smartly.
Link:
https://mp.weixin.qq.com/s/oM0V0MymMbanJddzABYDDQ
6. Open Assistant: LAION launched the open source ChatGPT project
The effect of ChatGPT is amazing, but it is not open source. Apparently someone in the open source community couldn't hold back. Christoph Schuhmann, the organizer of LAION (the representative work is the famous data set LAION-5B), started the project with a video call with Yannic Kilcher (a well-known YouTube big V in the AI circle), and soon the Discord discussion group of the project entered a lot People, very active. The original project name was open-chat-gpt, and it was changed to Open Assistant about a week later.
Link:
https://hub.baai.ac.cn/view/22872
7. Domestic developers initiated the ChatRWKV project to make open source ChatGPT
ChatRWKV is similar to ChatGPT, but powered by RWKV (100% RNN) language model, which is currently the only RNN that can match Transformer in quality and scaling, while being faster and saving VRAM.
Link:
https://zhuanlan.zhihu.com/p/603840957
8. The road to AGI: the essence of large language model (LLM) technology
After the appearance of ChatGPT, many people were surprised or awakened. The surprise is because I didn't expect the effect of the large language model (Large Language Model) to be so good; the awakening is the epiphany that our cognition and development concept of LLM is far from the most advanced ideas in the world. According to the author, as a group of people who were surprised and awakened, they are also typical Chinese people. Chinese people are good at self-reflection, so they began to reflect, and this article is the result of reflection.
Link:
https://mp.weixin.qq.com/s/eMrv15yOO0oYQ-o-wiuSyw
9. Jeff Dean tweeted: Google's year-end summary "the third bullet" and vigorously develop Jax
As algorithms and hardware become more complex, and operations scale larger, so does the complexity of the software required to perform everyday tasks.
In this post, the researchers outline numerous advances in ML systems across Google over the past year that have enabled Google to support serving and training of complex models while easing implementation complexity for end users. At the same time, the article also mentions how Google uses ML itself to improve and design the next generation of system stack research.
Link:
https://mp.weixin.qq.com/s/TVMYYPK_Ct_dEROzrBnZvg
10. Like TensorFlow, will Nvidia's CUDA monopoly be broken?
Google had great advantages in machine learning model architecture, training, and model optimization in the early days, but now it is difficult to give full play to these advantages. In terms of hardware, it is difficult for other AI hardware companies to weaken Nvidia's dominance. Until the advent of PyTorch 2.0 and OpenAI Triton, the default software stack for machine learning models will no longer be Nvidia's closed-source CUDA.
Link:
https://mp.weixin.qq.com/s/dGpf6DOyaozMwpOtp8vS-g
11. OneFlow v0.9.0 is officially released
This update contains 640 commits. For a complete update list, please check the link: https://github.com/Oneflow-Inc/oneflow/releases/tag/v0.9.0. Welcome to download and experience the new version, and look forward to your feedback. OneFlow v0.9.0 mainly includes 9 new highlights and optimizations.
Link:
https://mp.weixin.qq.com/s/8Vb9fIQs0vSiM5_0M3SaGg
OneFlow recently ported the original GLM project to the One-GLM project which uses the OneFlow backend for training. Thanks to the seamless compatibility between OneFlow and PyTorch, we ported GLM quickly and smoothly, and successfully ran through the pre-training task (training GLM-large).
In addition, since OneFlow natively supports many functions and optimization technologies of DeepSpeed and Apex, users no longer need these plug-ins to train large models such as GLM. More importantly, the performance and memory usage of the GLM model ported to the current OneFlow can be greatly improved after simple tuning.
Link:
https://mp.weixin.qq.com/s/dkTGXuJV38KuLb4_LmM20Q
13. A16Z: Generative AI platform, who is in control?
Unlike many hot technology trends that get over-hyped before the market catches up, the generative AI craze is accompanied by high market popularity and real market benefits. Models like Stable Diffusion and ChatGPT are setting historical records for user growth, with some apps achieving $100 million in annual revenue within less than a year of launch.
The comparison showed that AI models outperformed humans by orders of magnitude on certain tasks. There is enough early data to suggest that a paradigm shift is taking place.
Link:
https://mp.weixin.qq.com/s/bh5uw06IzTCO9jQBa-rlfQ
14. 35 pictures, intuitive understanding of Stable Diffusion
The Stable Diffusion is versatile and is a versatile model. First it can generate images from text (text2img). The image above is an example from text input to image generation. In addition, we can also use Stable Diffusion to replace and change images (we need to enter text and images at the same time).
This article introduces the internal structure of Stable Diffusion. Understanding the internal structure allows us to better understand the composition of Stable Diffusion, how each component interacts, and the meaning of various image generation options/parameters.
Link:
https://mp.weixin.qq.com/s/8C2RqYrHZTpFFzaHIbPhRw
15. "Zero" code changes, static compilation doubles the reasoning speed of Taiyi Stable Diffusion
Recently, the OneFlow team has adapted the OneFlow backend for Taiyi Stable Diffusion, which has greatly improved the inference performance, and can also produce maps in one second. Many developers are curious about what optimization "secrets" OneFlow uses, which will be briefly explained later.
Link:
https://mp.weixin.qq.com/s/XaR1W8yKPYxN5PR1RPMepA
Since most teams currently develop based on the translation API + English Stable Diffusion model, when using the unique Chinese narrative and expression, it is difficult for the English version model to give the correct matching picture content, which is difficult for some domestic users not very convenient.
To this end, Zhiyuan Research Institute produced the first AltDiffusion that supports 9 languages. Recently, the OneFlow team has adapted the OneFlow backend for it, which has greatly improved the inference performance, and can also produce maps in one second.
Link:
https://mp.weixin.qq.com/s/whJlFifyzcCAX5DqA7hA_A
17. Runway released the video generator GEN-1, the result is 73.83% higher than Stable Diffusion 1.5, and 88.24% higher than Text2Live
Founded in 2018, Runway is an AI video editing software provider. It mainly provides a series of tools and platforms for designers, artists and developers. The products are to help professionals generate various content. Its published GEN-1 can actually and consistently synthesize new videos by applying the composition and styling of image or text prompts to the structure of the source video, with amazing demonstrations. GEN-1 is currently still in closed beta.
Link:
https://hub.baai.ac.cn/view/23978
everyone else is watching
-
Faster than fast, open source Stable Diffusion refreshes the drawing speed
-
OneEmbedding: Training a TB-level recommendation model with a single card is not a dream
-
GLM training acceleration: up to 3 times performance improvement, 1/3 memory saving
-
Welcome Star, try the latest version of OneFlow: https://github.com/Oneflow-Inc/oneflow/
https://github.com/Oneflow-Inc/oneflow/