On November 30, the open source generative AI platform Together AI announced on its official website that it had received US$102.5 million (approximately 730 million yuan) in Series A financing. This round was led by Kleiner Perkins, with participation from Nvidia, Emergence Capital, NEA, Prosperity 7, Greycroft, and others.
Public information shows that Together AI was founded in June 2022 and is headquartered in Silicon Valley, San Francisco, USA. It received US$20 million in seed round financing on May 15 this year and is a very fast-growing enterprise in the field of large model open source.
In terms of products, Together AI firmly believes that open source is the future of large models. Since its establishment, it has released a number of open source products that have been well received by enterprises and developers.
They are the ChatGPT-like open source model RedPajama-INCITE, the open source 30 trillion training data set RedPajama-Data-v2, and the open source large language model training and inference accelerator FlashAttention v2.
Below, the "AIGC Open Community" will introduce these main open source products with their addresses.
RedPajama-INCITE: is a ChatGPT-like open source model released by Together AI in early May this year. It mainly includes two parameters: 3 billion and 7 billion. It is characterized by powerful computing capabilities. It consumes low power and can be run on laptops and ordinary graphics cards. It is suitable for small and medium-sized enterprises and individual developers.
3 billion parameter open source address: https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1
7 billion parameter open source address: https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-7B-v0.1
RedPajama-Data-v2:In March this year, Together AI opened up a 1 trillion training data set for the first time, which was highly praised by developers and downloaded nearly 200,000 times.
Recently, RedPajama-Data-v2, a 30 trillion training data set specially used for training large language models, has been open sourced. This is currently the largest open source data set.
This data set contains100 billion text documents, 30 billion of which have been tagged, supporting 5 languages: English, French, Spanish, German and Italian. Language
30 trillion open source training data sets
github address: https://github.com/togethercomputer/RedPajama-Data
Huggingface address: https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2
FlashAttention v2: This is a large language model training and reasoning accelerator released by Dr. Tri Dao, Chief Scientist of Together AI, on July 17 this year. It has been used by OpenAI, Anthropic Used by well-known technology giants such as , Meta and so on, its performance is evident.
FlashAttention-2 improves the training and fine-tuning efficiency of large language models by 4 times and achieves 72% model FLOP utilization in training on NVIDIA A100.
It has almost become the most used training and inference accelerator for Transformer architecture models, supporting more than 100 mainstream large language models.
FlashAttention v2 open source address: https://github.com/Dao-AILab/flash-attention
In addition, as the influence of ChatGPT continues to expand its audience and start-ups grow exponentially, Together AI has also added a new generative AI cloud platform service.
Together AI cooperates with NVIDIA, Crusoe Cloud, Vultr and other partners to operate a cloud computing center in Europe and the United States to provide inference, training and other services for generative AI start-ups. For example, Vincent video platform Pika Labs, which just received US$55 million in financing, is their loyal user.
The material of this article comes from the official website of Together AI. If there is any infringement, please contact us to delete it.