Big data analysis company Databricks Inc has also recently joined the competition in the field of generative AI, releasing an open source large-scale language model called Dolly . The model was named Dolly to pay tribute to the first cloned sheep, Dolly.
Generative AIs like ChatGPT and Bard use data from tens of thousands of different websites, and the amount of data they use is staggering, and it takes thousands of powerful GPUs to train AI using this data. Provide support behind.
Databricks hopes that by open-sourcing Dolly and its training data, anyone can develop a truly human-like AI without investing millions of dollars, and this type of AI is no longer something that only large technology companies can afford , and millions of small companies will also be able to benefit from it.
In addition, allowing each small company to build its own model without sending the data through an API to the service provider behind the proprietary model can also protect the sensitive data and proprietary intellectual property of the small company. In addition, some companies may have different trade-offs in model quality, cost, and desired behavior, and these companies can also adjust the model according to their needs.
Ali Ghodsi, CEO of Databricks, said that Dolly only needs very little data and very short time to complete the training.
For $30, a server, and three hours, we can teach Dolly to start interacting at a human level.
Instead of creating its own model from scratch, Databricks based it on GPT-J, a natural language processing model that EleutherAI open-sourced in 2021. GPT-J is a natural language processing AI model based on GPT-3 consisting of 6 billion parameters. The model is trained on an 800GB open-source text dataset and is comparable to similarly sized GPT-3 models.
Despite a much smaller model, at 6 billion parameters, and a smaller dataset and training time (ChatGPT's 175 billion parameters), Dolly still exhibited the same "magical humanity" that ChatGPT demonstrated, Databricks said. Interaction capabilities".
Databricks has currently open sourced a simple Databricks notebook on GitHub , which developers can use to build Dolly on Databricks themselves.