What did the development of the innovative ChatGPT bring to the company behind it, OpenAI?
According to The information, the birth of ChatGPT not only brought unprecedented attention to OpenAI, but also attracted Microsoft's "multi-year, billions of dollars" investment expansion.
What can't be ignored is that for this startup company that developed ChatGPT, because the large model requires huge computing resources and data, and the operation behind it also pays a lot of money, only in 2022, OpenAI's total losses amounted to $540 million, compared with only $28 million in revenue it generated.
*How much does it cost to train and run ChatGPT? *
If you want to ask why OpenAI has lost so much, many people will say it directly: if you don’t spend money, you can’t make a big model.
In fact, since OpenAI launched ChatGPT in November last year, the outside world has been highly curious about the operating costs of ChatGPT.
For a long time, OpenAI has not directly disclosed the specific overhead cost. However, many netizens still found clues and deduced the general idea.
It is reported that as early as 2020, OpenAI used Google cloud services, which cost up to 75 million U.S. dollars at that time. It was not until later that Microsoft's Azure platform was used that the spending on cloud service hosting was gradually reduced.
However, Microsoft previously revealed on a blog that "to support the implementation of the OpenAI model, it not only used tens of thousands of GPUs, but also spent hundreds of millions of dollars":
At that time, in order to support OpenAI to train large models, Microsoft developed a new set of Azure artificial intelligence supercomputing technology, and also established supercomputing resources in Azure. The design and specificity of these resources enabled OpenAI to train an increasingly powerful AI model.
To train this set of models, Microsoft uses thousands of Nvidia AI-optimized GPUs in the infrastructure, which are connected in a high-throughput, low-latency network based on Nvidia Quantum InfiniBand communication for high-performance computing.
In this regard, Phil Waymouth, a key figure in the cooperation between Microsoft and OpenAI, Microsoft's senior director in charge of strategic partnerships, said that the scale of the cloud computing infrastructure required by OpenAI to train its models is unprecedented, larger than anyone in the industry has tried to build. Network GPU clusters are much larger. Later, Bloomberg reported that Microsoft had spent hundreds of millions of dollars on the project.
It is based on this that Microsoft's Azure cloud now hosts the ChatGPT project so that OpenAI does not have to invest in a physical computer room.
Some foreign media have analyzed that considering Microsoft's current charging standards, the cost of an A100 GPU is $3 per hour, and the cost of each word generated on ChatGPT is $0.0003. At least eight GPUs must be in use to run on a ChatGPT. Therefore, based on the fact that the replies generated by ChatGPT usually have at least 30 words, it will cost OpenAI nearly 1 cent for each ChatGPT reply. By such estimates, OpenAI is likely to cost at least $100,000 per day or $3 million per month in operating costs.
In addition, Dylan Patel, principal analyst at research firm SemiAnalysis, said in an interview that as many as 700,000 OpenAI dollars can be burned per day when using ChatGPT to write cover letters, generate course plans, and polish profiles on dating apps. Dollars, it costs 36 cents per query. This is because ChatGPT requires a lot of computing power to respond quickly to the user's prompt words.
At the same time, from the perspective of computing power, analysis shows that currently an NVIDIA A100 GPU can run a model with 3 billion parameters in about 6 milliseconds. At this rate, a single NVIDIA A100 GPU would take about 350ms to output a word on ChatGPT. Since ChatGPT-3.5 has more than 175 billion parameters, and currently ChatGPT can output about 15-20 English words per second, if the user wants to obtain the output result of a single query, at least 8 A100 GPUs are required to perform the operation.
In fact, whether a large model is closed source or open source, it is very costly, as long as the practitioners engaged in AI models know it. Previously, when Meta released the open source LLaMA large model, it also mentioned in the paper, "When training a 65B parameter model, our code processes about 380 tokens/sec/GPU on 2048 A100 GPU and 80GB of memory. . This means that training on our 1.4T labeled dataset takes about 21 days."
For this reason, some Weibo blogger @BohuTANG also made a conversion:
2048 GPUs * 21*24 * 1$ ~ 100w knife, this is the training cost after determining the data set and parameters
Undoubtedly, even if ChatGPT with a larger number of parameters and a wider data set does not reveal the specific training and operating costs, just think about it, it will not be low.
*Attracting top talents is also a huge expense*
Of course, the root cause of the huge gap between OpenAI's expenditure and revenue is not only the high cost of training language models, but also part of the cost of hiring expensive technical personnel.
In February of this year, foreign media reported that OpenAI had poached more than 12 AI technical experts from Google. And just before OpenAI launched ChatGPT, at least 5 former Google researchers participated in it.
Among them, former Google researchers Barret Zoph, Liam Fedus, Luke Metz, Jacob Menick, and Rapha Gontijo Lopes all appeared in the acknowledgment section of OpenAI's official ChatGPT blog post.
At the same time, OpenAI co-founder and chief scientist Ilya Sutskever was also a member of the Google Brain team. In 2012, he jointly proposed the famous AlexNet with Alex Krizhevsky and Geoffrey Hinton, which promoted the revolution of deep learning. In 2015, he left Google and joined OpenAI the following year. Today, he is also one of the most important leaders behind ChatGPT and GPT-4.
To recruit these top talents, OpenAI not only creates a comfortable academic, research, and business atmosphere, but also salaries and benefits are also indispensable.
In 2016, OpenAI paid Ilya Sutskever more than $1.9 million, according to the New York Times.
In addition, according to OpenAI's official recruitment information, it is recruiting more than 40 positions, and the salary level of ordinary ChatGPT software engineers and machine learning research scientists is between US$200,000 and US$370,000 (approximately RMB 1.382 million to RMB 2.557 million) .
The higher the level of the supervisor, the salary will also increase accordingly. For example, the salary of the ChatGPT mobile engineering supervisor is between US$300,000 and US$500,000 (approximately RMB 2.074 million-3.456 million).
In comparison, if you directly poach talents from the past, the estimated salary will be higher. After all, ability and salary are often directly proportional.
*Foreign media: OpenAI CEO plans to raise $100 billion to fight for the realization of AGI*
Of course, in order to ensure the pace of OpenAI’s sustainable research on AI, sources revealed that OpenAI CEO Sam Altman may intend to raise up to $100 billion in funding in the next few years to achieve the company’s goal of creating general artificial intelligence (AGI).
Realistically speaking, as the number of ChatGPT users increases, the expenditure will increase.
On the one hand, in order to support language models and train new versions, OpenAI expects to invest more in cloud computing than ever before.
On the other hand, as Reddit, StackOverflow, Twitter and other platforms have expressed their desire to refuse to be "white prostitutes", they want to charge AI model companies for the accessor's previous free data sets. Earlier, the New York Times reported that Elon Musk terminated OpenAI's access to Twitter's dataset after discovering that it was only paying $2 million a year. Therefore, in the new year, in order to obtain more new data sets, OpenAI's data costs will undoubtedly soar.
Finally, though, at the beginning of this year, Microsoft said it would invest billions of dollars in OpenAI. However, according to The Verge, as part of this investment, Microsoft has an upfront right to 75% of OpenAI’s profits until it recoups the $10 billion invested, plus an additional $3 billion that Microsoft invested in the company earlier. Then when OpenAI's profits reached $92 billion, Microsoft's share of OpenAI's profits fell to 49%. Finally, when OpenAI's profits reach $150 billion, OpenAI will acquire 100% of the company's shares. During this time, Microsoft also gained priority access to OpenAI's software, enabling it to resell OpenAI's software to its Azure customers, along with its artificial intelligence products utilizing OpenAI technology.
For this reason, it is not surprising that OpenAI's losses doubled last year. However, as OpenAI launches ChatGPT Plus, API and other paid services are gradually on the "right track", OpenAI expects revenue of 200 million US dollars in 2023. Some investors also believe that ChatGPT may become a profitable tool for the company at some point in the future. At the moment, OpenAI has at least achieved a leading position in the field of large models.