ChatGPT ignited the wave of general AI, following the agricultural revolution, industrial revolution, and computer technology revolution, it may also set off an AI technology revolution.
The industry pays more attention to ChatGPT's AI algorithm, but OpenAI has evolved into a platform service serving hundreds of millions of users. In the past 3 months, the SLA of ChatGPT is about 99%, which means that it is unavailable for about 15 minutes a day on average, and the overall technical architecture and reliability have also attracted much attention.
The status map of the OpenAI website in the past 90 days (green indicates availability, others indicate abnormality)
ChatGPT has not officially shared their technical architecture, so it is difficult to know the big picture of the architecture with 100% accuracy. This article tries to focus on the following aspects:
- Internet public information (twitter, linkedIn, etc.)
- OpenAI's latest recruitment job requirements
- OpenAI several failure reports
- Github code
Draw a big picture of ChatGPT's main software technology stack.
1. Cloud service
OpenAI is an AI start-up company in the cloud era. All its businesses are set up on the public cloud. It was supported by AWS in the early days of its business. The following dialogue is OpenAI's answer to the public information by 2021:
The above paragraph comes from the introduction of an early article on OpenAI's official website. It can be seen that a large number of AWS services have been used.
In the past two years, it has received huge investment from Microsoft, and it is clear that Azure will provide services. It is certain that ChatGPT is turning to a multi-cloud solution based on Azure, and AWS services will gradually decrease.
In addition, Terraform multi-cloud management service is used to manage cloud resources.
2. Database
First ask ChatGPT itself, basically there is no clear answer. However, more accurate information can be obtained from the official website positions and fault reports:
The core business data of ChatGPT is stored in the relational database PostgreSQL, which is introduced on the official website, where hundreds of millions of user accounts, AKs, and conversations are stored.
The fault report on 2023.2.20 also shows that there is a problem with the main database PostgreSQL. And mention the use of PgBouncer's connection pool service.
At the same time, the Redis cluster was used as the cache service. On March 20, 2023, a security hole broke out in ChatGPT. Some users could see other people’s chat records, because they stepped on the bug of Redis-py’s session disorder in the connection cancellation state. CEO Sam Altman also apologized on twitter.
In addition, I still hope to understand CosmosDB in the recruitment process. It is not yet confirmed what scenario it will be used in. CosmosDB is a multi-mode database launched by Azure. It supports compatibility interfaces such as MongoDB, Cassandra, PostgreSQL, and Gremlin. It is Microsoft's flagship product in the NoSQL field.
Before Patrick-McFadin (Cassandra Committer) mentioned on LinkedIn that OpenAI uses Cassandra, but in the end the discussion came down to ChatGPT's own nonsense, and it is not confirmed whether Cassandra is actually used or Cassandra's API is used in CosmosDB.
In addition, CosmosDB also provides a distributed PostgreSQL interface through the acquired Citus. It is not confirmed whether Citus is used to complete the distributed database architecture.
ChatGPT uses Snowflake, a new generation of cloud-native data warehouse, and uses Tableau for data analysis. Snowflake, which supports multi-cloud deployment, is very beneficial for cross-cloud migration of business from AWS to Azure.
3. Front end
In terms of Web front-end, it is clear that TypeScript language and React framework are used.
On the mobile side, ChatGPT has not yet officially released an APP, but it is already recruiting iOS and Android engineers, and it should be soon.
4. Application and service programming language
As an AI-based technology company, OpenAI chooses Python as its core language. Whether it is an AI position or a platform software engineer position, it is necessary to be proficient in the Python language.
The platform service chooses Python, uses the popular Flask framework in Python, and uses components such as OpenAPI.
5. AI technical framework
The core of the AI technology framework is Pytorch, which may also use Tensorflow.
OpenAI has developed its own GPU-oriented algorithm framework, Triton, and open sourced it on github (
https://github.com/openai/triton), which is used to replace NVIDIA's CUDA, with the goal of developing machine learning algorithms more efficiently. Triton's programming languages use c++ and python.
The more detailed AI training and deployment service architecture has yet to be explored. The following figure is a typical architecture diagram of a large model shared by Dr. Chen Wei on the Internet, for reference:
Source: https://zhuanlan.zhihu.com/p/611464068
6. Application deployment and monitoring operation and maintenance
Container Service : Kubernetes
Monitoring and operation and maintenance : Prometheues (recruitment operation and maintenance development engineers mentioned that they need to understand PromQL)
Log service: Splunk
Programming languages: Golang, Python
7. Reference documents
1. ChatGPT software engineer job description:
https://openai.com/careers/software-engineer-chatgpt
2. OpenAI 2023.2.20 failure analysis report:
https://status.openai.com/incidents/mq5jgswy45fr
3. Github address of OpenAI open source Triton:
https://github.com/openai/triton
4. Patrick McFadin discusses whether ChatGPT uses Cassandra on Linkin:
https://www.linkedin.com/posts/patrick-mcfadin-53a8046_this-isnt-your-regular-chatgpt-post-so-activity-7031372446536515584-P8fg/?utm_source=share&utm_medium=member_desktop
8. Finally
Author of this article: Ye Zhengsheng, programmer of NineData, CEO of Nine Data, and former senior technical expert of Alibaba Cloud.
NineData official website: www.ninedata.cloud, provides enterprise-level database SQL development tools, data replication, comparison, backup and other products, and provides a SQL service (SQL AI Copilot) similar to ChatGPT, which is free to use and does not need to be downloaded.
Welcome everyone to leave a message or reprint!