I thought that ChatGPT had been popular for a while at the end of last year, and the popularity would gradually drop. Unexpectedly, its popularity continued unabated. I have also been paying attention to and using it since the end of last year, including ChatGPT web version, GPT3 API and ChatGPT internal test API, as well as the experience of other artificial intelligence algorithms at home and abroad.
For the convenience of use, I took the time to develop a simple RdChat program to encapsulate these AI interfaces. As the popularity of ChatGPT increased, the server crashed this morning. In order to repair the server, I had to shut down, restart, change the configuration and so on. During the whole process, users cannot interact with RdChat at all. This is obviously unscientific, but I am too amateur.
Solving this server problem is naturally associated with distributed. My initial idea is shown in the figure below, use one server for data exchange and basic business logic, and use multiple sub-server nodes to complete AI interface tasks at the same time. Of course, these AI interfaces may not be limited to ChatGPT.
1 Deployment Features
After rushing to work all day, I built the above-mentioned complete system, including a master server and multiple sub-node servers. The deployment effect mainly has the following characteristics.
- Each sub-node server is hot-swappable, and can be used immediately after access.
- The number of sub-node servers can be reduced at any time according to the business or user scale.
- The abnormal function of some nodes affects the main program function of RdChat.
- For child node functions, non-stop updates are possible.
- The performance pressure of a single host is relieved.
2 deployment effect
The distributed deployed ChatGPT application is more stable and more flexible to upgrade.