Questions for App Developers: Are Large Models the Real Silver Bullet?

A large model that is regarded as a silver bullet

After ChatGPT became popular, the large model seems to be regarded as a real silver bullet, and all experience problems should be solved through the large model:

Can you talk to the big model to book a ticket?
Natural language generates SQL to simplify report analysis?
Large model to help the elderly operate software?
Can it be used to identify sensitive information?
......

It seems that large models are the real silver bullet in the field of natural language engineering. But I vaguely remember what Fred Brooks, the author of "The Mythical Man-Month", said "There is no silver bullet in software engineering"; Foam". Here I want to briefly discuss with you the limitations of the large model and the real applicable scenarios. I am not a professional in algorithms, and I hope that the algorithm masters will express their opinions more.

Are mockups the silver bullet?

Connecting some software functions to large models, problems such as accuracy may also be solved through a lot of training. But when it comes to end users, the following problems may cause the large model to not be the optimal solution.

expensive fees

Wu Jun's book "Top of the Wave" believes that an important reason for the rapid expansion of the Internet and computer software industry for so many years is its low expansion cost. In traditional industries, such as Ford Motor, every time a car is sold, it must pay the production cost of a car, and even expand the factory building, etc., which ultimately makes the scale expansion of Ford Motor no longer cost-effective, and has to give up market share to other companies. manufacturer. However, computer software can be replicated and expanded at almost zero cost, and Internet software adds a user with almost no server costs, so it is easy to form a winner-takes-all situation. This matter may not be the same in large model software. OpenAI's ability to expand the free use of large models to such a large scale is largely due to Microsoft's investment. According to rumors, Microsoft has invested tens of billions of dollars in OpenAI. Let's not discuss the cost-recovery of hundreds of millions of training expenses spent by big manufacturers for the time being. It is not a good deal to only look at the cost of API calls. At present, the qps of each machine of the application I maintain is about 300 on average (based on the average of 8 hours a day, non-peak). If such a machine on Alibaba Cloud is based on 2M bandwidth, the annual rental fee is about 3373 yuan. It only costs 9 yuan per day. And if the application is fully connected to the large model, each call is generated by the large model. Currently, Open AI is charged by token. The price of the cheapest GPT-3.5 Turbo model is $0.0015 per 1000 token input, $0.002 For every 1,000 token output, we simply estimate it as 2 cents per 1,000 token input and output, which is 0.02 yuan.

Even if each request only costs 10 tokens, assuming the machine is 200 qps, 8 hours a day, it will cost 0.02*(200*60*60*8*10/1000)=1152 yuan a day. Models consume more than 100 times the cost of the application server.

It is even more unknown what the cost of Open AI itself is. Some people even think that Open AI is selling at a loss in order to quickly seize the market.

This greatly increases the scale cost of the application, and it is almost impossible to be a product that is completely free for users to use.

Although I believe that with the advancement of technology in the future, the cost will drop significantly, but the high probability is not recently.

slow computing speed

For ChatGPT's pure chatbot, the problem of slow computing speed can be alleviated by streaming output word by word. However, for application engineers who want to generate interface parameters or SQL through it, they must wait for it to be completely generated before calling the interface to return the result to the user.

The development of high-performance computers for so many years has made users accustomed to the quick-response interface. The time of modern people is very precious, and it is impossible to wait for a longer time in order to save a few steps.

redundant function

Large models are powerful, capable of answering scientific questions, writing poems, making up stories, and even generating software to decipher serial codes. . .

But what is the use of these functions for me to generate application interface parameters? On the contrary, it is easy to cause legal risks. Users can guide the large model to answer the answer with prejudice and discrimination through simple "AI poisoning". However, preventing this requires a huge price, and it is even impossible to prevent it. For example, the very interesting ChatGPT before Granny bug[1].

Whenever a technology is hot, engineers are always eager to try. When big data is booming, even if there are only a few pieces of data in the system, Flink must be used. The large model also has similar problems. Just to generate several CRUD interface parameters, a large model that is expensive, slow and prone to legal problems is used. While ignoring the advantages of simple, fast and easy-to-control traditional computing.

What are the NLP techniques other than large models?

There is a very intuitive difference between the large model and traditional NLP technology in terms of computing power consumption and capabilities, so it can be popular. What is the definition of a large model? The Wikipedia entry [2] defines it as a deep learning model with more than one billion parameters in the neural network. All the large models are actually derived from the Transformer paper published by Google in 2017. We temporarily think that all Transformer-based models are large models. The following figure is the source of the large-scale model development tree that is very popular on the Internet [3]:

Before the big model became popular, although there were Tmall Genie, HKUST Xunfei and other dialogue robot products in China, it seems that there were not many applications that provided functions through natural language. Even DingTalk, an application specializing in IM, seems to have no idea of realizing application functions through natural language.

But according to our foreign friends, because of the high labor costs abroad, many of their applications provided functions through dialogue a long time ago. They even have an app for paying electricity bills that supports paying electricity bills through conversations.

I am not a practitioner in the field of NLP myself, so I can only offer suggestions for NLP technologies other than large models:

rule language model

1. In fact, it is hard coding that programmers often say. It uses a method similar to regular pattern matching to process natural language. Although it sounds very "Low" now, some products used this before the big model became popular. Although this algorithm The speed is fast, but it consumes manpower and professional knowledge. Fortunately, some language experts have already made some open source frameworks, such as ChatterBot[4], Will[5], etc. Will was integrated into Slack in 2018.

statistical language model

1. It is no longer necessary for programmers to encode rules, but to use some statistical methods (such as tfidf, principal component analysis) to calculate the characteristics of sentences, such as the frequency of words, which words often appear together, and so on. Use this statistical knowledge to calculate new sentences, common theme analysis, sentiment analysis, etc. are similar techniques.

2. It works well for simple text classification. Many online spam automatic identification is said to use this technology.

neural language model

1. The NLP technology we are most familiar with now, before the big model, there are RNN, LSTM, etc., which were later crushed by the big model based on Transformer.

future

I think the big model is not a silver bullet, it may develop in two directions in the future:

Text-based assistants, such as writing assistants, oral language coaches, expert consultation, etc., are also the old line of big models;
Integrated automation vendors, such as Zaiper[6], Alfred[7], etc., make a unified automation assistant payment product, and all software natural language operations are performed through a unified entrance. This is more convenient for users, instead of going to a separate assistant for each application to ask questions.

For other simpler tasks such as text classification, topic recognition, and sentiment analysis, perhaps traditional NLP has a more suitable solution.

As an application developer, not an AI professional, there may be many unprofessional places in this article. This article is purely an introduction, hoping to attract more professionals to discuss.

Reference link:

[1] ChatGPT "Grandma's Vulnerability" became popular, and it was able to deceive the Win11 secret key. The information security issue is thought-provoking! - Know almost

[2] https://zh.wikipedia.org/wiki/Large language model

[3]https://github.com/Mooler0410/LLMsPracticalGuide

[4]https://github.com/gunthercox/ChatterBot

[5]https://pypi.org/project/will/

[6]https://zapier.com/

[7]https://www.alfredapp.com/

Author｜Xuan Heng