Why OpenAI: The reason why OpenAI can surpass Google is that they are really trying to understand the essence of "learning".
ChatGPT's self-training logic: AI will hide one of the words, and then guess what the word should be, a bit like the machine playing games with itself, so that its language understanding ability becomes stronger and stronger-I think At this point in time, when AI starts to use a large amount of unlabeled data to complete autonomous training, it is only a matter of time before the emergence of ChatGPT
ChatGPT is essentially a statistical model: No matter how amazing ChatGPT is, it is still a statistical language model. It is essentially based on all the data it has seen, and uses statistical prediction results for the next output. When it gets the data When there is logic, it will find out the logic through statistical methods, so that you can feel that its answer is very logical, but if it reads a lot of messy texts, it will also speak without logic, which is a natural result of the statistical language model. Defects.
What is the relationship between statistics and logic: one group thinks that the ultimate simulation can realize logic, although it is only based on statistics, but it seems that logic is actually equal to logic itself; the other group thinks that all simulations can only be roughly correct, and finally It still needs to be based on extremely precise rules. The difference between the two factions may be the difference between statistics and rules.
The debate on ChatGPT's tendencies: As humans use more and more feedback to interfere with its judgment, will that make ChatGPT have a certain trend of thought, such as political leanings? Tested, and found it wasn't completely neutral politically, was a left-wing liberal environmentalist.
About "Constitution AI": This is also proposed by Anthropic, called "Constitution AI". It is whether we can let AI realize self-evolution under a set of constitution, or a set of rules, and finally become in line with this set of rules. For example, each country has its own regulatory system. For example, in the Middle East, only AI that meets local regulations can enter the country. I think this will be a very good direction, which can greatly reduce the cost of AI absorbing information and improve its efficiency.
The impact on NLP practitioners: the traditional NLP methods in the past will definitely not be useful in this era, such as a large number of syntax trees, which are very cumbersome. The development process in the past was painful and required a large team. In order to make a very small thing, we will definitely not go back to that era, and the old technology can basically be replaced by a large language model (LLM, large language model).