ChatGPT has a variety of advanced features. Once released, it has attracted much attention. As a large-scale natural language processing model trained by OpenAI, it can realize a series of functions such as natural language generation, language translation, natural language understanding, and language summarization . Two months after its release, the monthly active users exceeded 100 million, making it the fastest growing consumer app in history.
In fact, the launch of ChatGPT officially marks the first year of generative AI commercial use. Generative AI is to learn content or objects from data through machine learning methods, and then generate new and completely original new content based on the learned model. It has been widely used in various fields, such as natural language processing, image generation, audio generate etc.
Gartner predicts that by 2025 , generative AI will account for 10% of all generated data , and the current proportion is less than 1% . The commercial prospects of generative AI are broad, and its commercial application directions mainly include:
For smarter information retrieval and processing. Recently, Microsoft announced that it will launch a new version of the Bing search engine that integrates ChatGPT . ChatGPT can partially replace the search engine function, retrieve existing knowledge bases based on user questions, and provide more intuitive answers. In the future, ChatGPT is expected to be connected to a full set of Office tools to assist users in summarizing, extracting, and translating information.
Vertical services for professional fields. Generative AI can be widely used in professional service fields such as e-commerce, advertising marketing, and coding. It can replace some primary professional work and become a human assistant, helping companies save a lot of labor costs and improve production efficiency.
But how does ChatGPT integrate with existing technologies? Let's start with ChatGPT+ knowledge map and ChatGPT+ office automation as examples:
1. ChatGPT + Knowledge Graph
Factual errors are a relatively big problem that ChatGPT currently has. When answering some questions, it will inevitably give people a feeling of " serious nonsense " . The solution is how to intervene in it and introduce external knowledge is processed,
One way to introduce external knowledge is to provide links during the answering process. Although there are factual errors in the answers, manual verification can be performed through the links to solve the problem of factual errors.
Another way to introduce external knowledge is the knowledge graph. The knowledge map is a knowledge base based on binary relationships, which is used to describe entities or concepts in the real world and their relationships. The basic unit is the [entity- relationship - entity ] triplet , between entities Interconnected through relationships to form a network structure.
Fundamentally speaking, the knowledge map is essentially a knowledge representation method, which defines the domain ontology to carry out the knowledge structure ( concept, entity attribute, entity relationship, event attribute, and relationship between events ) of a certain business domain. Accurate representation, making it a canonical representation of knowledge in a specific domain. Subsequently, structured data is extracted from various data sources by means of entity recognition, relationship extraction, event extraction, etc., knowledge is filled, and finally stored in attribute graph or RDF format.
In fact, when targeting PTM ( not LLM) in the early years, it was said that PTM (pretrained language model) is the knowledge base , which includes a large number of tasks such as knowledge probing for analysis and understanding, and LLM (chatgpt) is parameterized knowledge . The advantage of KG is that it is convenient for debugging , understandable by humans, and has a strong ability to express graph structures.
But these two points can be combined, especially in terms of reasoning ( common sense and domain reasoning ), business system interaction, hyperautomation, access and update of time-sensitive content, etc. There are many examples of combination.
For example, the text generation mapping of various graph tasks, KG itself does more suitable symbols, including numerical calculations, including rule reasoning, etc., because this piece is actually relatively weak for LLM , or Learning efficiency is too low. Integrating knowledge graphs into ChatGPT can be achieved in many ways. Give it enough correct knowledge, and then introduce knowledge management and information injection technologies such as knowledge graphs, and also limit its data range and application scenarios to make the content it generates more reliable.
For example, we can represent the entities and relations in the knowledge graph as embedding vectors, which are incorporated into the model as additional features to improve the performance of the model. This method can integrate both the structural information and semantic information of the knowledge graph into the model, so that the model can better understand and generate natural language text.
In a conversation, a knowledge graph can help the model understand the context of the conversation and provide more accurate information for answering questions. In the LaMDA paper, knowledge graphs are used to provide contextual information for conversations. By combining information from knowledge graphs, questions can be automatically generated to help users better understand the semantics and context between entities and relationships.
Baidu has officially released the generative large language model "Wen Xin Yi Yan" a few days ago, and its underlying "Wen Xin large model" ( Ernie 3.0 ) combines knowledge graphs. Before Wenxin, most LLM large models used plain text data. For example, the GPT-3 corpus of 175 billion parameters has 570GB of filtered text from common crawls. These original texts lack the explicit expression of knowledge such as language knowledge and world knowledge. Furthermore, most large models are trained in an autoregressive manner, and such models exhibit poor performance with traditional fine-tuning when adapted to downstream language understanding tasks.
Theoretically speaking, the introduction of knowledge graph will greatly enhance Wenxin's performance in understanding problems and solving practical problems in downstream applications. Therefore, Wenxin 3.0 uses a 4TB corpus composed of plain text and a large-scale knowledge map as training data, and uses various types of pre-training tasks to enable the model to learn more effectively from valuable lexical, syntactic and semantic information. different levels of knowledge. Among them, the pre-training task propagates three task paradigms, namely natural language understanding, natural language generation and knowledge extraction. Wenxin 3.0 showed advantages over the previous large models in the few-shot and zero-shot tasks, making its various indicators surpass the SOTA model at that time, and won the first place in the Super GLUE benchmark test.
On the other hand, ChatGPT 's excellent performance on zero-shot/few-shot can actually be fed back to the entire process of knowledge graph construction, such as using ChatGPT for entity recognition, relationship extraction, and event extraction, which can be achieved to a certain extent To alleviate the high cost problem in the process of implementing the knowledge map.
However, the factual errors and timeliness problems encountered by ChatGPT also exist in knowledge graphs. The knowledge map also needs to solve the problem of knowledge update. Moreover, if the knowledge map cannot guarantee the correctness of unstructured data sources, factual errors are bound to occur later, which undoubtedly requires attention.
2. ChatGPT + Office Automation
In the office automation scenario, there are already a variety of phenomenon-level applications combined with ChatGPT , such as:
ChatPDF can first analyze the uploaded PDF and create a semantic index for each paragraph in the file. When the user asks a question, the tool will send the associated segment to ChatGPT , and then let it interpret it in conjunction with the question;
ResearchGPT , you can directly upload the PDF or link of the paper you want to read, and then you can display the original text of the paper, and you can ask questions directly on the right.
DocsGPT , a tool that simplifies the process of finding information in project documentation. By integrating powerful GPT models, developers can easily ask questions about projects and get accurate answers.
ChatExcel , this new application can directly use natural language to query, modify and other operations on the data information in the table, just like an assistant who is proficient in Excel .
However, we can clearly see that behind these "ChatGPT+ office automation " tools, there is actually a document standardization and normalization processing module supporting it, which can effectively process documents in current complex formats, such as word/pdf/doc /excel , etc. for normalization processing, scanning version of pdf , etc., and use this as input. Combining with ChatGPT can greatly improve its product performance and user experience.
Well, that's all for today. We expect that the industry will closely follow the technology of ChatGPT , and explore more possibilities by combining various related technologies and final application scenarios.