Category: General Catalog of "Natural Language Processing from Entry to Application"
This paper introduces natural language processing application tasks such as information extraction, sentiment analysis, question answering system, machine translation and dialogue system. These tasks can directly or indirectly provide services to end users in the form of products, and are the main technologies for the application of natural language processing research.
information extraction
Information Extraction (Information Extraction, IE) is the process of automatically extracting structured information from unstructured text, which is convenient for computers to carry out subsequent processing. In addition, the extracted results can also be added to the knowledge base as new knowledge. Information extraction generally includes the following subtasks.
- Named Entity Recognition (NER): Extract each mentioned named entity in the text and mark its type, generally including person names, place names, and institution names, as well as proper names, such as book titles, movie names, and drug name etc. After finding the named entities mentioned in the text, it is often necessary to link these named entities to specific entities in the knowledge base or knowledge graph. This process is called Entity Linking. For example, "Washington" can refer to both the first president of the United States and the capital of the United States, which needs to be judged according to the context. This process is similar to the word sense disambiguation task.
- Relation Extraction: It is used to identify and classify the semantic relationship between entities mentioned in the text, such as binary relationships such as husband and wife, children, work units, and geographic location relationships.
- Event Extraction: Identify events that people are interested in from text and key elements such as time, place, and people involved in the event. Among them, events are often defined by specific trigger words (Trigger) mentioned in the text. It can be seen that event extraction is similar to the task of semantic role labeling, where trigger words correspond to predicates in semantic role labeling, and event elements can be considered as arguments in semantic role labeling.
The time of occurrence of an event is often critical, so temporal expression (Temporal Expression) recognition is also considered an important information extraction subtask, generally including two types of time: absolute time (date, week, month and holidays, etc.) and relative Time (e.g. tomorrow, two years ago, etc.). Use Temporal Expression Normalization to map these temporal expressions to specific dates or times of day. The following uses an example to comprehensively display the above information extraction subtasks. As reported by the following news:
On October 28, AMD announced the acquisition of FPGA chip giant Xilinx for $35 billion. The two chip companies, which have been rumored for years, have finally come together.
The information extraction results are as follows:
emotion analysis
Sentiment is an important psychological cognitive ability of human beings. Using computers to automatically perceive and process human emotions has become one of the important research contents in the field of artificial intelligence. Sentiment analysis in natural language processing mainly studies the emotions expressed by human beings through words, so it is also called text sentiment analysis. However, emotion is a relatively general concept, which not only includes an individual's attitude, viewpoint or tendency towards external things, such as positive and negative, but also refers to the person's own emotions (Emotion), such as joy, anger, sadness and Fear and so on. With the rapid development of the Internet, a variety of User Generated Content (UGC) has emerged, many of which contain people's emotions such as joy, anger, sorrow, and fear. Accurate analysis of these emotions is helpful to understand people's For a product's preference, keep abreast of the development of public opinion. Therefore, sentiment analysis has become one of the main applications of natural language processing technology.
Sentiment analysis can be divided into two main subtasks from the perspective of tasks, namely, sentiment classification (identifying the type or intensity of emotion contained in a text, where the text can be either a sentence or a chapter) and sentiment information extraction (extracting text emotional elements, such as evaluation words, evaluation objects and evaluation collocations, etc.). For the following user reviews:
这款手机的屏幕很不错，性能也还可以。
The results of sentiment analysis are as follows:
Since sentiment analysis has many application scenarios, such as the analysis of product reviews and public opinion analysis, sentiment analysis has attracted extensive attention from the industry and has become an important manifestation of the application of natural language processing research. In addition, sentiment analysis has also shown important research significance and broad application prospects in the fields of sociology, economics, and management. These demands continue to put forward higher requirements for sentiment analysis, and promote the connotation and extension of sentiment analysis research. Extend and go deep.
question answering system
Question Answering (QA) refers to a natural language processing system that accepts questions described by users in natural language and obtains answers from heterogeneous data through technologies such as retrieval, matching, and reasoning. Depending on the source of the data, question answering systems can be divided into 4 main types:
- Retrieval question answering system, the answers come from a fixed text corpus or the Internet, and the system completes the question answering by searching for relevant documents and extracting answers
- Knowledge base question answering system, the knowledge required to answer questions is stored in a structured form such as a database. The question answering system first parses the question into a structured query statement, and obtains the answer by querying relevant knowledge points and combining knowledge reasoning
- Frequently Asked Questions question answering system, through the retrieval of historically accumulated FAQs, to answer similar questions raised by users
- A reading comprehension question answering system that answers questions posed by users by extracting text fragments from a given document or generating an answer. In practical applications, the above multiple types of question answering systems can be used comprehensively to better answer questions raised by users.
machine translation
Machine translation (Machine Translation, MT) refers to the use of computers to automatically translate from one natural language (source language) to another natural language (target language). According to statistics, there are currently about 7,000 languages in the world, of which more than 300 languages have more than 1 million users. With the development of globalization and the widespread popularization of the Internet, the exchange of information between users of different languages has become more and more important. How to break through the language barrier between different countries and different ethnic groups has become a common problem faced by all mankind. Machine translation provides an effective technical means to overcome this problem. Its goal is to establish automatic translation methods, models and systems, break down language barriers, and finally realize automatic translation between any time, any place, and any language, and complete barrier-free translation for people. The dream of free communication. Since the birth of the field of natural language processing, machine translation has been its main research task and application scenario. In recent years, companies such as Google and Baidu have launched online machine translation services one after another. Companies such as HKUST Xunfei have also launched translation machine products, which can directly translate speech in one language into speech in another language, providing services for people with different languages. It facilitates communication between people.
The following is an example of Chinese-English translation, in which both the source language (Chinese) and the target language (English) have been word-segmented:
S: Beijing is the capital of China.
T: Beijing is the capital of China.
Machine translation methods generally use sentences as the basic input unit to study the mapping function from source language sentences to target language sentences. Since the birth of machine translation, researches have mainly been carried out around two methods: rationalism and empiricism. The so-called "rationalism" refers to a rule-based method; while "empiricalism" refers to a data-driven statistical method, which is manifested as a research method based on a corpus (translation instance database) in the field of machine translation. The machine translation method based on deep learning that has emerged in recent years uses deep neural networks to learn the implicit translation rules from source language sentences to target language sentences, that is, all translation rules are encoded in the model parameters of the neural network. This method is also known as Neural Machine Translation (NMT).
dialogue system
Dialogue System (Dialogue System) refers to an intelligent system that uses natural language as the carrier, and the user and the computer achieve specific goals through multiple rounds of interaction. Among them, specific goals include: completing specific tasks, obtaining information or recommendations, obtaining emotional comfort and social companionship, etc. In the 1950s, Turing proposed the "Turing Test" for evaluating the intelligence level of computer systems, which was conducted in the form of natural language dialogue. The dialogue system can be directly applied to many scenarios such as voice assistants, smart speakers and car voice systems. Dialogue systems are mainly divided into Task-Oriented Dialogue and Open-Domain Dialogue. The former is a task-oriented dialogue system, mainly used for automatic business assistants in vertical fields, etc., with clear task objectives, such as completing specific tasks such as air ticket reservations and weather inquiries. The latter is a dialogue system aimed at social interaction, usually aimed at chatting, emotional escort, etc., so it is also called a chat system or a chat robot (Chatbot), which is highly open in terms of domain and topic.
The following is an example of a human-computer dialogue in an open domain dialogue system, where U represents the user's utterance (Utter-ance), and S represents the reply of the dialogue system. The main goal of this type of dialogue system is to improve the turn of dialogue and user satisfaction. Compared with the accuracy of the dialogue, the open domain dialogue system pays more attention to the diversity of dialogue and the degree of attraction to users.
U: The weather is really nice today!
S: Yes, very suitable for outdoor sports.
U: What kind of sports do you like?
S: I like playing football, how about you?
A task-based dialogue system generally consists of three modules executed sequentially, namely natural language understanding, dialogue management and natural language generation. Among them, the main function of the Natural Language Understanding (NLU) module is to analyze the semantics of the user's utterance, and the usual representation is the domain, intention and corresponding slot value of the utterance. As for user utterance:
U: Help me book a ticket to Beijing tomorrow
The results of natural language understanding are as follows:
The Dialogue Management (DM) module includes two sub-modules: Dialogue State Tracking (DST) and Dialogue Policy Optimization (DPO). A dialog state is generally represented as a list of semantic slots and values. For example, the current dialogue state (usually a list of semantic slots and their corresponding values) is obtained by tracking the dialogue state of the results of natural language understanding of the above user utterances:
[Arrival place=Beijing; Departure time=Tomorrow; Departure place=NULL; Quantity=1].
After obtaining the current dialogue state, carry out strategy optimization, that is, choose what kind of strategy to use next, which is also called an action. There are many actions, such as the place of departure, the type of cabin, etc. can be asked.
In task-based dialogue systems, the work of the Natural Language Generation (NLG) module is relatively simple, usually by writing templates. For example, if you want to ask about the place of departure, you can directly ask "Where are you starting from?", and then give feedback to the user through speech synthesis (Text-to-Speech, TTS). The above three modules can be executed in a loop forever, and the dialogue state will also change with each user's utterance. Then, adopt different reply strategies until the user's booking needs are met.
References:
[1] Che Wanxiang, Cui Yiming, Guo Jiang. Natural language processing: a method based on pre-training model [M]. Electronic Industry Press, 2021. [2] Shao Hao, Liu Yifeng. Pre-training
language model [M] ]. Electronic Industry Press, 2021.
[3] He Han. Introduction to Natural Language Processing [M]. People's Posts and Telecommunications Press, 2019 [
4] Sudharsan Ravichandiran. BERT Basic Tutorial: Transformer Large Model Combat [M]. People's Posts and Telecommunications Publishing Society, 2023
[5] Wu Maogui, Wang Hongxing. Simple Embedding: Principle Analysis and Application Practice [M]. Machinery Industry Press, 2021.