Knowledge base requirements are ubiquitous in all walks of life, such as the historical fault knowledge base in manufacturing, the content knowledge base of game community platforms, the product recommendation knowledge base of e-commerce, and the registration recommendation knowledge base system in the medical and health field, etc. In order to ensure the timeliness and accuracy of the recommendation system, Amazon cloud technology requires a large amount of data/algorithm/software engineers' human input and material input including hardware. Secondly, in order to further improve search accuracy, how to guide users to search descriptions more accurately and make full use of user behavior to optimize search engines are also common user pain points. In addition, how to directly give answers to users' questions based on the enterprise knowledge base is also a technical bottleneck that many enterprises will encounter.
There are many scenarios in various industries
Requires search and Q&A based on enterprise knowledge base
#1. Build an equipment maintenance knowledge base and question answering system: use historical maintenance records and maintenance manuals to build an enterprise knowledge base, and maintenance personnel can rely on this knowledge base to quickly locate and repair problems.
#2. Build an IT/HR system intelligent question answering system: use the internal IT/HR manual to build an enterprise knowledge base, through which internal employees can quickly solve problems encountered in IT/HR.
#3. Build a search and question-and-answer system for an e-commerce platform: use product information to build a product database, and consumers can quickly learn about product details through search + question-and-answer.
#4. Build a game community automatic question answering system: Use game information (such as game introduction, game strategy, etc.) to build a community knowledge base, which can automatically answer questions provided by community members based on the knowledge base.
#5. Build an intelligent customer chat robot system: By combining with the call center/chat robot service, it can automatically respond to questions raised by customers based on the enterprise knowledge base.
#6. Build an intelligent education guidance system: use textbooks and question banks to build knowledge bases at different stages of education, and simulate and assist teachers/parents to teach children.
In order to solve the above scenario requirements, Amazon cloud technology can be used to combine search and large language models. First of all, a knowledge base can be established by using the data assets accumulated by the enterprise itself. Secondly, for specific question answering tasks, the search function can be used to effectively recall the knowledge base, and then the recalled knowledge can be used to enhance the large language model. With this approach, the solution to the question answering task can be achieved.
In terms of enterprise knowledge base establishment and search services, Amazon Cloud Technology has Amazon OpenSearch, a cloud-hosted search service, and Amazon Kendra, an AI/ML-based intelligent enterprise search service. Although the above-mentioned services can provide basic search engines and frameworks, and solve the pain points of users' large investment in hardware and difficult management, the above-mentioned services cannot meet the needs of document-based Q&A. In order to solve the gap between user needs and services, with the help of Amazon cloud technology services, a large language model enhancement solution based on intelligent search was constructed. This solution builds a search engine based on Amazon OpenSearch/Amazon Kendra, combined with a large language model hosted on Amazon SageMaker, to provide a one-stop intelligent knowledge base search question answering platform.
Based on intelligent search
Introduction to the Large Language Model Enhancement Solution
Five Core Contents
#1. Smart Search
The traditional word segmentation search method that only relies on keyword matching can provide fast and effective queries in many scenarios, but it also has some inherent limitations. Such as matching some irrelevant words including stop words, inability to recognize synonyms and lack of abstraction ability. In order to solve these problems, the intent recognition model can be used in this solution to extract key information, so that it can effectively avoid the interference caused by stop words and other impossible words on the search, and introduce AI/ML methods to assist in the realization of semantic search. Specifically, the same vector encoding model is used to encode the semantics of the search sentence and the document database, and the knn method is used for vector matching during the retrieval process. In this solution, a search engine is built based on Amazon OpenSearch and Amazon Kendra. Provide word segmentation search, fuzzy query and AI/ML assisted search functions. It is no longer limited to a certain search method, but integrates all search methods to complement each other organically.
#2. Smart Guide
The reasons for the inaccurate search are, on the one hand, the lack of capabilities of the search engine itself, and on the other hand, the lack of accurate and specific search terms. Therefore, this solution proposes a guided search mechanism to help search personnel gradually enrich the input search sentences, and finally achieve the purpose of improving search accuracy.
Take the following search process of the large-scale equipment maintenance knowledge base in the manufacturing industry as an example. The knowledge base stores historical maintenance records, including fields such as failure symptoms, failure reasons, and maintenance solutions.
- When the user enters the search term "circuit", in addition to returning entries related to the circuit from the knowledge base, some prompt words will be given, such as "door system", "control system", etc. Associated failures are often accompanied by potentially problematic systems, prompting the user to further enrich the current search description.
- When the user further inputs "mainboard", the joint query of "circuit" and "mainboard" will be performed, relevant entries will be returned, and new prompt words will be given further.
- Users can repeat the above process until more accurate results are found.
Acquisition of prompt words: According to the actual situation, methods such as manual marking, unsupervised clustering, supervised classification, and large language model (LLM) can be used to extract and inject them into the database in advance.
#3. Smart optimization
Usually, due to the iterative update of the knowledge base, the accuracy of retrieval may gradually decrease over time. On the one hand, it is often impossible to guarantee that the database and search engine will achieve good results after one-time construction. On the other hand, it is because outdated knowledge has not been effectively dealt with. Therefore, this program proposes to continuously optimize the search engine based on user behavior.
Specifically, it includes two steps:
- User behavior collection: collect historical user behavior, such as the user's score for a search term.
- Model training and deployment: Through user behavior, sort out the correlation between search terms and knowledge base. Use this correlation to train and deploy a rearrangement model, which can give users more preferred content a higher weight score based on historical user behavior.
It is worth noting that this model is based on the traditional machine learning model xgboost, so the amount of training data required and the resources required for reasoning are very small (for example, only dozens of data and t3.small models are required), Therefore, different rearrangement models can be trained based on different users/user groups to achieve the purpose of personalized search for thousands of people.
#4. Smart Q&A
Question answering based on private knowledge base is another widely used scenario, such as intelligent customer chat robot system, IT/HR system intelligent question answering system, etc.
If only a search engine is used, the content related to the question can only be extracted from the database based on the question, and the answer cannot be given directly.
If only a large language model (Large Language Model, LLM) is used, question answering based on a private knowledge base cannot be performed. A feasible way is to directly send the private knowledge base and questions to LLM in the form of a prompt, and then let LLM give an answer. However, due to the limitation of LLM Token, it is impossible to input too many knowledge bases at one time.
Therefore, in this scenario, the two are combined. When the user asks a question, first use the search to extract the knowledge related to the question, then give the question and the extracted knowledge to LLM for summary, and finally give the answer to the question directly.
#5. Unstructured Data Injection
The enterprise knowledge base that can be retrieved by search engines is a kind of structured data, but often the original knowledge of enterprises is stored in unstructured data, which comes from multiple channels and includes multiple formats, such as Words , PDF, Excel, etc.
In order to help enterprises quickly utilize these structured data, this solution provides an unstructured data injection function, which automatically splits paragraphs and vector codes the enterprise's knowledge documents to establish a structured enterprise knowledge base.
Model technical details
#1.LLM
In the past six months, the large language model (LLM) has achieved rapid development in the field of natural language processing. Large language models are usually based on the Transformer architecture, trained on large-scale web text data, and their core is to use a self-supervised goal to predict the next word in a partial sentence. Amazon Cloud Technology has launched the large language model Titan and the large language model platform Amazon Bedrock. In addition, many research institutions have launched open source large language models, such as Alpaca of Stanford University and ChatGLM of Tsinghua University. These large language models have powerful text processing capabilities and are widely used in scenarios such as intelligent question answering, text summarization, and text generation.
#2.Embedding
Various types of unstructured data widely exist in daily life and work scenes, such as text, pictures, videos, etc. In order to process these unstructured data, the Embedding model is usually used to extract the characteristics of these data, and convert the data characteristics into vectors, through Feature vectors are used for analysis and retrieval of these unstructured data. The general pre-trained language model has the function of vectorizing text, and you can choose a suitable pre-trained model as the embedding model according to different scenarios and languages.
#3.Intent Detection
The main function of search intent recognition is to analyze the core search needs of users. For example, in the e-commerce scene, the electronic product that the user is looking for is a computer or a mobile phone, whether it is for a home scene or an outdoor scene, etc. If If intent recognition is not accurate, many irrelevant products will be displayed to users, resulting in a very poor user experience. Therefore, accurate intent recognition is very important. Intention recognition mainly includes category prediction and entity recognition models. The category prediction model mainly uses a text multi-category model. According to the user behavior data of the platform, the probability that the query text belongs to each category is predicted. The entity recognition model recognizes the entity words in the query text. Entity words are dimensional information describing the product, such as brand, color, material, etc. After identifying the entity words of the query text through the entity recognition model, the search engine is used for precise query .
#4.Controlled Text Generation
Controllable text generation is based on traditional text generation, adding control over the generated text, such as specifying the keywords, format, style, etc. of the generated text, so that the generated text meets our expectations, such as generating the same style as someone texts, generate reports with fixed content formats, generate complete novels based on simple story lines, and more. Controllable text generation includes methods such as finetune the pre-trained model, retrain the text generation model, and reconstruct the output results of the pre-trained model. After the large language model is launched, prompts can now be used to guide the large language model to generate controllable text. According to different scenarios and text generation goals, prompts with different formats and contents can be designed to generate text that meets the needs.