Thank you for reading the series of blogs "Building the Next Generation Knowledge Base of Enterprises Based on Intelligent Search and Large Models". The whole series is divided into 5 articles. It will systematically introduce new technologies such as how large language models can empower traditional knowledge base scenarios and help the industry Customer cost reduction and efficiency increase. The update directory is as follows:
The first article "Introduction to Typical Practical Scenarios and Core Components"
Part Three "Langchain Integration and Its Application in E-commerce" (this part)
Part Four "Manufacturing/Finance/Education and Other Industries Practical Scenarios" (coming soon)
The fifth "Integration with Amazon Kendra" (coming soon)
background
In this series of blogs "Building the Next Generation Knowledge Base of Enterprises Based on Intelligent Search and Large Models", the previous two articles have introduced "Introduction to Typical Practical Scenarios and Core Components" and "Guide to Quick Deployment by Hand". This article mainly introduces LangChain and Open source large language model integration, combined with Amazon cloud technology's cloud basic services, builds an intelligent search question answering solution based on the enterprise knowledge base.
Introduction to LangChain
LangChain is an open source framework that uses the ability of large language models to develop various downstream applications. Its core concept is to implement a common interface for various large language model applications and simplify the development difficulty of large language model applications. The main module schematic diagram is as follows:
Index: Provides various interfaces for document import, text splitting, and vectorized text storage and retrieval. In addition to common word, excel, PDF, and txt, the imported document types also include API and database interfaces. Vectorized storage includes various A vector-like database that also includes Amazon's OpenSearch search engine. Through the Index module, it is very easy to process various types of external data to provide large models for reasoning.
Prompts: Convert user input and other external data into prompt words suitable for large language models, including prompt word management, prompt word optimization, and prompt word serialization. By adjusting the prompt words, the large language model can be used to perform various tasks, such as text generation text (including chat, question and answer, summary, report, etc.), text generation SQL, text generation code, etc. In addition, the model can be used to perform few shot learning, currently the optimal practice of prompt words for each task is in full swing. Through the prompt words, the capability boundaries of large language models are continuously explored. LangChain provides an easy-to-use prompt word management tool.
Models: Provides management and integration of various large language models. In addition to closed-source large language model API interfaces, it also provides integration interfaces for open source large language models in multiple open source model warehouses, as well as large language models deployed on the cloud. Language model interface, such as the large language model interface deployed on Amazon SageMaker Endpoint.
Memory: It is used to save the context state when interacting with the model, and is a key component to realize multiple rounds of dialogue.
Chains: The core components of LangChain, which can combine the above components to complete specific tasks. For example, a chain can include prompt templates, large language models and output processing components to complete user chat functions. For various tasks, LangChain provides different chains. For example, the question answering task provides the Question Answering Chain, the text summary task provides the Summarization Chain, the text generation SQL task provides the SQL Chain, and the mathematical calculation task provides the LLM Math Chain. Chain can be customized, and the Chain component provides different model reasoning modes, including stuff, map_reduce, refine, map-rerank, etc. According to specific task requirements, an appropriate Chain and model reasoning mode can be selected to complete the task.
Agents: LangChain's advanced functions can decide which tools to call based on user input, and can combine a series of Chains to complete complex tasks, such as asking: How much gold can I buy with my account balance? Agents query the account balance through SQL chain, find the real-time gold price by calling the LLM of the web page query interface, and complete the final task by calling LLM Math to calculate the amount of gold that can be bought. This series of logical operations can be configured in Agents.
In addition to LangChain, another commonly used open source large language model application development framework is LIamaIndex. LIamaIndex has a rich data import interface, especially the support for structured data is more friendly. In addition, LIamaIndex's Index encapsulates the question-answering logic of multiple modes, which is easy to use but lacks flexibility. LIamaIndex supports integration with LangChain, and the two frameworks can call each other.
Based on intelligent search
Large Language Model Augmentation Solution Guide
Combining the various functional interfaces of LangChain and the basic services of Amazon Cloud Technology, we have constructed Amazon Cloud Technology's solution guide for large language model enhancement based on intelligent search. The architecture diagram is as follows:
Compared with the architecture diagram of the first article in this series of blogs "Typical Practical Scenarios and Introduction to Core Components", the main differences of this article are as follows:
In the Intelligent QA function, the Lambda code that implements the functional logic integrates the LangChain and LIamaIndex frameworks, using the open source framework to improve the flexibility and scalability of the solution, and the ability of LangChain can be used to quickly add new functions in the future.
Added the DynamoDB component, which is used to save the user's multi-round dialogue records and realize the multi-round dialogue function.
The workflow diagram of the scheme is as follows:
The workflow of the scheme and the integration with LangChain are as follows:
1. The data processing and data storage module integrates LangChain's Index interface, supports a variety of external data imports, summarizes various types of data, and forms an enterprise knowledge base:
For unstructured data, it mainly supports common documents such as word, excel, PDF, and txt. Through text extraction and text splitting of unstructured data, multiple text blocks are obtained, and text blocks are obtained by calling the vector model deployed in SageMaker Endpoint. The corresponding vectors, and finally store the text block and vectors into the OpenSearch search engine.
For structured data, the SQL Index integrated with LlamaIndex is used to read the database to obtain corresponding data. At present, the SQL language query database is mainly generated through rules, and the SQL language generated through the open source large language model is still in commissioning.
If you need to query the network to obtain real-time information, you can read the URL information through the Index interface, or query the real-time network information through the search engine interface.
2. The intent recognition module integrates LangChain's RouterChain interface, and automatically selects the appropriate data source to answer the user's question by making semantic judgment on the user's input question.
3. The Prompt engineering module integrates the Prompt interface of LangChain to manage and optimize Prompts with different tasks, scenarios and languages. At the same time, it integrates the Memory interface of LangChain, stores the user's questions and answers in DynamoDB, forms the History information of the user's question and answer, and serves as the reasoning basis for the large language model of the multi-round dialogue task.
4. The large language model module integrates the Model interface of LangChain, calls the open source large language model deployed in SageMaker Endpoint, and performs reasoning on various types of tasks.
5. The feedback and optimization iteration module, by recording the answers to the questions, analyzes the problems in the solution, and optimizes them in the knowledge base and Prompt project in time.
Compared with other LangChain-based knowledge base privatization deployment solutions, this solution uses cloud-native services of Amazon Cloud Technology, including SageMaker, OpenSearch and Lambda, etc. These cloud-native services can use the auto scaling mechanism according to the actual online traffic conditions, Quickly expand or contract resources to ensure the smooth operation of online business with the best cost performance.
You can easily debug the intelligent question and answer function through the debugging page of the solution:
enter:
query: user question text
Session ID: the session id requested by the user, the dialog information will be saved in DynamoDB with the session id as the key
Language: language selection, currently supports Chinese and English, the text splitting method, text vector model and open source large language model are different for different languages
Prompt: prompt word debugging, based on the default prompt word, you can test the impact of different prompt word texts on the results
Index: Index of OpenSearch stored data
Top_k: The number of relevant texts put into the large model reasoning. If the documents are more standardized, the documents and the query are easy to match, and Top_k can be reduced to increase the certainty of the answer
LLM Temperature: The temperature parameter of the large language model. The temperature parameter controls the degree of randomness of the large language model. The smaller the temperature, the higher the certainty of the answer
output:
Answer: The inference output of the large language model to the question
Confidence: Confidence of the answer
Source: the text related to Query and placed in the large model reasoning
Application of Intelligent Search-Based Large Language Model Enhancement Solution Guide in E-commerce Scenarios
Intelligent customer service
With the development of the e-commerce industry, consumers have higher and higher requirements for the service quality of online shopping, and customer service has become an indispensable part of e-commerce services. In order to save costs, merchants tend to use smart customer service to automatically answer simple questions, and manually answer questions that cannot be solved by smart customer service. However, the intelligent customer service systems of major e-commerce platforms often give preset answers through keywords, and cannot fully understand the questions raised by customers. There is a gap between the answers and the needs of customers, and the user experience is not friendly. The semantic understanding and inductive reasoning capabilities of the large language model are very suitable for intelligent customer service scenarios. The current application of the solution in intelligent customer service scenarios includes:
1. Intelligent Q&A based on enterprise knowledge base. Use enterprise knowledge base data, combined with the reasoning ability of large language models, to achieve accurate question and answer.
In the following example, after importing the e-commerce return and exchange sample document, the customer asks the answer to the return question:
As can be seen from the above example, the large language model understands that the pants purchased by the customer belong to the clothing category, and there is a corresponding 7-day no-reason return policy, but further acquisition of the purchase date is required to confirm whether the return can be made, and finally a further acquisition of customer purchases is generated. dates and answers explained about the 7-day no questions asked return policy.
2. Multiple rounds of chat. The solution will save the information about the interaction between the user and the big language model, and embed prompt prompt words as context information in the next conversation, so that the big language model can know the context information to answer subsequent questions.
3. Text summary extraction and text report generation. By inputting the telephone call records in the customer service, quickly generate a call summary, and accurately extract the demand points mentioned by the user in the call records, so that the subsequent analysis of the call records and customer follow-up marketing. The following is a case of a customer consulting a product purchase issue:
Intelligent Recommendation
At present, the e-commerce industry is gradually entering the era of content e-commerce, and live broadcast delivery accounts for an increasing proportion of e-commerce transactions. One of the characteristics of live delivery delivery is that the interaction between the anchor and the audience is very frequent, and the audience usually asks questions about the live broadcast scene. The intelligent question-and-answer system can be used as a shopping guide to assist the anchor to answer questions. For example, in a live broadcast room where the anchor is promoting sneakers, the audience may ask: "What series of sneakers are suitable for playing football in rainy days?", "Which color look better for fat people?" , "Which pair of sneakers are available for couples?", the intelligent question-and-answer system can combine product description information, product evaluation information, and user purchase record information to directly output product recommendations or product usage suggestions for user questions, improving the interaction between anchors and audiences s efficiency.
The product sample data used by the intelligent question answering system to generate answers:
In the above example, after providing information about the appropriate use scenarios of different series of sneakers, the intelligent question answering system can recommend suitable products to users according to the questions raised by customers.
Frequently encountered problems and solutions
At present, the intelligent question and answer function based on the enterprise knowledge base is the most demanded and implemented function. When the solution is implemented in different scenarios, it often needs to be tuned according to the specific scenario requirements. Common problems and solutions encountered during implementation include:
1. How to find exactly the text relevant to the question in the knowledge base?
We mainly use enterprise rule documents and previous customer service question-and-answer data as the knowledge base. The documents are stored in docs format, and sentences and paragraphs are used to split the data. However, the main problem is that multiple sentences or paragraphs form a logical paragraph, and the size of the logical paragraph Not one, if split by sentence or paragraph, it is possible to separate the logical paragraph, for example, the question and answer may be split into two text blocks for storage, or part of the previous logical paragraph may be added to the next logical paragraph Some of them are stored, so that the corresponding answers cannot be found correctly according to the questions during retrieval. The schematic diagram of error splitting is as follows:
solution:
Convert the docs document processing to csv format storage, put the content of each logical segment into one line, and manually separate the logical segments.
The split text block is divided into multiple vectorized texts by sentences or paragraphs for vectorization, the entire logical block text is used as the retrieval text, and the vector text, vector and retrieval text are stored as a piece of text information.
When searching, the text vector is used to match the question vector, and the entire logic block text is used as the retrieval text to be sent to the large model for reasoning.
Through measures such as manual separation of logical blocks, fine-grained text vectors, and coarse-grained text recall, the accuracy of relevant text recall has been greatly improved.
2. How to evaluate whether the answer output by the large language model is fabricated?
To evaluate whether the answer is reliable by calculating the confidence of the answer, the calculated dimensions include the following three aspects:
Calculate the similarity between the question and the answer. Usually, the similarity between the answer and the question generated by the large language model is relatively high. Occasionally, the similarity is relatively low. In this case, the answer is likely to be irrelevant to the question, and occasionally the similarity is very In the case of high, the answer text is basically a repetition of the question text, and the answer in this case cannot be used.
Computes the similarity of the answer text to the related text used for inference by the large model. If the similarity between the answer text and the relevant text used by the large model inference is low, the answer text is likely to be fabricated by the model.
Calculate the coincidence rate of the relevant text list recalled by the answer text in the database and the relevant text list recalled by the question text in the database. If the coincidence rate of the 2 lists is low, the confidence of the answer will also be low.
Reasonably adjust the thresholds of the three scores according to different scenarios to judge whether the answer is convincing.
Summarize
At present, in addition to the implementation of the e-commerce industry introduced above, the LangChain-integrated intelligent search-based large language model enhancement solution guide has landing cases in manufacturing, finance, education, and medical industries. At the same time, the solution has been integrated with Amazon Kendra services , please pay attention to our upcoming follow-up blog for details.
The author of this article
He Bo
Amazon cloud technology industry solution architect, worked in Ali for six years, responsible for the development of recommendation algorithms, search algorithms and multi-modal matching algorithms, and has extensive experience in the application of machine learning models in various scenarios in the e-commerce industry.
Ji Junxiang
Senior Solution Architect of Amazon Cloud Technology, responsible for the end-to-end design and implementation of innovative scenarios in the rapid prototyping team.
Tian Bing
Amazon cloud technology application scientist, has long been engaged in the research and development of natural language processing, computer vision and other fields. Supports data lab projects, and has rich algorithm and implementation experience in large language models, object detection, etc.
Zheng Hao
Amazon GCR AI/ML Specialist SA. Mainly focus on Language Model training and reasoning, search and inference algorithms and related optimization and program construction based on the Amzon AI/ML technology stack. In Ali, Ping An has many years of experience in algorithm research and development.
I heard, click the 4 buttons below
You will not encounter bugs!
