A quick guide for you to build a chatbot website that can accept external documentation as context.
As information and knowledge emerge on my screen every day, we are challenged by the natural limitations of human reading and memory, making it increasingly difficult to keep up with the information. Now, large-scale language models (LLMs) like ChatGPT and Llama offer a potential solution to store and process large amounts of information, requiring only simple prompts to quickly and directly obtain responses. However, the performance of LLMs depends on the quality of the data they are trained on, and although most of their datasets are already very large, these data are still "yesterday's" data.
I am often troubled because I have difficulty reading user manuals or drug leaflets, which contain a lot of unfamiliar information, or finding specific data in technical papers or reports, even if a new book is very attractive to me. Read impatiently. Currently, ChatGPT or Bard can't directly help me get the information I need.
After creating multiple chatbots based on GPT APIs and other libraries, such as private chat, voice chat, and image chat, now I'm thinking about building a document-based chatbot that can learn new knowledge from various data sources, And provide accurate and helpful responses to my inquiries at an acceptable cost.
To achieve this goal, there are many methods that can be leveraged, one of which is contextual learning and the other is fine-tuning. Since the fields of study are so diverse, contextual learning techniques are my best bet. After some research, I decided to use LlamaIndex as the basis for my document chatbot for contextual learning, and continued to use OpenAI's completion and embedding capabilities for the demo.
In this article, I'll walk you through the development process, making sure you understand and are able to replicate your own documentation chatbot.
1. Block Diagram (system block diagram)
In this simple chatbot, we use llama-index as a base and develop a Streamlit web application that provides a display of user input and document query interactions. Using the llamaIndex toolkit, we don't have to worry about API calls in OpenAI, as its internal data structures and LLM task management easily remove the complexity of embedding usage or issues with hint size limitations.
2, OpenAI API secret
LlamaIndex is designed to be compatible with various LLMs, and by default, it uses OpenAI's text-davinci-003 model and text-embedding-ada-002-v2 for embedding operations. Therefore, when we decide to implement a document chatbot based on the OpenAI GPT model, we should provide our OpenAI API key to the program. The only thing we need to do to plug in our key is to provide it via an environment variable:
import os
os.environ["OPENAI_API_KEY"] = '{my-openai_key}'3、LlamaIndex
LlamaIndex is a Python library that provides a central interface between user-private data and large language models. It provides the following functions:
Data Connectors: LlamaIndex can connect to a variety of data sources, including APIs, PDFs, documents, and SQL databases. This allows you to use existing data with LLMs without implementing additional code. This is actually the key reason why I chose it as an app.
Hint limit: LlamaIndex can handle hint limits like Davinci's 4096 marker limit. This ensures that your LLMs can still generate accurate results even when the context is large. This saves developers a lot of time managing token calculations and splits.
Indexing: LlamaIndex creates indexes on raw language data, making LLMs faster and easier to access.
Hint Insertion: LlamaIndex provides a way to insert hints into the data. This allows you to manage the interaction of LLMs with your data.
Text Segmentation: LlamaIndex can split text into smaller chunks, which can improve the performance of LLMs.
Query: LlamaIndex provides an interface to query the index. This allows you to get knowledge-enhanced output from LLMs.
LlamaIndex provides a comprehensive toolset for working with LLMs. This includes data connectors, indexes, prompt inserts, prompt limits, text splits, and queries. You can find complete documentation here.
Instructions
Here are the general steps to use LlamaIndex:
Install the LlamaIndex package.
!pip install llama-indexA) If your application just needs to load some document files at a time and query their contents, the following lines of Python code should be enough.
Step 1 - Load the documentation file.
from llama_index import SimpleDirectoryReader
SimpleDirectoryReader = download_loader("SimpleDirectoryReader")
loader = SimpleDirectoryReader('./data', recursive=True, exclude_hidden=True)
documents = loader.load_data()SimpleDirectoryReader is one of the file loaders in the LlamaIndex toolset. It supports loading multiple files under a user-supplied folder, in this case the subfolder './data/'. This magic loader function can support parsing various file types such as .pdf, .jpg, .png, .docx, etc., so you don't have to convert the file to text yourself.
Step 2 - Build the index.
from llama_index import LLMPredictor, GPTSimpleVectorIndex, PromptHelper, ServiceContext
from langchain import OpenAI
...
# define LLM
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-davinci-003"))
max_input_size = 4096
num_output = 256
max_chunk_overlap = 20
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
index = GPTSimpleVectorIndex.from_documents(
documents, service_context=service_context
)When calling this method, LlamaIndex will interact with the LLM you defined to build the index. In this demo, LlamaIndex calls the embedding method through the OpenAI API.
Step 3 - Query Index
With an index, querying is as simple as typing a query without contextual data.
response = index.query("How to use Keytruda in cervical cancer cases?")
print(response)Step1.ex — load various data resources
In addition to the SimpleDirectoryReader method for reading local document files, there are many open source readers listed on llamahub.ai that can be loaded via the download_loader function. They are very simple to use. For example, if I wanted to load the entire page from Wikipedia for the second book in the Three-Body Problem series, The Dark Forest, the code would look like this:
from llama_index import download_loader
WikipediaReader = download_loader("WikipediaReader")
loader = WikipediaReader()
documents = loader.load_data(pages=['The_Dark_Forest'])In the process of index building, due to the need to use LLM to process a large amount of data, after the index is built, it should be stored on the local disk, so that if future queries are still based on the same document, it is most economical to load the index from disk and time-efficient approach.
Step2.ex — save and load the generated index
In the process of index building, due to the need to use LLM to process a large amount of data, after the index is built, it should be stored on the local disk, so that if future queries are still based on the same document, it is most economical to load the index from disk and time-efficient approach.
# save to disk
index.save_to_disk('index.json')
# load from disk
index = GPTSimpleVectorIndex.load_from_disk('index.json')Step3.ex — Set response mode
When calling the index.query() method, the useful parameter response_mode can be used to specify the response style to take into account the prompt cost.
default (default) — suitable for detailed answers
compact (compact) — suitable for cost-sensitive situations
tree-summarize — good for summarizing answers
response = index.query("...", response_mode="default")4. Web application development
As a continuation of the project in my previous article, we'll continue building a document chatbot application using the handy Streamlit library.
Streamlit is an open source Python library that makes it easy to create interactive web applications. It is intended to be used by data scientists and machine learning engineers to share their work with others. Streamlit applications can be created with just a few lines of code and deployed to the web with a single command.
It provides a variety of widgets that can be used to create interactive applications. These widgets include buttons, text boxes, sliders, and charts. You can find the usage of all widgets in its official documentation:
https://docs.streamlit.io/library/api-reference
A typical Streamlit code for a web application can be as simple as the following:
!pip install streamlit
import streamlit as st
st.write("""
# My First App
Hello *world!*
""")Then just enter the following command to run the website online:
!python -m streamlit run demo.pyIf run successfully, it will print out the URLs the user can visit.
You can now view your Streamlit app in your browser.
Network URL: http://xxx.xxx.xxx.xxx:8501
External URL: http://xxx.xxx.xxx.xxx:8501The implementation of the entire Doc ChatBot includes the following functions:
Create a file upload widget via the Streamlit st.file_uploader() method to accept user-uploaded document files.
After uploading successfully, save the file in the ./data/ folder, and use LlamaIndex to generate a document object, and then call the st.sidebar() method to print the converted content in the left area for user reference.
Call the llamaIndex method to generate an index object for the document files in the ./data/ folder and store it in the website root directory ./.
If an index file (index.json) exists in the ./site root directory, a pair of Streamlit widgets st.text_input() and st.button() will be activated, allowing the user to query the document currently being loaded.
Every time the website is opened without uploading a new document, the existing index file will be loaded, waiting for the user's query.
Here is the whole demo code for your reference:
import os
os.environ["OPENAI_API_KEY"] = '{my_api_key}'
import streamlit as st
from llama_index import download_loader
from llama_index.node_parser import SimpleNodeParser
from llama_index import GPTSimpleVectorIndex
from llama_index import LLMPredictor, GPTSimpleVectorIndex, PromptHelper, ServiceContext
from langchain import OpenAI
doc_path = './data/'
index_file = 'index.json'
if 'response' not in st.session_state:
st.session_state.response = ''
def send_click():
st.session_state.response = index.query(st.session_state.prompt)
index = None
st.title("Yeyu's Doc Chatbot")
sidebar_placeholder = st.sidebar.container()
uploaded_file = st.file_uploader("Choose a file")
if uploaded_file is not None:
doc_files = os.listdir(doc_path)
for doc_file in doc_files:
os.remove(doc_path + doc_file)
bytes_data = uploaded_file.read()
with open(f"{doc_path}{uploaded_file.name}", 'wb') as f:
f.write(bytes_data)
SimpleDirectoryReader = download_loader("SimpleDirectoryReader")
loader = SimpleDirectoryReader(doc_path, recursive=True, exclude_hidden=True)
documents = loader.load_data()
sidebar_placeholder.header('Current Processing Document:')
sidebar_placeholder.subheader(uploaded_file.name)
sidebar_placeholder.write(documents[0].get_text()[:10000]+'...')
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-davinci-003"))
max_input_size = 4096
num_output = 256
max_chunk_overlap = 20
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
index = GPTSimpleVectorIndex.from_documents(
documents, service_context=service_context
)
index.save_to_disk(index_file)
elif os.path.exists(index_file):
index = GPTSimpleVectorIndex.load_from_disk(index_file)
SimpleDirectoryReader = download_loader("SimpleDirectoryReader")
loader = SimpleDirectoryReader(doc_path, recursive=True, exclude_hidden=True)
documents = loader.load_data()
doc_filename = os.listdir(doc_path)[0]
sidebar_placeholder.header('Current Processing Document:')
sidebar_placeholder.subheader(doc_filename)
sidebar_placeholder.write(documents[0].get_text()[:10000]+'...')
if index != None:
st.text_input("Ask something: ", key='prompt')
st.button("Send", on_click=send_click)
if st.session_state.response:
st.subheader("Response: ")
st.success(st.session_state.response, icon= "")Finish
After finishing development, I did some quick experiments.
I uploaded the wikipedia page of the book "Dark Forest" and asked a question that usually takes me to read several times to answer, to my surprise this chatbot perfectly answered my question by combining several plots question.
I then asked ChatBot to read a medical leaflet for Keytruda, a humanized antibody drug for cancer, and asked about its use in the treatment of a certain type of cancer. Answers seem to be very comprehensive, covering different aspects of information. Such learning support can definitely save a lot of time and provide more accurate information in cases where domain knowledge is difficult to understand.
I hope this article is helpful to you, thanks for reading! By using LlamaIndex and Streamlit, we successfully created a document chatbot application. It can read and index a variety of data sources, and then provide useful answers to users' queries. This application is not only easy to use, but also can greatly improve work efficiency and become my personal assistant.
At the end of the article, I would like to remind you that it is not easy to create an article. If you like my sharing, please don’t forget to like and forward it so that more people in need can see it. At the same time, if you want to gain more knowledge of front-end technology, welcome to follow me, your support will be the biggest motivation for me to share. I will continue to output more content, so stay tuned.
Original:
https://levelup.gitconnected.com/how-to-create-a-doc-chatbot-that-learns-everything-for-you-in-15-minutes-364fef481307Author: Yeyu Huang
Indirect translation, some self-adapted and added parts, the translation level is limited, it is inevitable that there are omissions, welcome to correct