[Tencent Cloud Lab] Integrating AI technology with vector database: building a next-generation intelligent customer service platform

Preface

Why is it said that a vector database is indispensable for good use of large models?

Before answering this question, let us first understand what is a vector?

For example, when we see this fruit, we know it is an apple. But before the word "apple" was invented, how did people describe the fruit? By observing its characteristics such as color, size, shape, texture, etc., we can define the fruit more clearly. Expressing these features numerically, we can get a vector.

Once the complex image is converted into a computer-readable digital representation, the computer can "know" the apple. But when a new Apple comes out, will computers still recognize it? Of course, because the new image has a high similarity to the closest vector in vector space, the computer can identify it by comparing the similarity between the vectors.

Vector dimensions in AI training

The artificial intelligence that astonishes us today often learns and trains through thousands of vector dimensions. They are like the eyes of the large AI model. When the large AI model encounters huge vector data, how can this golden pair make silicon-based organisms Smarter?

Take the large language model as an example. To put it simply, during training, the words and sentences fed to it will first be converted into vector data. When multiple groups of similar languages ​​appear in the training data, in the high-dimensional space composed of vector data, similar words will be closer, and the language The model can gradually capture the semantics and grammar between words.
For example: It will be clearer apples and watermelons is semantically close, but is far from bus. Next, the model needs to understand the context. At this time The Transformer architecture comes into play, starting from each word itself and observing the relationship weight between it and other words.
For example, in this sentence "The apple I bought yesterday is delicious", the relationship between "very delicious" and "I" has the largest weight, and the weight result is recorded as a new dimension. If it is more complicated, it is also converted into a weighted vector, and the original model is query and calculated. Generate the answer with the highest weight and output it to you, and the question and answer is completed

A powerful tool for quickly retrieving unstructured data — vector database

The training and inference process of large models is more complex. They need to process a large amount of unstructured data such as text, images, audio and video, and transform it Study for vector data. The scale of these data can easily exceed 100 million, and the vector dimensions may be as high as thousands. When choosing a database, Traditional databases can only perform one-to-one correspondence between rows and columns,
and then output accurate answers, but vector databases , is specially designed for retrieval of unstructured data. It combines item data into a three-dimensional high-dimensional Space, fuzzy retrieval in space can quickly output the answer with the highest weight.

Hippocampus for AI – Tencent Cloud Vector Database

The large AI models in the industry adopt a pre-training model and use collected data for training. It is difficult to update the knowledge base in real time. After accessing the vector database, it is like plugging a high-performance external hard drive into the model. , supports learning from the latest information such as the Internet, and is cheaper and more efficient than retraining model fine-tuning and other methods.
During online reasoning, large models actually have no memory function. Short-term memory can only be achieved by re-entering previous questions and answers, and the input content is also limited in length.
Tencent Cloud Vector Database (Tencent Cloud VectorDB) is provided to users as a service that specializes in storing and retrieving vector data. It can serve as the hippocampus of AI, allowing Artificial intelligence has a memory history. Questions and answers can be used as new training corpus and entered into the vector database for permanent storage. Even when users ask repeated similar questions, the vector database will directly give cached answers. This makes the AI ​​​​large model become smarter the more it is used. , the more you use it, the faster it becomes.

1. Introduction to Tencent Cloud Vector Database

Heavy-duty combination, industry-leading

LLM+vector database to create exclusive RAG applications

Intelligent capabilities

Tencent Cloud Vector Database can be used in conjunction with the large language model LLM. After text segmentation and vectorization, the enterprise's private domain data can be stored in the Tencent Cloud vector database to build an enterprise-specific external knowledge base, thereby providing prompt information and assisting the large model in subsequent retrieval tasks. Generate more accurate answers.

Product Highlights

high performance	Maintains million-level OPS and millisecond-level query latency
low cost	Full-process platform hosting without any installation, deployment, operation and maintenance operations
large scale	A single index supports 1 billion vector data scale
Simple and easy to use	Users can quickly operate the database through API, and development efficiency is high
High availability	Provides multi-copy high availability features to improve disaster recovery capabilities
Stable and reliable	Based on the vector search engine developed by Tencent Group, nearly 40 business lines are running stably.

2. The role of AI technology in intelligent customer service

The key role of AI technology in intelligent customer service platforms

Intelligent customer service is a solution based on AI technology. It stores corporate knowledge (such as product information, game information, etc.) through a vector database, and uses LLM for natural language processing to accurately and efficiently answer customer questions and improve customer satisfaction. Reduce enterprise customer service costs

Advantages and challenges of integrating AI technology and vector databases

challenge	Advantage
Reasoning questions to generate answers	Provide external knowledge base for LLM through vector database to improve the accuracy of large model answers
Customer service guidance skills	LLM understands guided speech. When the user's question is incomplete, it gradually guides the user to ask questions through interaction with the user, making the service more friendly.
Multiple rounds of dialogue	Through in-depth semantic analysis and precise understanding of multiple rounds, LLM achieves natural and smooth conversations and enhances the sense of intimacy with users.

3. Construction of intelligent customer service platform

Material preparation

1. VisitTencent Cloud Vector Database, click to experience now

2. Create a new vector database instance. After the configuration is complete, clickApply now

After submission is completed, just wait for creation

3. After creation, remember to enable the external network access address and configure the whitelist. What I configured here is0.0.0.0/0

Raw data acquisition

Git address: https://github.com/SophonPlus/ChineseNlpCorpus

Just get one of the categories! !

Project Development

Create database

Introduction of related dependencies

import tcvectordb
from tcvectordb.model.enum import FieldType, IndexType, MetricType, EmbeddingModel
from tcvectordb.model.index import Index, VectorIndex, FilterIndex, HNSWParams
from tcvectordb.model.collection import Embedding, UpdateQuery
from tcvectordb.model.enum import FieldType, IndexType, MetricType, ReadConsistency

Create database

client = tcvectordb.VectorDBClient(url='http://lb-esqjs4rv-35ijkpnybtm1fsuc******', username='root', key='gD0Ip*******', read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY, timeout=30)
db = client.create_database(database_name='db-test')

print(db.database_name)

Notice:

1. VectorDBClienturl and key in should be filled in with the vector database you applied for (key is the secret key)

2. read_consistency: Set the read consistency, which is a non-required parameter. The default value is EVENTUAL_CONSISTENCY. The possible values ​​are as follows:
   1. ReadConsistency.STRONG_CONSISTENCY: Strong consistency.
   2. ReadConsistency.EVENTUAL_CONSISTENCY: Eventual consistency.

Create Collection

# 第一步，设计索引
index = Index(
            FilterIndex(name='id', field_type=FieldType.String, index_type=IndexType.PRIMARY_KEY),
            VectorIndex(name='vector', dimension=768, index_type=IndexType.HNSW,
              metric_type=MetricType.COSINE, params=HNSWParams(m=16, efconstruction=200)),

)

ebd = Embedding(vector_field='vector', field='text', model=EmbeddingModel.BGE_BASE_ZH)

# 第二步：创建 Collection
coll = db.create_collection(
            name='intelligent',
            shard=1,
            replicas=0,
            description='this is a collection of test embedding',
            embedding=ebd,
            index=index
        )
print(vars(coll))

Notice:

1. The vector databaseCollection is designed to index rather than design the structure of the table
2. Required index: The two fields of primary key id and vector field vector are currently fixed and required
3. Except for the two fieldsid,vector, if there are other fields, it is not necessary to define them. Directly pass Embedding Just add
4. create_collection Parameters can be directly referred to Tencent Cloud Vector Database, the explanation is very clear
5. EmbeddingThe function of Tencent Cloud VectorDB is to provide the ability to convert unstructured data into vector data. I really love this.
6. Since BGE_BASE_ZH defaults to 768, when setting VectorIndex's dimension, the value is 768, otherwise an error will be reported!

After running the above code, the library name and collection of your vector database have been created!

Embedding

About Embedding I have to mention it separately. The engineers at Tencent Cloud are so considerate

You need to know howEmbedding converts original text into vectors for data insertion. How much convenience does this provide to developers! (Everyone knows, I was tortured!!)

Connect to the database and write raw data

Introduce dependencies

import tcvectordb
import pandas as pd
from tcvectordb.model.document import Document, Filter, SearchParams
from tcvectordb.model.enum import FieldType, IndexType, MetricType, ReadConsistency

Connect vector database

client = tcvectordb.VectorDBClient(url='http://lb-esqjs4rv-3**********', username='root', key='gD0Ips0RA***********9CBBE', read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY, timeout=30)
# 指定写入原始文本的数据库与集合
db = client.database('db-test')
coll = db.collection('intelligent')

Parse csv data and write raw text (embedding)

documents = []
count = 1
for index, row in data.iterrows():
  st_info = '{}:{}'.format(row["title"], row["reply"])
  count += 1
  print(f'{
      
      st_info} /n 11')
  res = coll.upsert(
    documents=[
      Document(
        id=str(count),
        text=st_info,
        title=row['title'],
        reply=row['reply'],
      )
    ],
    build_index=True
  )

Part of the csv data is as follows

Notice:

1. Except for the id and text fields that must be written, any other fields can be written.
2. upsert will perform overwriting. If the document id already exists, the new data will directly overwrite the original data (delete the original data and insert new data)
3. The parameter build_index is True, which means that the data is written and the index is re-created at the same time.

Similarity query

There is no problem with the test results

Similarity query localization

code show as below

import tcvectordb
from tcvectordb.model.enum import FieldType, IndexType, MetricType, EmbeddingModel, ReadConsistency
from tcvectordb.model.index import Index, VectorIndex, FilterIndex, HNSWParams
from tcvectordb.model.document import Document, Filter, SearchParams




client = tcvectordb.VectorDBClient(url='http://lb-esqjs4rv-*********', username='root', key='gD0Ips0RA********BBE', read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY, timeout=30)
# 指定写入原始文本的数据库与集合
db = client.database('db-test')
coll = db.collection('intelligent')

searchword = input('请输入查询问题:')
doc_lists = coll.searchByText(
                 embeddingItems=[searchword],
                 params=SearchParams(ef=200),
                 limit=3,
                 retrieve_vector=False,
             )
# printf
for i, docs in enumerate(doc_lists.get("documents")):
                print(i)
                for doc in docs:
                        print(doc)

Test Results:

Here I directly select a question in the csv to ask. It can be seen that the matching degree of the first of the three returned items is 0.9, which is no problem at all, and this answer is also the corresponding answer in the csv.

Notice:

1. searchByText provides the ability to batch similarity queries based on text input by embeddingItems
2. limit is used to limit the number of search conditions for each unit. For example, vector passes in three sets of vectors and limit is 3, then limit limits the similarity vectors of the top 3 returned by each set of vectors.
3. params specifies the query parameters corresponding to the index type. The HNSW type needs to set ef to specify the traversal range of the query; the IVF series needs to set nprobe to specify the number of units to query.
4. retrieve_vector specifies whether to output vector fields

Add new data

New data is essential for the database, so how does the vector database implement data insertion?

code show as below :

import tcvectordb
from tcvectordb.model.enum import FieldType, IndexType, MetricType, ReadConsistency
from tcvectordb.model.index import Index, VectorIndex, FilterIndex, HNSWParams
from tcvectordb.model.collection import UpdateQuery
from tcvectordb.model.document import Document, SearchParams, Filter


#create a database client object
client = tcvectordb.VectorDBClient(url='http://xxxxxxxxp-beijing.xxxxxx.com:40000', username='root', key='xxxxxxxx', read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY, timeout=30)


# 指定写入数据的数据库与集合
db = client.database('db-test')
coll = db.collection('intelligent')

# 写入数据，可能存在一定延迟
res = coll.upsert(
            documents=[
                Document(id='0001', text="腾讯云向量数据库（Tencent Cloud VectorDB）是一款全托管的自研企业级分布式数据库服务，专用于存储、检索、分析多维向量数据。该数据库支持多种索引类型和相似度计算方法，单索引支持 10 亿级向量规模，可支持百万级 QPS 及毫秒级查询延迟。腾讯云向量数据库不仅能为大模型提供外部知识库，提高大模型回答的准确性，还可广泛应用于推荐系统、自然语言处理等 AI 领域。",title='腾讯云向量数据库', is_best='1' ),
                Document(id='0002',
                       text="数据写入/检索自动向量化，对齐传统数据库的使用体验，用户无需关注向量生成过程，极大降低使用门槛。",
                       title='Embedding功能', is_best='1'),
                Document(id='0003',
                       text="腾讯云向量数据库 Tencent Cloud VectorDB 基于腾讯集团每日处理千亿次检索的向量引擎 OLAMA，底层采用 Raft 分布式存储，通过 Master 节点进行集群管理和调度，实现系统的高效运行。同时，腾讯云向量数据库支持设置多分片和多副本，进一步提升了负载均衡能力，使得向量数据库能够在处理海量向量数据的同时，实现高性能、高可扩展性和高容灾能力。",
                       title='产品架构', is_best='1'),
            ],
            build_index=True
        )

Let's write some custom data and see if it can be queried?

According to the feedback results, we can see that there is no problem with data execution. So how can we implement intelligent customer service?
Reference link Tencent Cloud Vector Database provides us with the http api development manual here! What are you waiting for? Go and experience it now! !

4. Vector database api

Through the above introduction, we have already understood the use of Tencent Cloud vector database. Next, we will start the local service to complete the interface call, thereby realizing the development of intelligent customer service.

1. Build FastAPI and enable local interface calling

from fastapi import FastAPI

import tcvectordb
from tcvectordb.model.enum import ReadConsistency
from tcvectordb.model.document import SearchParams


app = FastAPI()

@app.get("/search")
async def search_docs(searchword: str):
    client = tcvectordb.VectorDBClient(
        url='http://lbxxxxxxx.com:40000',
        username='root',
        key='xxxxxxxx',
        read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY,
        timeout=30
    )
    db = client.database('db-test')
    coll = db.collection('intelligent')

    doc_lists = coll.searchByText(
        embeddingItems=[searchword],
        params=SearchParams(ef=200),
        limit=1,
        retrieve_vector=False,
    )

    results = []
    for i, docs in enumerate(doc_lists.get("documents")):
        result = {
    
    }
        result["index"] = i
        result["docs"] = [doc for doc in docs]
        results.append(result)

    return results

Put the above code into main.py, and then enter uvicorn main:app --reload in the terminal to start the program

The effect is as follows:

Browser for interface access:

So far, our interface has no problem, but if we connect to the front-end program, there will be a problem === 跨域
For this problem, we can add middleware to enable cross-domain resource sharing

# 添加中间件以启用跨域资源共享
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # 允许所有来源的跨域请求，你也可以指定具体的来源
    allow_credentials=True,
    allow_methods=["*"],  # 允许所有HTTP方法的请求
    allow_headers=["*"],  # 允许所有头部的请求
)

The complete code is as follows:

from fastapi import FastAPI

import tcvectordb
from tcvectordb.model.enum import ReadConsistency
from tcvectordb.model.document import SearchParams


app = FastAPI()
# 添加中间件以启用跨域资源共享
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # 允许所有来源的跨域请求，你也可以指定具体的来源
    allow_credentials=True,
    allow_methods=["*"],  # 允许所有HTTP方法的请求
    allow_headers=["*"],  # 允许所有头部的请求
)
@app.get("/search")
async def search_docs(searchword: str):
    client = tcvectordb.VectorDBClient(
        url='http://lbxxxxxxx.com:40000',
        username='root',
        key='xxxxxxxx',
        read_consistency=ReadConsistency.EVENTUAL_CONSISTENCY,
        timeout=30
    )
    db = client.database('db-test')
    coll = db.collection('intelligent')

    doc_lists = coll.searchByText(
        embeddingItems=[searchword],
        params=SearchParams(ef=200),
        limit=1,
        retrieve_vector=False,
    )

    results = []
    for i, docs in enumerate(doc_lists.get("documents")):
        result = {
    
    }
        result["index"] = i
        result["docs"] = [doc for doc in docs]
        results.append(result)

    return results

Okay, now you can call it normally! !

5. Frequently Asked Questions

1. embedding When writing data The text field has a length limit, causing an error to be reported when writing data. This needs to be noted
2. Some of the parameters explained on the official website are not very clear, so you need to explore them yourself.

6. Summary

After the overall experience, I feel that Tencent Cloud Vector Database is really a treasure. The writing speed is also very good. If you have experience with it, please try it out.

1. The setting of embedding saved me a lot of time. I no longer have to worry about how to convert my original data into vectors
2. There are also many official documents, so you don’t have to worry about getting stuck!
   In short, Tencent Cloud Vector Database has made great efforts in user experience and learning cost, so there is no need to worry about usage and operation problems. For specific information, you can also check out the official insights on Tencent Cloud Database: