introduce
Document-oriented proxies are starting to gain traction in the commercial world. Companies are increasingly utilizing these tools to leverage internal documentation, thereby enhancing their business processes. A recent McKinsey report[1] highlighted this trend, showing that generative AI could add $2.6-4.4 trillion per year to the global economy and automate 70% of current work activities. The study identified customer service, sales and marketing, and software development as the main sectors that will be affected by the transformation. Much of the change comes from the fact that, through the use of solutions such as document-oriented agents, employees and customers can more easily access the information that powers these areas within the company.
With current technology, we still face some challenges. Even if you consider new large language models (LLMs) with a 100k token limit, these models still have a limited context window. While 100k tokens may seem like a large number, it's a small number when we consider the size of the database powering customer service departments and the like. Another problem that often comes up is the inaccuracy of the model output. In this article, we provide a step-by-step guide to building a document-oriented agent that can process documents of any size and provide verifiable answers.
We use a vector database, ChromaDB, to enhance model context length capabilities, and Langchain to facilitate integration between different components in the architecture. As our LL.M. we use OpenAI's chatGPT. Since we want to serve our application, we use FastAPI to create endpoints for users to interact with our proxy. Finally, our application is containerized using Docker, which allows us to easily deploy it in any type of environment.
Vector Databases: The Heart of a Semantic Search Application
Vector databases are critical to unleashing the power of generative artificial intelligence. These types of databases are optimized to handle vector embeddings—representations of data that contain rich semantic information from raw data. Unlike traditional scalar databases, which struggle to cope with the complexity of vector embeddings, vector databases index these embeddings, relate them to their source content &