Since its launch in November 2022, ChatGPT has attracted widespread attention around the world. Its underlying technology model has also received unprecedented attention from everyone in the banking industry from top to bottom.
01Compared with traditional AI small models, large models have the following three core advantages: efficiency improvement, personalized output and interaction capabilities
Figure 1: Three core advantages of large models
Large models support unstructured data set training and have high cold start efficiency. During the model building process, small models rely on expert experience for feature engineering and model adjustment. In comparison, large models can automatically learn from massive amounts of unstructured data during the construction phase, enabling faster model building, data processing, and result generation, significantly improving cold start efficiency.
Large models can provide hyper-personalized content output for each user. The large model greatly improves the efficiency of personalized demand identification and content output tasks. The large model can learn from individual data and feedback, and create and generate new, self-consistent graphics, text and even code responses based on individual needs and contextual relationships to achieve ultra-personalized content output and provide users with more accurate customized solutions. .
Large models have powerful problem understanding capabilities. Small models often lack understanding of the context when answering questions, resulting in less relevant answers and mechanical expressions. Large models have stronger natural language understanding capabilities and can break down complex tasks based on context and understand requirements more accurately.
02 At this stage, in bank marketing and operation management scenarios, large models can give full play to their advantages
The application field of large models in banks can cover all aspects from the front desk to the back office. Every business line and functional department of the bank has the potential to tap into the application potential of large models. Under the premise of safety and controllability, once large-scale application is achieved in banks, it is expected to bring significant cost reduction and efficiency improvement.
Figure 2: Current application of large models in the banking value chain
In the early stages of application exploration, when banks plan to implement large-scale models, they usually prefer a small number of scenarios to test the water first and proceed step by step. When selecting early test scenarios, it is necessary to focus on the feasibility of implementing a large model, as well as whether it can enhance the awareness of all employees and serve as a demonstration effect within the organization. At this stage, banks mainly focus their large model applications on marketing and operations management.
In the marketing process, the personalized output and interactive capabilities of large models can assist pre-sales business development in real-time assistance scenarios for account managers. In the field of wealth management, financial account managers often need to process a large number of product documents and be familiar with the characteristics of various products and target customer groups. During interactions with customers, account managers need to accurately grasp their needs and provide them with customized investment product plans, thereby leaving a professional impression. Key issues such as how to accurately understand customer needs and how to select products that meet the needs are directly related to the success rate of business development.
Before the emergence of large models, although banks had a relatively mature system in terms of knowledge base, the application method was relatively mechanical and the efficiency was limited. The introduction of large models helps to tap the value of upper-level applications in the knowledge base, and through its personalized output and interactive capabilities, helps financial account managers improve service quality, thereby increasing the success rate of business development.
DataOps is an integrated concept of data development and operations oriented to the full life cycle of data. By reorganizing data-related process systems and tools, it builds an automated data pipeline that integrates governance, development, and operation.
Figure 3: Combining large models with knowledge bases to improve efficiency
For example, a bank uses a financial manager Q&A auxiliary tool that combines large models with knowledge graphs, to increase knowledge acquisition efficiency by 50% and business success rate by 15%. In the pre-sales process, this auxiliary tool uses the understanding ability of large models to summarize customer needs, and then generates solutions targeting these needs based on the 4K model, including customer portraits, asset allocation, market conditions and product content. product and wording suggestions. Hyper-personalized product recommendations can accurately touch customers' pain points, while professional speaking advice improves the professionalism of financial managers and helps build trusting relationships. Large model auxiliary tools can help financial managers communicate with customers and improve sales efficiency. At the same time, financial managers can also search here when they encounter general questions, such as professional knowledge, laws and regulations, or competitive product information. The financial manager Q&A auxiliary tool explains complex professional questions in easy-to-understand language for financial managers to provide answers to customers. The problem-solving rate is significantly improved compared to traditional knowledge bases.
In the operation management process, large models can be used in knowledge document question and answer scenarios to help improve work efficiency. In the use of traditional internal knowledge document assistants, banks need to manually organize FAQs and train small models, and then continuously maintain and optimize the system. However, because it mainly relies on pre-set answers for training, the knowledge assistant can only recognize the content of shorter sentences and provide standard answers in the knowledge base. In practical applications, semantic interpretation errors frequently occur, resulting in inaccurate answers. In addition, this type of robot has poor conversational interaction capabilities and cannot perform text summarization, content expansion or text polishing operations. Users need to manually browse to obtain the corresponding information after retrieving the document.
After introducing large models, banks do not need to configure task-based question and answer processes in advance, and users can interact using natural language when querying. Based on different strategies, the large model can not only reason and generate answers, but also give answers based entirely on FAQs, thereby providing more accurate answers. At the same time, after adding a large model, the knowledge document question and answer assistant can also integrate the contents of multiple documents to provide users with summarized and personalized questions and answers.
Recently, a leading city commercial bank has conducted internal testing of a self-developed large-model financial reporting assistant. Empowering business staff to reduce report writing time from hours to minutes. . In the fine-tuning process of the large model, the city commercial bank uses a financial enterprise-level knowledge base composed of procurement database, public data, and internal privatized data to output high-quality and accurate knowledge supply to the large model. . In the application process, make full use of the strong interactive capabilities of large models to disassemble the requirements input by users, and convert the disassembled steps into codes that can be input into other business subsystems or small models for calling.
For example, the large model converts the natural language input by the user into a query language for knowledge graph search. At this stage, the accuracy of this link is more than 90%. In addition, the large model can also perform secondary processing on the recalled content and present it to the user. For example, the financial report assistant can classify products according to specific dimensions, obtain the return rate of each product in recent months, and then summarize and output it in the form of a table or other structured data.
During the practice of this city commercial bank, the survey found that business staff showed a high degree of concern for the standardization requirements for prompt word projects. Therefore, the bank has built a prompt word engineering platform. Based on standard business scenarios and oriented to the ease of use of results, the bank has developed large model prompt words that meet the standards for precipitation and reuse. In addition, the prompt word engineering platform also supports comparison of the effects of different prompt words.
The city commercial bank began internal research on demand scenarios and designed a large-model system architecture in February 2023. By September 2023, it had released two versions of the large-model financial reporting assistant. The first version was released in June and focused on policy and economic interpretation and analysis report writing. The second version was released in August and began comprehensive internal testing, with enhanced operational analysis and auxiliary decision-making functions.
Figure 4: Large model implementation plan of city commercial banks
In the planning of the implementation of the large model, the city commercial bank used the financial report assistant as the starting point to first allow most business staff to quickly experience the improvement in office efficiency brought by the large model. In the future, the bank will conduct implementation exploration in both horizontal and vertical directions. Vertically, the Financial Report Assistant has been upgraded from summary capabilities to analytical capabilities, and scenarios such as corporate operation analysis, corporate profitability analysis, and corporate asset quality analysis have been explored. Horizontally, we will expand the capabilities of the Financial Report Assistant to other business directions, with the goal of improving efficiency, and prioritize the application of internal service scenarios such as financial investment consulting, post-loan corporate operating status analysis, and intelligent customer service assistance.
03 The application of large models in banks still faces two major difficulties: the illusion of large models and the timeliness of answers
Until the problem of large model illusion cannot be solved, the application of large models will be mainly for internal use and will not be able to meet customers for the time being. Although the ability to understand the intention of large models has reached more than 90%, the problem of large model illusion cannot be completely solved for the time being. This requires large model applications to be used under human supervision. At this stage, the output accuracy of large models can reach about 70%-80%. Because banks have high requirements for accuracy and controllability, large-model customer-facing applications cannot be implemented for the time being.
Currently, banks have two main solutions to the problem of large model illusion:
1. Apply the answer after a second manual review.
2. Combined with the knowledge base, through the prompt word project, the large model is restricted to strictly use the document content in the knowledge base to answer, and when there is no response answer, the large model is prevented from answering such questions.
Large model knowledge updates are slow, but this problem can be temporarily solved when combined with plug-in knowledge base applications. Although the problem of timeliness of large model answers has not been completely solved, there are solutions. Large models need to infuse the latest data into the model through pre-training, so it is difficult to include the latest knowledge, and the timeliness of its answer content is also limited. For example, at this stage, GPT4 is trained based on data before January 2022. Therefore, when asking about China's GDP data in 2022, GPT4 cannot provide an answer.
In order to solve the problem of timeliness of large model answers, companies usually use plug-in knowledge bases to integrate large models with knowledge bases. For example, when banking staff have the latest data to analyze, users can upload the latest documents and incorporate the documents into the knowledge base. Q&A on this document will be available the same day. But in fact, this cannot essentially solve the problem of timeliness of large models, because the update and maintenance of the knowledge base also needs to be done manually. This solution only transforms "updating the large model" into "updating the knowledge base."
04 Large models and small models each have their own advantages. In order to achieve demonstration effects, strong alliances are needed
Figure 5: Comparison of differences between large and small models
The large model has lower cold start cost and strong generalization ability, while the interpretability and accuracy of the small model are higher than that of the large model. Therefore, only by leveraging the advantages of the large model on the basis of the original small model can the application value be better released. For example, in the smart customer service scenario, using large models to structure the data during the construction phase can speed up the cold start of small models. In the application stage, using large models to split complex problems can help small models output answers more accurately. Finally, in generative scenarios, the creative output capabilities of large models can make up for the shortcomings of small models in this regard. The combination of the two can achieve a demonstration effect within the organization in a short period of time.
Since various application scenarios in banks have high requirements on interpretability and accuracy, there are currently two mainstream ways to combine large and small models into implementation. The first is to cascade large models with small models. For example, large models are used to capture customer historical transaction records, communication information, etc. from complex data sources, automatically mark the data, and output structured data to enrich small model data sets and improve small model prediction effects. The second type is the central brain that uses large models as small models to solve problems. For example, a large model is used to decompose a complex problem into solution steps, and small model capabilities are invoked in each step, and finally the large model is used to provide summary output.
In the future, in addition to more general large-model implementation applications, such as structured output of unstructured data, code assistants, etc., banks are actively exploring other application directions. Banks face strong supervision and have high requirements for information accuracy. Until the big model illusion is completely solved, bank core systems cannot deploy large model applications. In the future, non-core system applications will be used as Explore directions. Mainly apply the ability of large models to understand complex content to achieve efficiency improvements.
In terms of customer-facing applications, the cascading combination of large models and small models has opened up a new direction of exploration for bank telesales agents. At this stage, there are two major problems in the implementation of large-model telemarketing agents. First, the problem of large-scale model illusion cannot be solved. Secondly, large-scale models require a certain amount of time to reason about answers, and voice call scenarios require delay. Smaller, larger models cannot suffice. The above problems all arise because the large model is responsible for "generating answers". If the large model is used to "select answers", the problems can be avoided. Banks can set up FAQ questions and answers that can be used by customers who pass compliance audits, and record their voices. The big model is responsible for understanding customer problems during interviews and selecting the most appropriate answers in the FAQ for output. Finally, based on the customer's answers, it is judged whether the customer has loan intention and the customer is marked. Since the flexibility of answers in this unexplored direction is relatively weak, in the future, priority will be given to implementing experiments in the scenario of waking up sleeping customers.
In terms of event-driven risk warning, the cascade of large models and small models provides new ideas for loan repayment ability warnings. At this stage, enterprises rely on time-series knowledge graphs to completely sort out the chain of financial risk events and conduct warnings and predictions before new risks occur. The large model can monitor the occurrence of new events in real time on the front end. When relevant events occur, it can understand the content of the event, comprehensively integrate the company's operating conditions, conduct multiple rounds of interaction with the time series knowledge graph, determine the event chain, simulate risk evolution, and issue alarms.
To sum up, large models will be implemented and promoted in the banking industry from point to point, from use case development to scenario expansion, and finally form an end-to-end full-process application. In the future, when it comes to large model capacity building, banks will continue to pay attention to issues related to the construction of intelligent computing centers caused by export controls, as well as data capacity building issues closely related to input corpus for large models.