Big data industry innovation service media
——Focus on data and change business
In today's era of rapid digital development, data has become the core driving factor of business operations and management decisions. Whether it is a large multinational enterprise or a new start-up company, it has become crucial to gain insight into data correctly and quickly. However, traditional BI tools often have certain technical barriers for users, requiring skilled operation skills and complex query statements, which makes it difficult for most enterprise employees to dig deep into the value of data.
So, how to make data analysis more intuitive and humanized?
On August 10th, Netease Shufan held an industry summit themed "Intensive cultivation of digital intelligence to accelerate innovation - Netease Shufan City Tour (Beijing)", released the latest progress and development of AIGC combined with data analysis, software development and other fields. results.
Wang Yuan, vice president of NetEase, executive director of NetEase Hangzhou Research Institute, and general manager of NetEase Shufan, introduced the application potential of AIGC
Among them, in the field of big data, Netease Shufan released a conversational BI product developed by integrating cutting-edge AIGC technology - Youshu ChatBI. This is a brand-new solution in the field of data analysis. It interacts with the BI platform through natural language to realize data query and analysis. This method is like having a conversation with a data expert, simple and direct, without complicated technical background. This innovation is expected to open a new paradigm of data analysis.
How did Netease Shufan do it? What are the significant differences between ChatBI and previous BI? How to solve the "nonsense" problem that the large model is widely complained about, this article will conduct an in-depth analysis of these problems.
AIGC changes the way of human-computer interaction, and data analysis is different from now on
Yu Lihua, General Manager of NetEase Shufan Big Data Product Line, believes that human-computer interaction is actually very important, and better human-computer interaction will often bring significant technological and industrial innovations, such as computer graphical interface, iPhone multi-touch The emergence of the industry has had an important impact on the industry.
After decades of development, the interaction between humans and computers has transformed from being complicated and difficult to use to being intuitive and humanized today. Simply put, human-computer interaction has gone through three significant stages of change, and changes in each stage have brought profound impacts on data analysis.
The first stage: program command line interaction
In the era when the computer was just born, the interaction with the computer basically relied on the program command line. Users need to be proficient in various commands in order to let the computer complete specific tasks. Although this method provides great flexibility for early computer experts and enthusiasts, it is obvious that for most ordinary users, its learning curve is relatively steep and the threshold for use is high.
The second stage: graphical interface interaction
As technology advanced, Graphical User Interfaces (GUIs) started to gain popularity. Users no longer need to memorize complex commands, they only need to operate with a computer mouse or touch screen of a mobile phone to complete the task. The appearance of the graphical interface significantly reduces the difficulty of computer use and greatly expands the computer user base. For data analysis, the graphical data display and drag-and-drop operation interface make the BI platform more intuitive and easy to use, but it still requires a certain amount of training and proficiency, and there are still quite a few barriers to use.
The third stage: AIGC-based natural language interaction
In 2023, large models and AIGC will quickly enter people's field of vision, opening up a new way of thinking about human-computer interaction for us. Users can directly talk to the machine in natural language, just like talking to another human being. This means that users can easily and directly interact with computers for data query and analysis, regardless of their professional skills or background knowledge.
NetEase Youshu ChatBI product application interface
In general, AIGC technology has completely changed the rules of the game for data analysis, bringing data analysis into a new era.
Open a new paradigm of data analysis with ChatBI
ChatBI is a revolutionary breakthrough in the field of data analysis. It perfectly combines BI and AI technology to provide enterprises with unprecedented convenience and efficiency. Specifically, compared to traditional BI, ChatBI has the following significant advantages:
1. Dialogue is analysis, which significantly reduces the threshold for data analysis.
Compared with traditional BI tools, ChatBI's conversational data analysis brings unique convenience to enterprises and individuals. Traditional BI usually requires in-depth training, programming knowledge, and a deep understanding of data structures, which makes employees with non-technical backgrounds feel at a loss during the data analysis process. But ChatBI breaks down this barrier, providing a natural and intuitive way to gain data insights.
In ChatBI, questions are no longer asked through complex query languages or drag-and-drop interfaces, but through everyday language. This means that even non-technical members of marketing, sales or management teams can ask directly and get the answers they need. Additionally, this direct interaction greatly speeds up response times, enabling real-time decision-making.
In addition, conversational data analysis also enhances data accessibility. Traditional BI tools may have a large number of options and configurations on the user interface, and ChatBI simplifies this process, allowing users to focus on the real problem rather than the operation of the tool. This convenience not only improves efficiency, but also makes data analysis more attractive, encouraging more people to participate in it, thereby promoting data consumption.
2. Report generation and interpretation, using business language to make data analysis results clearer and easier to understand.
ChatBI can not only generate data analysis reports, but also explain the analysis results in business language and management language. This approach is easier to understand and accept for non-technical decision makers. When data is no longer a cold number, but a "partner" with whom we talk, explain, and discuss, data analysis and business decision-making processes become more efficient.
Using the example of a marketing team, consider a scenario where a marketing manager wants to know how well a social media advertising campaign has performed in the last month, specifically which posts or ads received the most engagement and conversions.
In ChatBI, the marketing manager only needs to enter: "How effective was the social media advertising campaign last month, which content had the highest interaction and conversion rate?" The system may quickly respond: "Last month, we posted 'Summer Sale' post got the most engagement with 5,000 likes and 500 comments. The 'New Product Launch' video ad on Douyin had the highest conversion rate at 3.5%, bringing us 2,000 new users .”
3. More in-depth data mining capabilities to discover hidden business associations in big data.
Traditional data analysis often relies on preset rules and models, while ChatBI uses advanced AI technology to automatically learn rules from big data and handle more complex and in-depth analysis tasks. This means that ChatBI can help companies discover previously overlooked business connections and potential opportunities.
Let's look at an example.
Consider a large retail company that has multiple sales channels, including physical stores, online stores, and mobile apps. In addition, they also operate a loyalty point system where consumers can earn points after purchases and redeem them for goods or services at specific times or events.
Using traditional data analysis tools, you may get some superficial analysis results: in physical stores in some cities, the sales peak is at 3-5 pm on Friday; 10 point sales are the best.
However, digging deeper using ChatBI, they were more likely to find some more complex and subtle patterns, such as: consumers who shopped for home or kitchen supplies in a brick-and-mortar store on a Friday afternoon were more than 65 percent more likely to spend the next within 48 hours of purchasing culinary or food-related items through the mobile app. Further analysis revealed that this may have something to do with some consumers buying ingredients or other related products through mobile apps as they look to try out new recipes over the weekend after buying new kitchen supplies on Friday. To this end, the retail company can offer discounts on ingredients in the mobile app to consumers who buy kitchen supplies in physical stores, further promoting their secondary consumption.
4. Self-evolution ability to realize personalized BI.
A distinctive feature of ChatBI is its self-evolving ability. As usage increases, it will continue to learn and optimize to provide more accurate analysis results. More importantly, ChatBI can understand and adapt to the habits of each user, providing each user with a truly personalized service, which makes the data analysis process more flexible and efficient.
5. Open up data applications and promote data-driven business process automation.
ChatBI is not just a data analysis tool, it can also serve as a bridge between BI data analysis results and other application systems. By combining data analysis results with business processes of enterprises, ChatBI can automatically trigger corresponding business operations. This integrated approach not only improves productivity, but also brings greater flexibility and responsiveness to the enterprise, enabling true data-driven decision-making.
In this era of rapid development of big data and AI technology, ChatBI undoubtedly points out a new and more efficient way of data analysis for enterprises.
Solve the problem of AIGC's "nonsense" and create a commercially available ChatBI
It is worth pointing out that although ChatBI follows the design concept of large models, it does not mean that we can directly use general large models such as ChatGPT for data analysis and chart generation.
Although ChatGPT is a powerful model, it is not satisfactory in directly invoking data from databases or data platforms. This involves data integration between different platforms, behind which lies a series of complex technical product integration challenges. Furthermore, ChatGPT's performance in data visualization is relatively weak. All it can do is simple table display, while real data analysis requires more complex and detailed visualization charts.
More importantly, ChatGPT, as a general knowledge model, may suffer computational and interpretation biases in specialized domains such as data analysis. For example, when it comes to complex mathematical calculations or statistical analysis for specific datasets, ChatGPT may lead to inaccurate answers due to incomplete information, inference limitations of the model, or other factors. However, for BI products, data accuracy is the most basic requirement, and the potential inaccuracy of ChatGPT is unacceptable in the field of data analysis.
Yu Lihua believes that credibility is the core key to the mature commercial use of "AIGC+BI". Netease Shufan has made great progress through the four methods of understandable demand, verifiable process, user intervention and adjustable query conditions, and product operation and adjustment. Improve the credibility of the results of countless ChatBI.
How to do it? Judging from the innovative practice of Netease Shufan, the following methods help to solve the problem of "nonsense" existing in large models and AIGC, and improve the accuracy of data analysis:
Yu Lihua, General Manager of Netease Shufan Big Data Product Line, introduced ChatBI products
1. With the help of NL2SQL capability to achieve double verification, improve the accuracy of generated content and reduce the error rate.
Netease Shufan has significantly enhanced the interactive experience between users and databases by introducing NL2SQL (Natural Language to SQL) technology. In the traditional database query method, users need to clearly grasp the SQL language and the corresponding database structure to extract the required information. With NL2SQL, users only need to use natural language to describe their requirements, and the back-end system will convert them into appropriate SQL statements, which greatly simplifies the query process.
In an interview with the media, Yu Lihua pointed out that the previous natural language processing ability was actually relatively weak, which led to challenges in the past NL2SQL technology in dealing with vague or unclear queries and understanding complex query intentions. AIGC and large models have brought NL2SQL Greater context and intent understanding. This means that when dealing with ambiguous, ambiguous or complex user queries, the system can more accurately identify the real needs of users. For example, a user may ask "What is the product with the highest sales in the last three months?" This is a complex query involving time, value, and sorting. The powerful natural language processing capabilities of the large model ensure that it can accurately parse such requirements, and into a valid SQL query.
Large model + NL2SQL, this mechanism has opened up the entire link of user needs, data acquisition, and data analysis to ensure the accuracy and reliability of data. At present, Netease Shufan has been able to handle more than 300,000 different natural language questions, and its effect has reached the level of GPT-3.5 in terms of accuracy.
2. The granular knowledge of tables is enhanced, and relevant table knowledge is "fed" to the large model in real time according to user questions.
Another innovation of NetEase Shufan is to “feed” the large model with the knowledge of relevant table granularity in real time according to the user’s questions, so that the large model can understand the structured data more thoroughly and significantly improve the accuracy of the generated results.
Tables, as the core form of structured data, are the cornerstone of data decision-making. Real-time "feeding" of large model tabular knowledge means that the model is not only based on raw training responses, but gets real-time tabular data to answer. This is like asking an expert a question. He not only uses old knowledge, but also immediately checks the latest information to answer. The advantage of this method is: because it uses the latest data, it can ensure the timeliness of the answer; it improves the flexibility of the model so that it can answer more specifically, rather than based only on training knowledge.
3. Instill the company's proprietary knowledge into the model through customized prompts to improve the effect of personalized scene analysis.
Custom prompts, which infuse models with corporate know-how, are key to making models more personal and relevant. Pretrained models, while possessing broad knowledge, may lack an understanding of specific corporate cultures, business characteristics, workflows, and terminology. By customizing the prompts, the model can more accurately provide answers to the enterprise, ensuring that its output is consistent with the business characteristics of the enterprise.
Also, every business' needs are unique, and general answers may not apply in all scenarios. Hence, instilling proprietary knowledge into a model not only deepens its understanding of the business, but also ensures that it is provided with tailored strategies and recommendations, thereby improving the accuracy of decision-making.
4. The data model and query conditions are structured, and the user can intervene and adjust, and the data model or query conditions can be switched at any time.
Yu Lihua believes that in data-driven decision-making, users expect accurate, targeted and instant data analysis results. But automated data analysis systems, although efficient, may not always meet these requirements. Sometimes, the results output by the system may not exactly match the user's expectations or their specific business context.
At this time, it becomes particularly critical to structure the data model and query conditions and allow users to intervene and adjust. If the result given by ChatBI is wrong, the user knows where the mistake is, and the platform can assist the user to correct the mistake easily, which is the intended meaning of a trusted ChatBI. For example, a retailer may be interested in last month's sales data, but the system defaults to cumulative data for the current year. In an adjustable structure, the retailer can simply switch query criteria to get the information he needs without having to rebuild the entire query.
In addition, business environments and requirements are changeable. Today's analytical framework may not be applicable tomorrow. Allowing users to intervene, adjust, and switch data models or query conditions not only ensures the immediacy and accuracy of data analysis, but also gives users greater control over data, enabling them to respond quickly and make decisions based on specific needs.
5. Build a product operation feedback mechanism to continuously optimize product performance through data feedback and knowledge operations from users and administrators.
The development of an innovative product does not happen overnight, but becomes better through continuous iteration and optimization. To this end, NetEase Shufan has built an operational feedback mechanism involving users and administrators.
Users are usually the first to discover and experience data errors or inconsistencies, and their feedback is often highly targeted, helping to quickly locate and fix problems. For the feedback data, the administrator can mark it as "badcase" and optimize accordingly. This kind of marking and sorting makes the occurrence of problems no longer isolated events, but is included in a continuous optimization process. This also provides valuable training samples for subsequent data analysis and model training, which is helpful for continuous iteration and optimization of the model.
This feedback mechanism not only enhances the reliability of data, but also enhances the interactive relationship between users and products, and deepens the connection between users and products, thus realizing the two-way optimization of data and products. When users see their feedback being taken into account and processed, they are more likely to build trust in the product and continue to use it.
In the world of data analytics and content generation, accuracy isn't just about reputation and user experience, it's critical for decision support. Wrong data or analysis may lead enterprises to make unreasonable strategic decisions and bring immeasurable losses. If ChatBI conducts wrong data analysis and content generation, or even "nonsense", it will not only reduce users' confidence in the product, but may also damage the company's brand image.
Therefore, Netease Shufan attaches great importance to the accuracy of the content generated by the large model. Through the above methods, NetEase Shufan ensures the accuracy of the content generated by the large model, and creates a trusted ChatBI with understandable requirements, verifiable process, user intervention, and product operation, laying a solid foundation for the commercial application of this new data analysis method. established a solid foundation.
For NetEase Shufan, the release of the ChatBI product is not only a key step in building its AIGC+BI business blueprint, but also a reshaping of the existing digital intelligence competitiveness. Netease Shufan's persistence and in-depth exploration of this innovative direction will become an important driving force for the company's future development. In the future, by successively launching a series of products for different needs, Netease Shufan is expected to create a new and more intelligent business ecosystem, which will not only boost its own rapid development, but also promote the innovation and transformation of the entire industry.
For enterprises, the launch of ChatBI has undoubtedly lowered the threshold for data analysis and use, enabling more people to easily obtain and utilize data. This open and convenient feature will greatly expand the scale of data consumption groups, increase the breadth, depth and frequency of data consumption, and then promote the upgrading and transformation of enterprises in the direction of digitalization and intelligence. Today, when the global economy is increasingly driven by data and intelligent technology, this transformation has significant significance for the competitiveness of enterprises.
From a broader perspective, innovative products such as ChatBI are closely connected with China's current digital economy development strategy. According to the "Digital China Development Report (2022)" recently released by the State Internet Information Office of China, the scale of my country's digital economy has reached 50.2 trillion yuan in 2022, ranking second in the world in terms of total volume, and its proportion in GDP has increased to 41.5. %. Behind this trend is the joint promotion of many innovative products like ChatBI.
As Wang Yuan, general manager of Netease Shufan, pointed out at the meeting, the key driving force for the construction of digital China lies in digital intelligence productivity and software supply capabilities. If the construction of digital China is compared to a towering tree, then the development of digital intelligence productivity can promote digital innovation and application "to flourish". In the future, with the continuous improvement and promotion of such products, they will further promote the rapid development of China's digital economy and make greater contributions to the prosperity of the country and society. From this perspective, ChatBI is not only a successful product innovation, but also a profound insight and positive response to the future direction of business and social development.
Text: Yuemanxilou / Data Ape
