Resource costs reduced by 60%! Volcano engine ByteHouse helps digital marketing platform Qianchuan Network reduce costs and increase efficiency

For more technical exchanges and job opportunities, please follow the WeChat official account of ByteDance Data Platform and reply [1] to enter the official communication group
Recently, China's well-known content social platform integrated marketing company Qianchuan Network has reached a cooperation with the volcano engine ByteHouse . Qianchuan Network will upgrade the real-time query and analysis capabilities of the underlying data engine through the volcano engine ByteHouse to achieve high throughput, low latency, and high stability while reducing resource costs, and provide brands with a full range of public opinion monitoring services.
Qianchuan Network is an AI-driven social media big data solution provider that aims to use unique big data and independent research and development system empowerment to provide enterprise-level customers with more valuable KOL and public/private domain traffic management solutions. , and provides comprehensive digital marketing and management services such as social media listening, industry insights, SCRM system management, etc., to build a three-dimensional content ecology covering communication, placement, monitoring, evaluation, etc. for advertisers to create more valuable marketing services.
Today, with the rapid spread of digitalization, public opinion monitoring has become an essential tool for every organization and brand. Effective public opinion monitoring can help brands quickly respond to potential negative news and reduce the risk of reputational damage in an era of rapidly changing information. It can also allow companies to adjust marketing strategies based on public opinion data, improve consumer satisfaction, and seize industry opportunities. Qianchuan Network's mobile Internet advertising monitoring platform, social media data analysis platform, Xiaoying short video public opinion monitoring platform and other products provide brands with cutting-edge, professional and scientific strategy and project service support through fast, comprehensive and accurate advantages. .
Public opinion monitoring has high requirements for real-time data. Among the existing technical solutions, Qianchuan Network provides public opinion similarity retrieval capabilities through self-built Elasticsearch . However, from the perspective of performance and cost, it cannot fully meet the needs of Qianchuan Network. In terms of performance, the business requires the system to support real-time retrieval of tens of thousands of QPS, and it is difficult to achieve such high concurrency indicators using the ElasticSearch solution. In terms of cost, using the ElasticSearch solution is accompanied by high machine resource costs. As the amount of data continues to increase, machine resource costs will become increasingly uncontrollable. In addition, the data writing and query processes are relatively complex, resulting in high application development costs.
In this context, Qianchuan Network further seeks solutions to reduce costs and increase efficiency, and sets its sights on the volcano engine ByteHouse . As a cloud-native data warehouse, Volcano Engine ByteHouse provides extremely fast data analysis services, which can support real-time data analysis and massive data offline analysis. It has been tested by ByteDance internally and has been widely used in external Internet, games, finance, automobiles and other fields. produce actual results.
The core problem faced by both parties in smoothly migrating from Elasticsearch to ByteHouse is that thousands of online public opinion data reach billions, and the peak QPS reaches tens of thousands. ByteHouse must not only have the ability to handle complex OLAP analysis scenarios, but also support Qianchuan network's ultra-high QPS online query service, and achieve high throughput, low latency, and high stability.
In order to better solve the problem of high concurrency enumeration in Qianchuan network, ByteHouse adopts a multi-copy mechanism to write all data to the master node, automatically synchronize to multiple copies in real time, and achieve automatic load balancing and one-write-multiple-read through gateway query. This linearly improves concurrency capabilities.
ByteHouse high concurrency enumeration technology solution
In addition, ByteHouse also supports completing business logic through SQL writing and querying, further simplifying the data processing process and improving development efficiency. In terms of resource efficiency, ByteHouse adopts a flexible resource model that can support any change in the number of nodes and replicas. Users only need to purchase minimal resources at the beginning, and can be expanded immediately as business grows, making costs more controllable.
Tang Genping, head of big data at Qianchuan Network, said: " ByteHouse is very stable in querying and writing under billions of data. It not only helped us improve performance, but also further reduced costs. Currently, the total resource cost has been saved by 60%. %."
At present, videos, pictures, etc. have become the mainstream content carriers on the Internet. Qianchuan's Xiaoying short video public opinion monitoring product is based on data authorization from the short video platform , combined with large language model ( LLM ), natural language analysis (NLP) and other AI capabilities. It can help governments and enterprises quickly, comprehensively and accurately grasp the dynamics of public opinion in short videos. ByteHouse ’s soon-to-be-released vector search and full-text search functions will also further enhance Xiaoying’s capabilities in short video public opinion monitoring business.
Click to jump to Volcano Engine ByteHouse to learn more
Microsoft launches new "Windows App" Xiaomi officially announces that Xiaomi Vela is fully open source, and the underlying kernel is NuttX Vite 5. Alibaba Cloud 11.12 is officially released. The cause of the failure is exposed: Access Key service (Access Key) anomaly. GitHub report: TypeScript replaces Java and becomes the third most popular. The language operator’s miraculous operation: disconnecting the network in the background, deactivating broadband accounts, forcing users to change optical modems ByteDance: using AI to automatically tune Linux kernel parameters Microsoft open source Terminal Chat Spring Framework 6.1 officially GA OpenAI former CEO and president Sam Altman & Greg Brockman joins Microsoft

Guess you like