ByteDance AB’s experimental experience sharing: How can companies build a data-driven experimental culture?

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

For more technical exchanges and job opportunities, please follow the WeChat official account of ByteDance Data Platform and reply [1] to enter the official communication group

Recently, CCF TF held the 123rd sharing event. The theme of this issue is "User Experience Engineering". CCF TF is a regular inter-enterprise cooperation and exchange platform created by the China Computer Society for computer professionals in the corporate world. In this issue, experts with good practices in the field of user experience engineering, such as Huoshan Engine and Niuyougu Company, were invited to discuss together. In the era of digital intelligence, faced with complex user scenarios and ultra-high experience requirements, how to continue to pursue excellence and enhance product competitiveness under new trends through effective experience measurement and management and the use of A/B testing technology.

During the event, experts from the Volcano Engine AB test platform combined ByteDance’s AB experiment experience to conduct on-site sharing on the theme of "Data-driven Experimental Culture". AB testing is an essential part of the user experience optimization process, and is also known as the "gold standard" for effect measurement. ByteDance has a unique data-driven experimental culture. In such a soil, AB testing has also become a Bytedance is a powerful tool for optimizing user experience.

In enterprises, the establishment of a corporate experimental culture that can maximize the effectiveness of AB experiments can be focused on three dimensions: organization, methodology and tool application. At the corporate organizational level, it is necessary to establish and improve the process mechanism for AB experiments. Byte's AB experiments are closely integrated with product design and development, and every small decision will be experimentally verified before it is fully launched. At the same time, the supporting organizational mechanism is equally important. By building an AB test interface team, the entire link from experimental design to development, launch, and decision-making can be well run. A dual-track mechanism of "tools + consulting services" such as Volcano Engine DataTester is being used internally by Byte and is also providing services to external companies.

At the methodological level, an enterprise with a healthy AB experiment culture is an enterprise that combines growth ideas with AB experiment concepts, transforms business goals into core measurement indicators, and continuously selects the best results through the design, operation, and iteration of AB experiments. Optimal strategies approach business goals; companies should also assist in achieving scientific evaluation results through four steps: building an indicator system, setting optimization goals, managing experimental results, and accumulating decision-making experience.

At the tool level, a reliable and flexible AB experiment platform can save enterprises a lot of labor costs and time costs. In ByteDance, "everything can be AB tested," and the Volcano Engine DataTester, after years of business polishing by ByteDance, has the ability to be easy to learn and use, fast and safe, and can handle the experimental needs of complex scenarios. Data from mid-2023 shows that Byte has conducted more than 2.4 million AB experiments through DataTester, with more than 4,000 new experiments being added daily and more than 50,000 experiments being run simultaneously.

For user experience optimization, the difficulties in experience measurement include evaluation subjectivity and user diversity. AB testing can help the business team provide a more scientific data reference and provide feedback on the quality of the product experience through objective data comparison, but at the same time A good combination of data experiment culture concepts and advanced tools is required. With the support of mature experimental culture and AB experimental tools, ByteDance has practiced an effective way to improve product user experience, and enterprises can refer to the above experience and build or introduce AB testing "tools + consulting services" "The dual-track mechanism approach starts from the three dimensions of organization, methodology and tool application to build the company's own data-driven experimental culture, use experiments to make scientific decisions, and use data to drive growth.

Click to jump to DataTester to learn more

ByteDance AB’s experimental experience sharing: How can companies build a data-driven experimental culture?

Guess you like