Author: alishayang, Tencent TEG Data Analyst
We hope that every new product feature and feature revision can bring a better interactive experience, and the indicators that the project team pays attention to can be greatly improved. However, expectations do not represent real effects, and the real effects brought about by the function going live need to be objectively described through data. In this article, we will analyze the daily work of product data analysis, from data collection to product analysis framework combing, and then introduce how to feed back products and operations through data.
As the initiator of the product function, the product manager will have a preliminary idea in his mind when he selects the requirements from the many needs, which incremental users can the new function help the product cover, and which index improvement the new function can bring. How much is the increase? However, when product functions are launched and users use user behavior data, product and operation classmates often ask questions like this: Has the iterative optimization of functions achieved the expected results? What is the specific improvement effect? At this time, a very important step after the function goes live-"evaluation" is introduced.
In fact, "evaluation" is the process of turning business problems into data problems. Through the effect evaluation, we can determine the true difference between the online function and the expected effect. If the effect is better than expected, the evaluation result can be used as a support point for the promotion of operating classmates' functions; when the effect is obviously not up to expectations, further analysis of the reasons for not meeting expectations based on user behavior is required to provide suggestions for function optimization or revision. Therefore, "evaluation" is a necessary step after the new function is launched.
To carry out functional effect evaluation, complete data and a full understanding of the industry and products are two essential parts. Next, we will introduce the work we have done in data collection and specific product function evaluation.
Data collection: accurate business data and user behavior data are indispensable
Students involved in the test verify the accuracy and completeness of the buried point data to avoid rework after the requirement goes online
User behavior data plays a vital role in understanding user habits. Currently, user behavior data is reported mostly in the form of embedded points. The more content reported, the greater the impact on performance. Therefore, this leads to a dialogue that often appears when data students and development students review their needs:
Data classmate: I want to report this data, which is very important for analysis!
Development classmates: This report will affect page performance, so you can't do it!
In fact, everyone's goal is to make the product better. Therefore, when data students ask for points, they can think more about whether the requirements are necessary and whether there are other data that can be approximated instead? When aligning the requirements with the development classmates, you can also communicate more. Is there an implementation method that has a smaller impact on performance?
In addition, it is often encountered that data reporting problems are discovered after the buried points are online. Here are three representative examples that are often encountered:
Example 1-Incomplete data: Our product is a landing page product that serves advertisers. The delivery of advertisements will involve multiple traffic channels, and different channels have different restrictions on the time node for data reporting. Such situations may be possible Lead to incomplete reporting of certain channels;
Example 2-Inaccurate data: The overall form of our landing page, the header is the first image of the product, the next is the product detail page, and the last is the form content to be filled in for purchase. Under this page structure, we hope to be able to get the position where the user last jumped out of the page. But when we counted the reported data, we found that 90% of users reported page views less than 10% of the overall page length after completing the form. This is obviously inconsistent with conventional understanding;
Example 3-There is an abnormality in the data reporting link: The back-end students said that the buried point demand has been released and is available for use. But the data classmate opened the database and found that it was empty. Then I started to pull the developers to investigate the reason why the data did not arrive in the library. Is the data not collected at all? Or is it collected and not reported to the server? The server has received it and hasn't pushed it to the database? After verifying all of them, it may be found that it is caused by a problem in one of the links, but the entire investigation consumes a lot of manpower.
There are too many pits to step on, and we are thinking, whether to bury some requirements, like functional requirements, let test students help check before going online? The answer is feasible and very efficient! ! ! Therefore, we have sorted out and refined the whole process specification of the buried point data reporting and testing, and exposed the buried point data problem before the online release. Here is also a brief introduction to our specification:
Full process coverage: report from data -> receive data from server -> data storage, test students need to verify the accuracy and completeness of the data flow in each process;
Multi-dimensional test cases: Taking into account the complexity of landing page traffic channels, the specification requires test students to write multi-channel test cases based on the characteristics of channels, and pay attention to the data reporting of different channels;
Multi-user test case: In order to avoid using only one user to simulate click behavior, the specification requires test students to simulate the click behavior of different users in batches and pay attention to the data report of each simulated user;
With this specification, the sharp eyes of the test students can help us find many hidden data problems before going online, which indirectly saves a lot of time for data students to verify data and develop students to rework.
Data students should participate in the discussion of the needs of business data precipitation
It is generally understood that business data is more of the front-end and back-end development students, in order to ensure the complete function of the data that needs to be stored. If the data classmates do not align with the development classmates in the demand stage, it may lead to the occurrence of some fields that the data concerned are ignored by the development classmates. When the evaluation was started, it was found that the data dimensions were not enough, and the evaluation was started after the corresponding fields were promoted for research and development. The entire evaluation period was obviously lengthened.
Regarding the precipitation of business data, a small example from personal experience: the new function calls the interface for identifying abnormal accounts provided by the algorithm. At that time, the development students only marked the account with a label (0: normal 1: abnormal). When the data classmates evaluated the effect of the entire interface, they found that they needed to count the distribution of the algorithm interface score. At this time, it was discovered that the data stored by the developer was missing the scoring field. This is because the students did not fully communicate with the developers in the early data. This is why data analysis students need to participate in the discussion of business data precipitation requirements.
Product analysis: build a suitable product analysis framework to realize visual monitoring of analysis indicators
Each iteration of product function optimization is expected to have a positive impact on core indicators. This requires data and business to form an organic combination and promote each other's good situation. Good data quantification is not only a prerequisite for business growth, but also a starting point. With the deepening of growth theory, many products have set up their own North Star indicators as the key achievement indicators for measuring a strategic cycle of products. However, because the key indicators are too macro, they may not provide strong guidance for the formulation and implementation of business strategies.
Therefore, we need to disassemble the factors that can affect key indicators and map these factors to specific, achievable, and measurable behaviors, so as to ensure that the execution plan does not deviate from the general direction.
Here are the abbreviations of the three words OSM model (Objective, Strategy, Measurement) that we use most often in the product analysis framework , among them:
Objective (business objective): clarify what is the goal of business improvement
Strategy: What is the strategy that needs to be adopted in order to improve the goal
Measurement (evaluation index): Use data language as much as possible to describe whether the strategy has reached the improvement goal
Use the OSM model to sort out the data framework of your own products, and the differences in the data systems of different products are mainly reflected here. For example, the game industry may pay more attention to user retention rate; for e-commerce platforms, it may pay more attention to conversion rate.
Taking our own products as an example, Maple Page, as the landing page service of Tencent's e-commerce advertisements, provides advertisers with convenient landing page creation services, while also carrying the C-end user's order conversion process. We pay attention to the advertising consumption of advertisers on the maple page. Regardless of the impact of advertising pre-links, what really affects advertisers’ consumption at the landing page level is the conversion rate of C-end users on the landing page. The higher the conversion rate, the advertiser They are more willing to put more ads on the platform. Therefore, we will eventually improved its conversion rate as our core indicators, and this goal using the OSM model for dismantling.
Through the disassembly of the OSM model, we have determined a digital execution plan. The fluctuation of each evaluation index can be evaluated for its impact on the core index. Then, we can also know the specific effect of each functional iteration optimization. .
Data feedback: data comes from products and applies to products
After the effect evaluation, there is another important link, which is to promote the implementation of the analysis results. The analysis results that are not implemented are useless analysis.
As mentioned above, if the evaluation of the functional effect meets the expectations, we can promote the operation students to package and promote the function. However, if we find that the effect of the newly launched function is much lower than expected, at this time, is it just going to be done by synchronizing this effect? Regarding the two different situations mentioned here, let's take our newly launched functions as examples.
When the function is not up to the expected function, data analysis is the backing of the product students, and the data will feed back the product optimization iteration
In Moments native page ads, users need to go through two hops to reach the landing page of Maple Pages ( restricted by WeChat ads ). But the middle jump page will leave users more thinking time, resulting in users arriving at the landing page much lower than other traffic channels. Therefore, product students hope that through technical means, the outer material of the advertisement can be directly spliced with the landing page, so that users can click on the outer creative material to directly reach the landing page, thereby increasing the conversion rate. As shown:
But when the function went live, we found that the conversion rate of spliced ads was much lower than that of the market. As for why the effect is so different from the expected effect, it is the task of data analysis.
For the above possible reasons, we gradually carried out positioning analysis, and finally found that although native page splicing can increase the user's arrival rate, the information that users see on the first screen of the mobile terminal after splicing is extremely limited, and it is difficult to perceive that it is related to shopping. Related information (pictured above). In order to further verify the guess, we counted the difference between the average browsing time and the jump position of users in the original page spliced and non-spliced pages. In the end, it was found that, compared with non-spliced pages, the average page dwell time and jump position of users on native spliced pages were worse than non-spliced pages.
We have identified the reason for the poor effect. If we have a good idea on how to optimize at this time, we can directly talk to the product classmates about the strategy. But if there is no good plan for the optimization strategy, you can talk to the product classmates about the analysis results and their thinking about optimization. After all, product classmates are the ones most familiar with how to adjust product functions. When everyone has finalized the optimization plan, the next step is the implementation of the plan and the next round of effect evaluation~
When the functional effect reaches the expected, data analysis is the backing of the operation students to ensure that the operation promotion does not deviate
The landing page carries the user's order conversion. Before the "online payment" function is launched, only the "cash on delivery" payment method is supported. In order to better satisfy the payment habits of C-end users, we launched the "online payment" function.
After a long period of time after the full release of the function, we saw that the overall conversion rate of online payment orders has been better than the conversion rate of cash on delivery from the dimensions of order receipt/order generation, but the penetration rate of the entire function has not increased over time. The passage increases. New functions urgently need to be promoted by operating students. However, the question facing operating students is, in addition to which promotion method to choose, which businesses, categories, and groups should be promoted?
In order to better cooperate with the promotion of operating students, we give a data description of the usage of C-end and B-end users from the three dimensions of people, goods and fields. With these analyses, the operating students will be more confident in the choice of promotion crowd and goods.
One thing to note here is that every time an operating activity ends, the operating students will conduct a review of the results. At this time, students of data analysis should also pay attention to what is the specific effect of the content of operation promotion? Is there any significant difference between the data during the event and the data given before the event? The reason for the difference and so on.
Write at the end
Looking back on the work of product data analysis in the past six months, there is confusion and growth. As a senior data analyst said before, the highlight moment of a data analyst is when the strategy he proposed is adopted and produces good results. Thanks to the classmates who have been helping me in this process, together with outstanding people, I feel like a spring breeze.
Welcome to follow our video number: Tencent programmer
Latest video: Tencent "masked man"
Tencent Technology Official Exchange WeChat Group has been opened
Join the group and add WeChat: journeylife1900
(Remarks: Tencent Technology)