I have shared before: Growth is the best way for data analysts to contribute. Today, let’s take an example to see how to design growth experiments through data. Not much to say, whole!
Problem scenario:
A FMCG company with multiple series of products hopes to launch a new beverage (2 SKUs) to drive the overall sales amount. This model is newly launched and lacks experience, so it is planned to conduct experiments this year to observe the effect and promote it on a large scale.
Question: How to design a growth experiment to find problems in advance and ensure growth?
1. Fake data growth
Many newcomers raised their hands and said that I would:
1. Get all the information from the joint
bar , Tencent, and Ali big data. 2. Establish the user to the store-shelf-select-add to the shopping basket-checkout conversion funnel
3. Perform ABtest, and the user entering the store will automatically code and split the stream for AA\AB comparison
4. Establish user portraits to accurately identify the target user’s gender, age, income, hobbies
5. Establish an artificial intelligence big data model to accurately predict natural sales
The real problem is: no data. Because it is not our own channel, we can only get the number of purchases. Don't think about other data. If it does not exist, there is no one. Whether the store has any goods, you can rely on the inspection of the store to check regularly.
Then what should be done?
2. The most basic growth model
The simplest idea: The new product is to drive sales, so after the new product, you have to order more channels than before the new product. So the simplest model came out (as shown below):
So, it seems that the experimental design is also very simple:
1. Find a few stores
2. Shop the goods
3. Observe the sales after the shop
4. Make a deal
Is it really messed up?
3. Consider the basis of growth
The first question: Is it to find a store randomly or to find a target?
It is very likely that some stores are born to sell well, and some stores are born to sell poorly. If you don't analyze the store's past ordering situation in advance, it is likely to overestimate/underestimate the growth capacity. Pay special attention to the existence of full-time stores. If there are too many stores of this type, it may affect the overall judgment (as shown in the figure below):
When selecting pilot sample stores in the early stage, do a good selection in advance and consider:
Store location: Community store/CBD store/Pedestrian street store
Store performance: Overall performance is good/medium/poor
Category performance: Beverage category is good/medium/poor
Store time: new store/old store
These data can be obtained. Data 1 is recorded in the supervisor's patrol table, and data 2, 3, and 4 are recorded in the order form, so they are completely available. What needs to be done is to analyze the data in advance and do a good job of layering and labeling.
The intersection of so many dimensions raises a new question: how many stores should be selected for pilot projects. Statistics will tell you that the smallest sample of a single population is 30, and the best is 384, so that the sampling error is 5% under 95% confidence-but these have little to do with the problem at hand.
Because the immediate problem is:
First: It needs to be drawn on a store basis. It is possible that all stores may not add up to that much.
Second: The test is a new product, and the test cycle may be very long, which means that the supply may be insufficient.
Third: The test is a new product on the shelf. The business side needs to distribute the goods one store at a time, and the workload must be considered.
Therefore, when designing the number of samples, first have an estimate of the sales volume of a single store during the test period to ensure that it is in stock, so that it can be truly measured: whether it meets the expectation. After deciding on the total number of stores, fill in samples according to the above considerations. The final result is to ensure that each category has as many samples as possible.
If there are first-level, second-level, and third-level stores in advance, it will be much easier. Because of the first, second and third level classification, it is likely that factors such as sales capacity, store size, etc. have been comprehensively considered. But there are a few issues to pay attention to before using:
1. Whether the previous one, two and three classifications are still accurate. Don't have the situation of level 3 "level 2" level 1; it will be embarrassing to analyze afterwards.
2. Whether to consider the type in the first, second and third level. Avoid first-level stores that are all the same type of stores (for example, they are all CBD stores). After the evaluation, you will find that there is a serious lack of samples of other stores.
3. Whether the first, second and third levels are related to beverage sales. Note that the subject company is a full range of companies. It is very likely that the first, second and third levels are classified according to the overall performance. Corresponding to the beverage category, there will be an extreme situation of level 3> level 2> level 1.
As long as the above problems do not exist, use the 1, 2, and 3 classifications directly.
Considering the basis of growth not only makes the design more plump, but also greatly facilitates the evaluation afterwards. Avoid embarrassing questions such as:
1. Why the test results are not good, because all the shops are very poor
2. Why can't analyze the promotion potential, because all the stores of the same category are found
3. Why the first-level stores do not sell well, because Ya is born to sell badly
And in the post-analysis, it can conduct in-depth analysis of the various types of store labels, specific to the effect of each type of store label. In this way, there are clearer directions when iterating experiments, and more ideas when landing (as shown below).
So, considering this step is enough?
Fourth, consider the growth cycle
Step 2: Consider when and how long to test.
General merchandise has its own sales cycle, and beverages have a more special cycle, which may be concentrated in the summer, may also be affected by the climate of various places, or may be affected by the weather in the short term. Therefore, when designing the test cycle, you need to sort out the trend of beverages corresponding to similar prices, similar types, and similar target groups, so that you can have a global judgment (as shown in the figure below).
With the overall judgment, a relatively long observation period can be set to cover as many scenes as possible. In this way, various situations can be analyzed when evaluating and analyzing after the fact (as shown in the figure below).
Fourth, consider the implementation of growth
The third step: consider business landing actions.
When a new product is on the market, the three-piece set of publicity, distribution, and promotion is often greeted together. These landing actions are the final factors that determine the test results. These actions are all dependent on the execution of local branches/offices, and execution is crucial.
There is a very profound problem here: once the test results are not good...
- Is there no demand for the product itself?
- Or did the business fail to do well?
- Or is the data analyst wrong?
After monitoring the business execution process, you are qualified to say: The business is doing well/not well. Without monitoring the business execution process, people can say at any time: data analysis has not worked out. "Isn’t it all about artificial intelligence big data now? It must be that our data analysts are too stupid. The data analysts who hire a heady data analyst will definitely be able to figure it out clearly"-this pot is already prepared for you, so you must master it clear.
The information to be mastered includes:
- Distribution start time
- Time of completion
- Supplementary order time
With this information, you can do more analysis in conjunction with the order data:
- Has it been delayed for a long time?
- Has it started to move slowly?
- Is there any shop with eyes closed regardless of scale?
- There is no shortage
Of course, follow-up inspections must be followed, and several key dimensions can be added during inspections, such as:
- In summer, store the goods on the shelf instead of stuffing the freezer
- Hypermarkets don’t make piles, only shelves
- There are promotional materials that do not go to the store
These verification data must also be retrieved from the supervisor and analyzed together with the data to make it easier to see the results.
In this way, when interpreting the results, you naturally have more confidence: whoever has not been implemented is in place, you are not allowed to throw the pot to the product/data, and reflect on yourself if you have not done it well. This is also more conducive to finding answers to real questions.
V. Summary
The biggest problem in the field of data analysis in 2020 is that the learning process is book-based and divorced from reality. Books, teachers, and courses that teach data analysis, in order to make algorithms, statistical principles, user portraits, funnel models, and ABtest work, select some data sets with rich fields and clean clean data sets to make the algorithm run. Newcomers treat their work as reading books, and just run a few data sets and they are happy to think they have done it.
When the two intersect, the result is when newcomers encounter practical problems: either they are fantasizing that there is a panacea for their heads, or they are eager to move books to find the answer, or they rush to various groups to ask: "Are there any big players in the Internet beverage industry? , Urgent, online, etc., payable!". Only the ability of specific problems and specific analysis was lost.
To get rid of superstitions, be down-to-earth, carefully study business processes, and design reasonable methods is the way to solve the problem. Data is simple and data is simple, and data is rich. Data is rich. Simple data can be enriched through business process improvement. The combination of these three is the ability of a qualified data analyst. This article is about simple methods. If you are interested, please pay attention to the public account [Grounding Qi School]. In the next article, we will share a case of how to analyze a scenario with a lot of data. Let’s talk about UGC products. Stay tuned. .
Author: Chen grounded gas, micro-channel public number: down to earth school. A data analyst with ten years of experience has launched a series of data analysis courses and has more than 20,000 students.