Pay attention to the "front-end and backward" WeChat official account, you will get a series of "intentionally original" high-quality technical articles, topics including but not limited to front-end, Node.js and server technology
EDITORIAL
different from commercial products, internal tools / platforms generally do not have a clear and direct business value, it is necessary to measure the effectiveness of their value through quantifiable indicators, this paper attempts to establish a framework of indicators data can be directly applied, so that the internal tool / The value of the platform can also be seen and explained
1. Analyze the core elements of production activities
From an object-oriented perspective, front-end engineering is the relationship and interaction between objects and objects
(From the front-end engineering system from an object-oriented perspective)
Among them, objects are divided into two types: subject objects and object objects:
Objects are an abstraction of various entities in front-end application production activities. Some of the objects are subjects (such as people in different roles), while others are objects (such as tools, platforms and other specific things). Interactive behavior to complete the development and delivery of front-end applications
People and tools are the core elements directly related to productivity:
The more powerful and intelligent the tool, the higher the efficiency of human operation and the lower the mental burden
PS Mind refers to the ways and habits of people to recognize things, which will affect how users perceive the world around them and how to take actions. It depends on the cognitive situation of the corresponding role, memory, channels and methods for active and passive education, and based on The role's usage habits of competing products, etc., for details, see the four-xiang model of experience measurement of tool products (1)
2. Identify the key goals
of the tool. For tools, taking into account efficiency and experience is the constant goal, but different tools may have different focuses, such as:
-
Low-level tools that do not directly face users: such as building modules, publishing modules, etc., efficiency is relatively important, and experience is second
-
The upper-level tools that users directly interact with: such as debuggers, release platforms, etc., pay more attention to experience, although efficiency is also important
-
On the other hand, tools are always born to solve problems, and choosing a tool is nothing more than 4 situations:
-
Irreplaceable: The only tool that can solve the problem of the goal, there is no choice, so no matter how experience or efficiency is, I have to use it
-
The best experience: the best experience among similar tools, accurately meets the needs, and there is no obvious difference in efficiency with other tools
-
The most efficient: the most efficient one of the same tools, quickly solve the problem, obviously much faster than other tools
-
The experience is not bad, and the efficiency is decent: a tool of the same kind that balances experience and efficiency, has no obvious shortcomings, can barely solve the problem, and is not very troublesome to use
-
Excluding the cases where there is no choice, tools with better experience are more popular when there is no obvious gap in efficiency, and tools that can open the obvious gap in efficiency will be very popular if they don’t experience faults. There is no doubt that
- However, it should be noted that if the best options in terms of experience and efficiency have obvious shortcomings, users are more inclined to choose an alternative tool that is not good, rather than enduring its shortcomings for a long time:
what. . Yes. . I just don't want to use xxx
3. Establishing a measurement model of effectiveness value
After determining the key objectives, the next question is how to quantify efficiency and experience so that they can be measured
The calculation formula for measuring efficiency and analog work efficiency:
Work efficiency = total work/work time
tool efficiency can be defined as:
Tool efficiency = problem scale/operation time The
problem scale is still not a quantifiable thing, and is further characterized as time cost:
Tool efficiency = time cost (required without the tool) / time cost (resolved with the tool)
Then, there are 3 situations:
-
The ratio is equal to 1: The tools are the same with or without tools, and the tools do not bring efficiency improvements
-
The ratio is less than 1: It's better not to use it, because it takes more time to use tools
- Ratio greater than 1: The tool is more efficient, the larger the value, the more obvious the efficiency improvement brought by the tool
Measuring experience is
not like efficiency, which can be calculated through a unified rule to get an accurate value, but a measurement model can also be established:
Experience refers to the degree of overlap between the product and the user's mind (the mind line in the figure above). The closer the function and performance of the tool are to the user's psychological expectations, the higher the experience evaluation, which is reflected in:
-
Ease of use: mapping from the user's mind to the product's functions, the ultimate ease of use is intuitive and ready to use
- Stability: Mapping from the user's mind to the product performance, the ultimate stability is complete trust, never doubt that the tool will go wrong
which is:
工具体验 = 易用程度 * 稳定程度
In other words, the tool experience is the product of ease of use and stability. As long as there are some shortcomings that are not easy to use or unstable, the experience will drop sharply.
Measure performance value
summary, the tools to bring efficiency value is reflected in two aspects:
效能价值 = 效率价值 * 体验因子
among them:
-
Efficiency value: Reduce the time cost for users to solve problems, allowing users to solve problems more quickly
- Experience factor: reduce the user’s mental burden and allow users to solve problems more easily and happily
The two complement each other, experience upgrades may improve efficiency, and efficiency improvements may also drive experience
Therefore, under the premise that the experience is guaranteed, efficiency can be simply used as a measure of performance value, and an accurate ratio can be used to quantify the performance value
4. Choosing the right data indicators
After the measurement model has been established, then the specific data indicators will be boxed in
Based on the above analysis, the time cost (when the experience is guaranteed) directly reflects the time cost saved by the tool, which is closely related to the number of users, frequency of use, and duration of use:
-
Number of users: cumulative number of users, daily/weekly/monthly UV, daily new users, daily/weekly/monthly active users (number of users who have operated core functions during the period)
-
Frequency of use: Daily/weekly/monthly PV, function utilization rate, core operation times, daily average use times
- Use time: core operation time
PS function usage rate = number of users using a certain function / total number of users, which can also be used to measure the contribution of different functions to the overall
E.g:
每天节省的时间成本 = 日用户量 * 日功能使用率 * (不用该工具解决所需的时间 - 操作时间)
= 100 * 35% * (1.5人日 - 0.8人日）
= 24.5人日
In addition, some other side data can also reflect the effectiveness value:
-
User distribution: number of target users, user *** rate, percentage of users of each attribute, user *** rate of each attribute
- Distribution of output results: quantity, importance, average time, proportion of output results of each attribute
PS user *** rate can be simply understood as user *** rate = number of existing users / number of target users
E.g:
覆盖2/3的目标用户，包括60%以上的一线开发人员、10%的测试人员
覆盖8大产品线，半年支持40多个项目，包括效果极好的xx重点项目
Ease of
use can also be measured by some numerical values:
-
User satisfaction: number/rate of user complaints and inquiries, sample survey satisfaction
-
Operation difficulty: the number of misoperations
- Mental burden: the number of words in the help document and the number of notes
In addition, a common requirement collection method used by product managers is to observe the actual operations of real users and record the frustrations encountered by users. Don’t interrupt or rush to provide help in the process. You can often find some usage problems accurately.
The degree of
stability The degree of stability can be reflected from abnormal indicators, such as:
-
crash rate
-
number of bugs
- Operation failure times
Among them, operation failure is a vague definition, including runtime errors, service interface errors, and no search results, etc. Stability issues can easily destroy the user experience and greatly reduce performance. For example, the tool always crashes and is almost useless. , There is no way to talk about effectiveness value
5. If you have data, use data to speak. The
reason for establishing quantifiable data indicators is to use data to speak, verify some previous assumptions, and provide guidance for the iteration and optimization of tools:
-
Is the new feature supported by users? What is the function usage rate? Is the promotion channel effective?
-
Is user operation smooth, and is there a big gap between the actual time spent and expected?
- What is the outcome, is the ROI high enough, and is it necessary to continue?
Do things with PM's mature methodology