Front-end data buried point SDK
foreword
I believe that many people have not taken the time to understand it because they have not been exposed to the content related to data embedding in the project. They always feel that this is another aspect that they cannot touch. It is difficult to do well. This article will start from the two core aspects of understanding the data burying SDK and designing the front-end data burying SDK , and talk about the things about the front-end data burying.
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-E11w6S5n-1669194659305)(https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp /9ce5ec3fac27465b82974a5c0b777e8b~tplv-k3u1fbpfcp-zoom-1.image)]
Know the data buried point SDK
The full name of SDK is Software Development Kit , which is a software development kit. It is generally a collection of development tools used by software engineers to build application software for specific software packages, software frameworks, hardware platforms, and operating systems.
Why do you need front-end data burying?
For the product itself, we need to focus on the following aspects:
- What the user mainly does in the product, how long they stay, and how many times they visit
- What is the percentage of user clicks? Will there be some functional design that is invalid for users?
- Whether the user's core usage process is smooth, and whether the page feedback is normal and friendly
- What potential users may need to update the functionality
In general, the core of data burying is to collect data (with data, you can do whatever you want), only by analyzing the data can you better evaluate the quality and importance of the entire project (data is king), and can contribute to the product Optimization shows the direction ( data-driven product ).
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-F8KDafOP-1669194659309)(https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp /7015c657ea1f4e49a12c3e397765ae5b~tplv-k3u1fbpfcp-zoom-1.image)]
What aspects should be considered when burying front-end data?
The core of data burying is data collection, and the content related to data is nothing more than the following:
- The data is generated based on the application, because there will be no relevant data without the application
- The application itself needs to provide display, collection, and operation content, which is based on the platform. For example, the website is based on the browser platform.
- There must be users if there are applications and platforms, because the application itself is to provide users with useful functions to solve certain existing problems
- For developers, the application is the code, and the quality of the code running can also determine the quality of the application, and the explicit quality is reflected in errors or warnings
To sum up, data burying actually needs to consider three core aspects: user behavior, error warning, and page performance.
user behavior
User behavior is a series of operations performed in web applications, but there are many kinds of user operations, and it is impossible to record them all. Generally, the following behaviors of users need to be recorded:
- Number of pages viewed by users, PV (Page View)
-
- Each time a user visits a page in the website, it is recorded
1asPV, and if the user visits the same page multiple times, the number of visits will be accumulated
- Each time a user visits a page in the website, it is recorded
- Number of page viewers, UV (Unique visitor)
-
- A user who normally accesses the page through the network, usually a computer client or a user account is a visitor. Generally, multiple visits
24hwithin1situationUV.
- A user who normally accesses the page through the network, usually a computer client or a user account is a visitor. Generally, multiple visits
- User clicks on the button
-
- The above two can be considered as yes
自动式触发埋点, and the number of clicks on the button belongs to yes互动式触发埋点, which is convenient to understand the usage of this function button
- The above two can be considered as yes
error warning
Errors generated by the code running on the page may interrupt the user's core operation process. In order to prevent a large number of users from being affected, we need to obtain the error data of the production environment , so that developers can make timely repairs.
Generally speaking, errors in the code will include the following categories:
- Global errors , that is, errors that are not caught
- Local errors , i.e. errors caught by
try...catch、promise.then、promise.catchetc. - Interface request error , that is, an error when making a request and receiving a response
APIin - Component-level errors , i.e. errors that occur when using
Vue/Reactthe component
page performance
Page performance is actually a point that needs to be considered and optimized in front-end performance optimization. After all, if a website always has problems such as white screen, interactive freeze, and long loading time of page resources, it will definitely be impossible to retain users, especially users. The real environments are different, such as Windows x、MACOS、Android、iOSand so on , it is more necessary to count and collect relevant data, so as to facilitate centralized optimization processing and improve user experience.
The content related to page performance indicators was mentioned in the previous How to do front-end performance optimization (Part 1) - Straight to the point [1], here is a general summary:
- first draw(
First Paint，FP) -
- After the rendering process confirms that it wants to render the current response resource, the rendering process will first create a blank page, which is usually referred to as the time point of creating a blank page, referred to
First PaintasFP - The so-called white screen time actually refers to the time from creating this blank page to the time when the browser starts rendering non-blank content, such as the page background changes, etc.
- After the rendering process confirms that it wants to render the current response resource, the rendering process will first create a blank page, which is usually referred to as the time point of creating a blank page, referred to
- First ContentPaint(
First Contentful Paint，FCP) -
- When the user sees some "content" elements are drawn on the page, it is different from the white screen, it can be
文本first drawn, orSVGfirst appeared, orCanvasfirst drawn, etc., that is, when the first pixel is drawn on the page , this time point is calledFirst Content Paint, referred to asFCP
- When the user sees some "content" elements are drawn on the page, it is different from the white screen, it can be
- First Screen Time/Maximum Content Draw(
Largest Contentful Paint, LCP) -
LCPIt is a new performance metric,LCPfocusing on the performance metric of user experience. Compared with the existing metrics, it is easier to understand and reason. When the content of the first screen is completely drawn, this time point is called, referred toLargest Content PaintasLCP- Maximum content drawing should be done
2.5swithin
- first input delay (
First Input Delay, FID) -
FIDMeasures the time from when the user interacts with the page for the first time ( clicks a link , clicks a button , or a custom basedjsevent ) to when the browser actually starts processing the event- The first input delay should be done
100mswithin
- cumulative layout offset (
Cumulative Layout Shift, CLS) -
CLSis to measure visual stability in order to provide a good user experience- The cumulative layout offset should be kept at
0.1or less
- first byte arrival time (
Time to First Byte，TTFB) -
- It refers to the time when the browser starts to receive the response data from the server ( background processing time + redirection time ), which is an important indicator reflecting the response speed of the server
TTFBIf the time exceeds500ms, the user will feel obvious waiting when opening the webpage
After understanding why we need to do front-end data burying and all aspects of statistical data required for front-end data burying, we need to design our own front-end data burying SDK .
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-EUTtBB4x-1669194659310)(https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp /93b963e7bc0a4ac98d3102a4639c0e53~tplv-k3u1fbpfcp-zoom-1.image)]
Design front-end data buried point SDK
Here we only consider the core content of the data embedding point, so it will not be so comprehensive that it will not be involved, and it is impossible to design it comprehensively at the beginning. As long as the core functions are guaranteed, then it is enough to expand based on the core.
Determine options and data content
The unique identifier of the application — options.AppId
As a general tool set, the Data Buried SDK can be used by multiple systems, which means that it is necessary to ensure the uniqueness of each application. Generally speaking, when initializing the SDK, the access party needs to The provided ID of the current application .
Where does this ID come from? Generate it casually? Generally, the following steps are required:
- Generate a unique AppId for the current application on the corresponding monitoring system
- Passed in as one of the configuration items when the corresponding application accesses the SDK
In fact, it also involves request urlcontent , which is mainly used to send to the corresponding monitoring system, so optionsthe core content is simply designed as follows:
{
appId: '', // 当前应用唯一标识
baseUrl: '', // 数据发送的地址
}
复制代码
Data sending format—data
Since there are many types of data that need to be collected, it is best to define a more general data format to facilitate more friendly data collection.
Here is a brief definition of the data format, which is roughly as follows, and the format varies with the demand scenario:
{
appId: '', // 当前应用唯一标识
type: 'action' | 'performance'| 'network' | 'error', // 不同数据类型
pageUrl: '', // 页面地址
apiUrl: '', // 接口地址
userId: '', // 当前用户 id
userName: '', // 当前用户 name
time: '',// 触发记录的时间
data: {
}, // 接口响应结果 | 性能指标 | 错误对象 | 用户操作相关信息
}
复制代码
Determine how data is sent
If you want to ask what the most basic function of the front-end buried point is, it must be the ability to send data. Otherwise, even if there are applications, users, and data, they can only be stored locally and cannot be sent to the corresponding monitoring system. It is impossible to collect and count (data is equal to free).
So what are the ways to send data? For this problem , it is much easier to translate data sending into request sending, and the question becomes what are the methods of request sending?
Generally include the following (including but not limited to):
XMLHttpRequestfetchformformaction- request based on element
srcattribute -
imglabeledsrcscriptlabeledsrc
Navigator.sendBeacon()
The last one is chosen here, because Navigator.sendBeacon()it is specially used to send statistical data asynchronously [3] to the server through HTTP POST [2], and at the same time, it can avoid some problems of traditional technology sending analysis data.Web
Some problems of sending statistical data by traditional technology can
传送门be viewed directly through [4], and no additional explanation is given due to the limited space of the article.
SDK core code
Here we only consider the minimalist situation, the content of the designed SDK code is relatively simple, directly upload the code:
let SDK = null // EasyAgentSDK 实例对象
const QUEUE = [] // 任务队列
cosnt NOOP = (v) => v
// 通过 web-vitals 页面性能指标
const reportWebVitals = (onPerfEntry) => {
if (onPerfEntry && onPerfEntry instanceof Function) {
import('web-vitals').then(({
getCLS, getFID, getFCP, getLCP, getTTFB }) => {
getCLS(onPerfEntry) // 布局偏移量
getFID(onPerfEntry) // 首次输入延迟时间
getFCP(onPerfEntry) // 首次内容渲染时间
getLCP(onPerfEntry) // 首次最大内容渲染时间
getTTFB(onPerfEntry) // 首个字节到达时间
})
}
}
export default class EasyAgentSDK {
appId = ''
baseUrl = ''
timeOnPage = 0
config = {
}
onPageShow = null
onPagesHide = null
constructor(options = {
}) {
if (SDK) return
SDK = this
this.appId = options.appId
this.baseUrl = options.baseUrl || window.location.origin
this.onPageShow = options.onPageShow || NOOP
this.onPagesHide = options.onPagesHide || NOOP
// 初始化监听页面变化
this.listenPage()
}
// 设置 config
setConfig(congfig){
this.config = congfig
}
// 刷新任务队列
flushQueue() {
Promise.resolve().then(() => {
QUEUE.forEach((fn) => fn())
QUEUE.length = 0;
})
}
// 监听页面变化
listenPage() {
let pageShowTime = 0
window.addEventListener('pageshow', () => {
pageShowTime = performance.now()
// 页面性能指标上报
reportWebVitals((data) => {
this.performanceReport({
data })
})
// 执行 onPageShow
this.onPageShow();
})
window.addEventListener('pagehide', () => {
// 记录用户在页面停留时间
this.timeOnPage = performance.now() - pageShowTime
// 刷新队列前执行 onPageShow
this.onPageShow();
// 刷新任务队列
this.flushQueue()
})
}
// Json 转 FormData
json2FormData(data){
const formData = new FormData()
Object.keys(data).forEach(key => {
formData.append(key, data[key])
});
return formData
}
// 自定义上报类型
report(config) {
QUEUE.push(() => {
const formData = json2FormData({
...this.config,
...config,
time: new Date().toLocaleString(),
appId: this.appId,
pageUrl: window.location.href,
});
navigator.sendBeacon(`${
this.baseUrl}${
config.url || ''}`, formData)
})
}
// 用户行为上报
actionReport(config) {
this.report({
...config,
type: 'action',
})
}
// 网络状况上报
networkReport(config) {
this.report({
...config,
type: 'network',
})
}
// 页面性能指标上报
performanceReport(config) {
this.report({
...config,
type: 'performance',
})
}
// 错误警告上报
errorReport(config) {
this.report({
...config,
type: 'error',
})
}
}
复制代码
Report user behavior
Statistical PV and UV - automatically trigger buried points
The PV and UV have been introduced above. In essence, these two data statistics can be obtained in a report type of actiondata transmission. It mainly depends on the rules that the monitoring system uses to analyze and count the data. Here it is in SDKthe internal pageshow / pagehideTwo :
- Data related to PV/UV
pageshowand page performance can be reported in
window.SDK = new EasyAgentSDK({
appId: 'application_id',
baseUrl: '//aegis.example.com/collect',
onPageShow() {
window.SDK.actionReport({
data: {
} // 其他必要传递的信息
})
}
});
window.SDK.setConfig({
userId: UserInfo.userId, // 当前用户 id
userName: UserInfo.userName, // 当前用户 name
});
复制代码
```
- 在 `pagehide` 中主要用于计算用户停留在页面上的时间 `timeOnPage` 和 刷新任务队列
### 统计用户点击按钮 — 交互式触发埋点
假设我们希望记录某些按钮的使用次数的数据，可以在 `document` 上监听 `click` 事件，目的利用事件冒泡以便于不需要侵入不同按钮的 `click` 事件，比如：
```js
const TargetElementFilter = ['export_btn']
const findTarget = (filters) => {
return filters.find((filter) => TargetElementFilter.find((v) => filter === v)));
}
document.addEventListener('click', (e) => {
const {
id, className, outerHTML } = e.target
const isTarget = findTarget([id, className])
if (isTarget) {
SDK.actionReport({
data: {
id,
className,
outerHTML
}, // 其他必要传递的信息
})
}
})
复制代码
Report page performance
The content related to page performance belongs to SDKAutomatically trigger buried points, and users should not be allowed to manually access. In the above implementation, we report data pageshowthrough reportWebVitalsand performanceReport, and here we Googlechoose web-vitals[5] launched by to obtain The specific data related to page performance indicators, the corresponding code is:
// 通过 web-vitals 页面性能指标
const reportWebVitals = (onPerfEntry) => {
if (onPerfEntry && onPerfEntry instanceof Function) {
import('web-vitals').then(({
getCLS, getFID, getFCP, getLCP, getTTFB }) => {
getCLS(onPerfEntry) // 布局偏移量
getFID(onPerfEntry) // 首次输入延迟时间
getFCP(onPerfEntry) // 首次内容渲染时间
getLCP(onPerfEntry) // 首次最大内容渲染时间
getTTFB(onPerfEntry) // 首个字节到达时间
})
}
}
复制代码
The obtained data is roughly as follows:
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-uRmZzvFe-1669194659311)(https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp /4e02462554cf445e8b0ddc072e022509~tplv-k3u1fbpfcp-zoom-1.image)]
Report an error warning
global error
Global errors, that is, uncaught errors, can be captured through window.onerrorevents , and then report error data, as follows:
window.addEventListener('error', (reason) => {
const {
filename, message, error } = reason;
window.SDK.errorReport({
data: {
filename,
message,
error
}
});
})
复制代码
local error
Local errors, i.e. errors caught by try...catch、promise.then、promise.catchetc. , are roughly used as follows:
try {
throw new Error('error for test')
} catch(error) {
window.SDK.errorReport({
data: {
error,
},
})
}
Promise.reject(new Error('Promise reject for test'))
.then(
() => {
},
(reason) => {
window.SDK.errorReport({
data: {
error: reason
}
});
},
)
Promise.reject(new Error('Promise reject for test'))
.catch(
(reason) => {
window.SDK.errorReport({
data: {
error: reason
}
});
},
)
复制代码
Interface request error
Interface request error, that is, the error when making a request and receiving a response APIin . For the convenience of the example here, we can report the corresponding error in the second callback parameter of axiosits request interception and response interception The data information is roughly as follows:
// 创建axios实例
const service = axios.create({
baseURL, // api 的 base_url
timeout: 60000, // 请求超时时间
responseType: reqConf.responseType,
});
// 请求拦截
service.interceptors.request.use(
(config) => {
...
return config;
},
(error) => {
window.SDK.errorReport({
apiUrl: config.url,
data: {
error,
},
})
},
);
// 响应拦截
service.interceptors.response.use(
(config: any) => {
...
return config;
},
(error: any) => {
window.SDK.errorReport({
apiUrl: config.url,
data: {
error,
},
})
return error.response.data;
},
);
复制代码
Component level errors
Component-level errors, that is, errors that occur when using Vue / Reactframework components, can be captured and reported using the error capture methods mentioned in their official documents.
-
Vue[6] inerrorHandleris used to specify a global handler for uncaught errors thrown within the application:// App.vue onMounted(()=>{ throw new Error('error in onMounted') }); // main.ts const app = createApp(App) app.config.errorHandler = (error, instance, info) => { window.SDK.errorReport({ data: { instance, info, error } }); } 复制代码 -
ReactErrorBoundary[7] Error boundary relatedgetDerivedStateFromErrorandcomponentDidCatchhooks in// 定义错误边界组件 class ErrorBoundary extends React.Component { constructor(props) { super(props); this.state = { hasError: false }; } static getDerivedStateFromError(error) { // 更新 state 使下一次渲染能够显示降级后的 UI return { hasError: true }; } componentDidCatch(error, info) { // 可以将错误日志上报给服务器 window.SDK.errorReport({ data: { info, error } }); } render() { if (this.state.hasError) { // 自定义降级后的 UI 并渲染 、 return <h1>Something went wrong.</h1>; } return this.props.children; } } // 使用错误边界组件 <ErrorBoundary> <MyWidget /> </ErrorBoundary> 复制代码
at last
Now we understand the two or three things about the front-end data embedding SDK. Through the above example, you may think it looks relatively simple, but it is not so easy to really do a good job of data embedding. For example, you need to consider your SDK data The time of sending, the number of times of sending, whether it is necessary to integrate certain data information and send it only once, how to avoid network congestion, and so on.