Here I will share with you some of the knowledge I have summarized on the Internet, I hope it will be helpful to everyone
background
Just after the happy weekend weekend, when I came in the morning, I was suddenly notified by the operation that the online business was suspended, and the user could not place an order. Fuck me, I quickly entered the debug mode. After checking, it turned out that there was a problem with the data returned by the server. I quickly asked the server, and the boss replied that the business department configured the wrong package. Fortunately, we are not to blame, but we quickly wrote the review document, took the initiative to find our own responsibility, and took the big pot. We blamed our front-end for not doing a good job of front-end monitoring, which led to the online problem lasting for two days. I originally thought that the operation would say no to the refusal, and the blame was theirs, but unfortunately, they don't know much about the world, so the blame was placed on the technical department. Although, I still calm down and get out the front-end abnormal monitoring. Next time, I must not take the initiative to take the blame. I hope that friends who read this article will not take the blame casually ^_^
monitor
Because the point-buying process was done based on sentry before, the foundation has been laid, and it supports automatic point-buying, manual point-buying and data reporting. For related principles, please refer to the previous article How to Build a Data Platform from 0-1 (2) - Front-end Buried Points . The data reporting of this monitoring is also based on sentry.js. So how to design the whole process. Specific steps are as follows:
-
Monitoring data classification
-
Monitoring Data Definition
-
Monitoring data collection
-
Monitoring data reporting
-
Monitor data output
-
Monitoring data early warning
Data Classification
We mainly deal with front-end data errors, and general exceptions are divided into logic exceptions and code exceptions. Based on our project, because it involves revenue, we focus on logic errors in payment exceptions, and divide errors caused by other codes into a large category. Then the two major exceptions are subdivided, as follows:
-
Abnormal payment
1.1 Payment is successful
1.2 Payment failure
-
code exception
2.1 bindexception
2.1.1 js_error
2.1.2 img_error
2.1.3 audio_error
2.1.4 script_error
2.1.5 video_error
-
unhandleRejection
3.1 promise_unhandledrejection_error
3.2 ajax_error
-
vueException
-
peformanceInfo
data definition
Sentry-based reported data generally includes events and attributes. Here we define the payment exception event as "page_h5_pay_monitor" and the code exception event as "page_monitor". Then the attributes of the payment exception are roughly:
pay_time,
pay_orderid,
pay_result,
pay_amount,
pay_type,
pay_use_coupon,
pay_use_coupon_id,
pay_use_coupon_name,
pay_use_discount_amount,
pay_fail_reason,
pay_platment
// js_error
monitor_type,
monitor_message,
monitor_lineno,
monitor_colno,
monitor_error,
monitor_stack,
monitor_url
// src_error
monitor_type,
monitor_target_src,
monitor_url
// promise_error
monitor_type,
monitor_message,
monitor_stack,
monitor_url
// ajax_error
monitor_type,
monitor_ajax_method,
monitor_ajax_data,
monitor_ajax_params,
monitor_ajax_url,
monitor_ajax_headers,
monitor_url,
monitor_message,
monitor_ajax_code
// vue_error
monitor_type,
monitor_message,
monitor_stack,
monitor_hook,
monitor_url
// peformanceInfo 为数据添加 loading_time 属性，该属性通过entryTypes获取
try {
const observer = new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
if (entry.entryType === 'paint') {
sa.store.set('loading_time', entry.startTime)
}
}
})
observer.observe({ entryTypes: ['paint'] })
} catch (err) {
console.log(err)
}
data collection
Data collection is collected through event binding, the specific binding is as follows:
import {
BindErrorReporter,
VueErrorReporter,
UnhandledRejectionReporter
} from './report'
const Vue = require('vue')
// binderror绑定
const MonitorBinderror = () => {
window.addEventListener(
'error',
function(error) {
BindErrorReporter(error)
},true )
}
// unhandleRejection绑定 这里由于使用了axios，因此ajax_error也属于promise_error
const MonitorUnhandledRejection = () => {
window.addEventListener('unhandledrejection', function(error) {
if (error && error.reason) {
const { message, code, stack, isAxios, config } = error.reason
if (isAxios && config) {
// console.log(config)
const { data, params, headers, url, method } = config
UnhandledRejectionReporter({
isAjax: true,
data: JSON.stringify(data),
params: JSON.stringify(params),
headers: JSON.stringify(headers),
url,
method,
message: message || error.message,
code
})
} else {
UnhandledRejectionReporter({
isAjax: false,
message,
stack
})
}
}
})
}
// vueException绑定
const MonitorVueError = () => {
Vue.config.errorHandler = function(error, vm, info) {
const { message, stack } = error
VueErrorReporter({
message,
stack,
vuehook: info
})
}
}
// 输出绑定方法
export const MonitorException = () => {
try {
MonitorBinderror()
MonitorUnhandledRejection()
MonitorVueError()
} catch (error) {
console.log('monitor exception init error', error)
}
}
Data reporting
Data reporting is based on sentry, as follows:
/*
* 异常监控库 基于sentry jssdk
* 监控类别：
* 1、window onerror 监控未定义属性使用 js资源加载失败问题
* 2、window addListener error 监控未定义属性使用 图片资源加载失败问题
* 3、unhandledrejection 监听promise对象未catch的错误
* 4、vue.errorHandler 监听vue脚本错误
* 5、自定义错误 包括接口错误 或其他diy错误
* 上报事件： page_monitor
*/
// 错误类别常量
const ERROR_TYPE = {
JS_ERROR: 'js_error',
IMG_ERROR: 'img_error',
AUDIO_ERROR: 'audio_error',
SCRIPT_ERROR: 'script_error',
VIDEO_ERROR: 'video_error',
VUE_ERROR: 'vue_error',
PROMISE_ERROR: 'promise_unhandledrejection_error',
AJAX_ERROR: 'ajax_error'
}
const MONITOR_NAME = 'page_monitor'
const PAY_MONITOR_NAME = 'page_h5_pay_monitor'
const MEMBER_PAY_MONITOR_NAME = 'page_member_pay_monitor'
export const BindErrorReporter = function(error) {
if (error) {
if (error.error) {
const { colno, lineno } = error
const { message, stack } = error.error
// 过滤
// 客户端会有调用calljs的场景 可能有一些未知的calljs
if (message && message.toLowerCase().indexOf('calljs') !== -1) {
return
}
sa.track(MONITOR_NAME, {
//属性
})
} else if (error.target) {
const type = error.target.nodeName.toLowerCase()
const monitorType = type + '_error'
const src = error.target.src
sa.track(MONITOR_NAME, {
//属性
})
}
}
}
export const UnhandledRejectionReporter = function({
isAjax = false,
method,
data,
params,
url,
headers,
message,
stack,
code
}) {
if (!isAjax) {
// 过滤一些特殊的场景
// 1、自动播放触发问题
if (message && message.toLowerCase().indexOf('user gesture') !== -1) {
return
}
sa.track(MONITOR_NAME, {
//属性
})
} else {
sa.track(MONITOR_NAME, {
//属性
})
}
}
export const VueErrorReporter = function({ message, stack, vuehook }) {
sa.track(MONITOR_NAME, {
//属性
})
}
export const H5PayErrorReport = ({
isSuccess = true,
amount = 0,
type = -1,
couponId = -1,
couponName = '',
discountAmount = 0,
reason = '',
orderid = 0,
}) => {
// 事件名：page_member_pay_monitor
sa.track(PAY_MONITOR_NAME, {
//属性
})
}
The above is reported through sentry's sa.track, and the details will not be expanded.
Output and warning
The data is reported to the big data platform and stored in hdfs, and then we directly do scheduled tasks to read hdfs for certain filtering and output it to the DingTalk group through DingTalk webhook. In addition, if necessary, data backup can be done through hdfs to the data warehouse Then go to kylin for storage.
Summarize
Data monitoring is necessary for large platforms, especially those involving revenue. We must take this into consideration when designing projects. It is best to convince the server to provide corresponding code monitoring. It is best to have a layer on the ngnix layer or the cloud. Serious abnormalities can call you directly, and the current cloud platform has corresponding support. In this way, abnormalities can be detected in time, and the pot can be thrown out accurately after receiving it in the hand.
This article is reproduced in:
https://juejin.cn/post/7244363578429030459
If it is helpful to you, you are welcome to pay attention. I will update the technical documents regularly, and we will discuss and learn together and make progress together.
