Author: Dapeng
Cloud log team
Back-end development engineer
Small traditional monitoring range
High efficiency intelligent monitoring
After all how you use
Dapeng to give you Weapon
Traditional monitoring by monitoring item set a fixed value (threshold), when the monitoring indicators above this threshold will notify attention to this indicator items. Traditional monitoring business indicators generally applicable to a range of fluctuation:
For example, disk usage, CPU usage, etc., when the index exceeds a certain value it means that the system may be malfunctioning, but the face of fluctuations in a relatively large range of scenarios; for example a bank transaction of 09:00 ~ 18:00 of the trading volume, trading volume at other times may be zero, transactions are generally working, non-working day trading surge; such as traffic for a website to a great day, late at night traffic may be zero, if the use of traditional monitoring the above scene monitoring indicators, are often not well reflect the state of systems and services, generate a lot of false positives, increasing labor costs, and even make people numb to generate an alarm, distrust.
OMG, that we supposed to? !
So we joined the machine learning algorithms, monitoring indicators of past training, the abnormality judgment on the current value no longer depends solely on a fixed threshold, but earlier data, historical data, reference periodically by dynamic threshold method abnormality detection data.
Dapeng Auditorium
# # Seriously taking notes
Technology Architecture
Model trainer: Cloud log acquisition service at a fixed frequency of time series index is formed, conveyed to the model trainer, the trainer model mathematical model consisting of a series (which can be dynamically added), each model have been predictive value, and observations that existed prior to the prediction error, comparison error we will get a mathematical model that best matches the business. Using the best model out of this training, the future input point of time to obtain the predicted value, rendering future business map.
Abnormality detector: Mathematical model training predicted value and the actual observations there are some errors, the residual conveyed to a series abnormality detector, the abnormality detector is a mathematical model consisting of a series (which can be dynamically added), model model errors and outlier point service inspection will best match the model as the abnormality detection, the abnormality is detected subsequent to the point sends warning system.
Time series modeling
Time series data acquisition is not scattered, there is no law of a set of data, it tends to vary with a change in the business, and some highly cyclical rules, and some have a relatively smooth trend, we need to use the corresponding mathematical model to fit, is what we used several mathematical models.
Time series with different characteristics, different mathematical models to calculate the error is very different, we measure the degree of matching these mathematical models index from the list below.
After more than measure the merits of the prediction model, I get the best fit curve fitting business, get the best training model. Then enter a future time point to obtain the predicted value of that point in time, and then draw out the prediction curve.
abnormal detection
After the predicted future time points of data, how to detect whether or not the abnormal traffic data, we also have a corresponding abnormality detection model, as follows:
After computing the residual model using the above index, past service with outliers contrast, selects the closest anomaly detection model, as the subsequent detection of abnormality, when the model number data detected abnormality warning to immediately send inspection staff, preventing patients in the future.
Cloud log says heaven and earth
Really useful for log analysis
Monitoring Alarms everything OK
Stir-fried products to heaven
Dapeng auditorium, see you next time ~