[Numerical Calculation Method] Error

Table of contents

introduction

1. Error source

1. Model error

2. Observation error (measurement error)

3. Method error Truncation error

4. Rounding error (roundin off error)

2. Error classification

1. Absolute error

2. Relative error

3. Errors in machine learning

---

        When numerical methods are used to solve mathematical problems on the computer, errors will inevitably occur, and the approximate solutions of the problems are generally obtained. Therefore, error analysis and error estimation have become an important content in the study of numerical calculation methods.

introduction

1. Error source

1. Model error

        Model error refers to the error caused by the limitation or imperfection of the model itself. Model errors may originate from factors such as model assumptions, parameter selection, and insufficient training data. Methods to reduce model error include improving the model structure, increasing the amount of training data, optimizing model parameters, etc.

2. Observation error (measurement error)

        Observation error refers to the difference between the observed value and the real value due to the influence of various factors during the actual observation or measurement process. Observation errors may be caused by instrument precision, environmental conditions, human factors, etc. In order to reduce the observation error, the accuracy of the measurement can be improved by taking the average of multiple observations, improving the accuracy of the instrument, and controlling the environmental conditions.

3. Method error   truncation error

        Method error refers to the error introduced due to the limitations of the method or algorithm used. Method errors may arise due to approximate calculations, assumption violations, numerical stability, etc. Choosing appropriate methods and algorithms, and understanding their limitations and assumptions, can reduce methodological error.

        Truncation error is the error introduced by truncating numbers during calculations. When approximating a number with infinite decimal places, we need to truncate or round the number to fit the computer's finite number of digits. Such an approximation introduces truncation errors. Truncation errors can cause discrepancies between the results and true values. It arises from the loss of part of the original data during calculation. Truncation error generally increases as fewer bits are used for representation or computation.

        For example, to compute the value of a function , we use the finite Taylor expansion

         

Approximate replacement , the method error (also known as truncation error) at this time is:

        

4. Rounding error (roundin off error)

        Rounding error is the error introduced by rounding or truncating floating-point numbers during calculations. Because computers use a finite number of digits to represent numbers, for some calculations, round-off errors can accumulate and affect the precision of the final result. In numerical calculation, it is necessary to pay attention to the influence of rounding errors on calculation results, and take appropriate methods to control and adjust errors.

        Within the allowable range of error...

2. Error classification

1. Absolute error

        Absolute error refers to the absolute value of the difference between the approximate value and the true value. It expresses the size of the difference between the approximate value and the true value, regardless of the proportional relationship of this difference in the overall range. Absolute error is often used to measure the absolute degree of error to determine the accuracy of an approximation. The formula for calculating the absolute error is: absolute error = |approximate value - true value|.

       

where, represents the exact value

2. Relative error

        Relative error refers to the ratio between the absolute error and the true value. Relative error takes into account the proportional relationship between the difference between the approximate value and the real value relative to the real value, so it is more suitable for comparing errors of different magnitudes. Relative error is often used to measure the relative degree of error relative to the true value. The formula for calculating the relative error is: relative error = (absolute error / true value) × 100%.

          

3. Errors in machine learning

        In machine learning, error is often used to evaluate the performance of a model and how accurate its predictions are. The following are several common error metrics used in machine learning:

1. Mean Absolute Error (Mean Absolute Error, MAE): MAE is the average of the absolute error between the predicted value and the true value. It measures the average degree of difference between the predicted value and the real value, which is robust and will not be affected by outliers.

2. Mean Squared Error (Mean Squared Error, MSE): MSE is the average value of the square of the error between the predicted value and the true value. Compared with MAE, MSE gives higher penalties for large errors and is therefore more sensitive to outliers.

3. Root Mean Squared Error (RMSE): RMSE is the square root of MSE, which is consistent with the original data units. RMSE is easier to interpret when evaluating model performance because it is dimensionally consistent with the original data.

4. R-squared: R-squared is used to measure how well the model fits the observed data. It represents the proportion of the variance explained by the model to the total variance, and its value ranges from 0 to 1. The closer the R square is to 1, the better the model fits the observed data.

5. Log Loss (Log Loss): Log loss is usually used in binary or multi-classification problems to measure the difference between the predicted probability of the model and the true label. The smaller the log loss, the closer the prediction result of the model is to the real situation.

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score, log_loss

# 示例数据
y_true = np.array([3, 4, 2, 1, 6])
y_pred = np.array([2.5, 4.3, 1.8, 0.9, 5.8])
y_prob = np.array([0.1, 0.8, 0.3, 0.2, 0.9])  # 用于对数损失的预测概率

# 平均绝对误差（MAE）
mae = mean_absolute_error(y_true, y_pred)
print("MAE:", mae)

# 均方误差（MSE）
mse = mean_squared_error(y_true, y_pred)
print("MSE:", mse)

# 均方根误差（RMSE）
rmse = np.sqrt(mse)
print("RMSE:", rmse)

# R平方（R-squared）
r2 = r2_score(y_true, y_pred)
print("R-squared:", r2)

# 对数损失（Log Loss）
logloss = log_loss(y_true, y_prob)
print("Log Loss:", logloss)