Simple and crude understanding and implementation of machine learning linear regression (X): Improved linear regression - ridge regression, Boston prices forecast

Linear Regression

learning target

• Ownership in the implementation process of linear regression
• Application LinearRegression or SGDRegressor achieve regression prediction
• We know the assessment criteria and formulas regression algorithm
• Overfitting know the causes and solutions underfitting
• We know principle ridge regression and linear regression differences
• Application Ridge achieve regression prediction
• Application joblib achieve saving and loading models

Improvements 2.10 linear regression - ridge regression

1 API

• sklearn.linear_model.Ridge(alpha=1.0, fit_intercept=True,solver=“auto”, normalize=False)
  • It has l2 regularization of linear regression
  • alpha: regularization efforts, also called λ
    • λ Value: 0 ~ 11 ~ 10
  • solver: optimization method will automatically select the data
    • sag: If the data set, wherein relatively large, select the stochastic gradient descent optimization
  • normalize: whether the data is normalized
    • normalize = False: preprocessing.StandardScaler standardized data can be called before the fit
  • Ridge.coef_: regression weights
  • Ridge.intercept_: Return bias

Ridge method is equivalent to SGDRegressor (penalty = 'l2', loss = "squared_loss"), but SGDRegressor achieve a common stochastic gradient descent learning, recommended Ridge (realized SAG)

• sklearn.linear_model.RidgeCV(_BaseRidgeCV, RegressorMixin)
  • L2 regularization having linear regression, cross-validation may be performed
  • coef_: regression coefficient

class _BaseRidgeCV(LinearModel):
    def __init__(self, alphas=(0.1, 1.0, 10.0),
                 fit_intercept=True, normalize=False,scoring=None,
                 cv=None, gcv_mode=None,
                 store_cv_values=False):

2 regular observation of changes in the degree of impact on the results?

• The greater the intensity of regularization, the smaller the weighting factor
• Regularization smaller intensity, the greater the weighting factor will be

3 Boston prices forecast

def linear_model3():
    """
    线性回归:岭回归
    :return:
    """
    # 1.获取数据
    data = load_boston()

    # 2.数据集划分
    x_train, x_test, y_train, y_test = train_test_split(data.data, data.target, random_state=22)

    # 3.特征工程-标准化
    transfer = StandardScaler()
    x_train = transfer.fit_transform(x_train)
    x_test = transfer.fit_transform(x_test)

    # 4.机器学习-线性回归(岭回归)
    estimator = Ridge(alpha=1)
    # estimator = RidgeCV(alphas=(0.1, 1, 10))
    estimator.fit(x_train, y_train)

    # 5.模型评估
    # 5.1 获取系数等值
    y_predict = estimator.predict(x_test)
    print("预测值为:\n", y_predict)
    print("模型中的系数为:\n", estimator.coef_)
    print("模型中的偏置为:\n", estimator.intercept_)

    # 5.2 评价
    # 均方误差
    error = mean_squared_error(y_test, y_predict)
    print("误差为:\n", error)