scikit-learn 1.3. Kernel ridge regression

转载：https://blog.csdn.net/u010016927/article/details/75257804

核岭回归是结合岭回归（线性最小二乘L2范数正则化）与内核的技巧。因此，它在各自的内核和数据中学习空间中的线性函数。对于非线性核，这对应于原始空间中的非线性函数。

学习KernelRidge模式的形成是支持向量回归（SVR）相同。然而，使用不同的损失函数：KRR采用平方误差损失而支持向量回归使用\ε不敏感损失，两者结合L2正则化。相反，SVR，拟合kernelridge可以在封闭的形式完成，通常是更快的为中型数据集。另一方面，学习的模型是不稀疏的，因此比SVR慢，在预测时刻学习的稀疏模型

下面的图比较KernelRidge与SVR在人工数据集，它由一个正弦目标函数和强噪声添加到每第五个数据点。获悉KernelRidge模型和SVR的策划，其中的复杂性/正则化和带宽的RBF核已经使用网格搜索优化。学习功能非常相似；然而，拟合KernelRidge是约七倍的速度拟合SVR（都用网格搜索）。然而，100000的目标值的预测是三倍以上速度与SVR已经学到了稀疏模型只使用约1 / 3的100个训练数据作为支持向量。

下图比较了不同大小的训练集的拟合和kernelridge和SVR预测时间。Fitting KernelRidge比SVR中训练集的速度（小于1000份）；然而，对于更大的训练集的SVR的尺度更好。关于时间的预测，SVR比所有尺寸的训练集KernelRidge更快因为学到的稀溶液。注意稀疏度和预测时间取决于SVR的参数ε和c；ε＝0对应稠密模型

sklearn.kernel_ridge.KernelRidge

class  sklearn.kernel_ridge. KernelRidge ( alpha=1,  kernel='linear',  gamma=None,  degree=3,  coef0=1,  kernel_params=None ) [source]

Kernel ridge regression 核岭回归.

Read more in the User Guide.

Parameters:

alpha : {float, array-like}, shape = [n_targets] Small positive values of alpha improve the conditioning of the problem and reduce the variance of the estimates. Alpha corresponds to (2*C)^-1 in other linear models such as LogisticRegression or LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number. 更小的正值参数alpha可以提升模型训练效果，并且可以减少训练误差。 这里的alpha为 (2*C)^-1，等同于逻辑回归和线性SVC等其他线性模型减少误差参数的效果，如果一个数组被传递，则假定惩罚是特定于目标的。因此它们必须在数量上相对应。 kernel : string or callable, default=”linear” Kernel mapping used internally. A callable should accept two arguments and the keyword arguments passed to this object as kernel_params, and should return a floating point number. gamma : float, default=None Gamma parameter for the RBF, laplacian, polynomial, exponential chi2 and sigmoid kernels. Interpretation of the default value is left to the kernel; see the documentation for sklearn.metrics.pairwise. Ignored by other kernels. degree : float, default=3 Degree of the polynomial kernel. Ignored by other kernels. coef0 : float, default=1 Zero coefficient for polynomial and sigmoid kernels. Ignored by other kernels. kernel_params : mapping of string to any, optional Additional parameters (keyword arguments) for kernel function passed as callable object.

Attributes:

dual_coef_ : array, shape = [n_samples] or [n_samples, n_targets] Representation of weight vector(s) in kernel space X_fit_ : {array-like, sparse matrix}, shape = [n_samples, n_features] Training data, which is also required for prediction

See also

Ridge

Linear ridge regression.

SVR

Support Vector Regression implemented using libsvm.

References

• Kevin P. Murphy “Machine Learning: A Probabilistic Perspective”, The MIT Press chapter 14.4.3, pp. 492-493

Examples

>>> from sklearn.kernel_ridge import KernelRidge
>>> import numpy as np
>>> n_samples, n_features = 10, 5
>>> rng = np.random.RandomState(0)
>>> y = rng.randn(n_samples)
>>> X = rng.randn(n_samples, n_features)
>>> clf = KernelRidge(alpha=1.0)
>>> clf.fit(X, y) 
KernelRidge(alpha=1.0, coef0=1, degree=3, gamma=None, kernel='linear',
            kernel_params=None)

Methods

fit(X[, y, sample_weight])	Fit Kernel Ridge regression model
get_params([deep])	Get parameters for this estimator.
predict(X)	Predict using the kernel ridge model
score(X, y[, sample_weight])	Returns the coefficient of determination R^2 of the prediction.
set_params(\*\*params)	Set the parameters of this estimator.

__init__ ( alpha=1,  kernel='linear',  gamma=None,  degree=3,  coef0=1,  kernel_params=None ) [source]

fit ( X,  y=None,  sample_weight=None ) [source]

Fit Kernel Ridge regression model

Parameters:	X : {array-like, sparse matrix}, shape = [n_samples, n_features] Training data y : array-like, shape = [n_samples] or [n_samples, n_targets] Target values sample_weight : float or numpy array of shape [n_samples] Individual weights for each sample, ignored if None is passed.
Returns:	self : returns an instance of self.

get_params ( deep=True ) [source]

Get parameters for this estimator.

Parameters:	deep : boolean, optional If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:	params : mapping of string to any Parameter names mapped to their values.

predict ( X ) [source]

Predict using the kernel ridge model

Parameters:	X : {array-like, sparse matrix}, shape = [n_samples, n_features] Samples.
Returns:	C : array, shape = [n_samples] or [n_samples, n_targets] Returns predicted values.

score ( X,  y,  sample_weight=None ) [source]

Returns the coefficient of determination R^2 of the prediction.

The coefficient R^2 is defined as (1 - u/v), where u is the regression sum of squares ((y_true - y_pred) ** 2).sum() and v is the residual sum of squares ((y_true - y_true.mean()) ** 2).sum(). Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.

Parameters:	X : array-like, shape = (n_samples, n_features) Test samples. y : array-like, shape = (n_samples) or (n_samples, n_outputs) True values for X. sample_weight : array-like, shape = [n_samples], optional Sample weights.
Returns:	score : float R^2 of self.predict(X) wrt. y.

set_params ( **params ) [source]

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns:	self :