原
吴恩达机器学习 - 评估假设
正则线性回归:
可视化数据:
Code:
load ('ex5data1.mat');
plot(X, y, 'rx', 'MarkerSize', 10, 'LineWidth', 1.5);
- 1
- 2
结果为:
代价函数:
公式(并正则化):
Code(填充在linearRegCostFunction.m中):
t = X*theta-y;
J = t'*t/(2.0*m) + lambda/(2.0*m)*theta(2:end)'*theta(2:end);
- 1
- 2
求正则线性回归梯度
公式:
Code:
grad(1) = (X(:,1)'*t)/m;
grad(2:end) = (X(:,2:end)'*t./m) + (lambda/m).*theta(2:end);
- 1
- 2
训练样本
效果如下:
偏差和方差
样本数与训练误差、交叉验证误差的关系
Code(learningCurve.m):
function [error_train, error_val] = ...
learningCurve(X, y, Xval, yval, lambda)
%LEARNINGCURVE Generates the train and cross validation set errors needed
%to plot a learning curve
% [error_train, error_val] = ...
% LEARNINGCURVE(X, y, Xval, yval, lambda) returns the train and
% cross validation set errors for a learning curve. In particular,
% it returns two vectors of the same length - error_train and
% error_val. Then, error_train(i) contains the training error for
% i examples (and similarly for error_val(i)).
%
% In this function, you will compute the train and test errors for
% dataset sizes from 1 up to m. In practice, when working with larger
% datasets, you might want to do this in larger intervals.
%
% Number of training examples
m = size(X, 1);
% You need to return these values correctly
error_train = zeros(m, 1);
error_val = zeros(m, 1);
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return training errors in
% error_train and the cross validation errors in error_val.
% i.e., error_train(i) and
% error_val(i) should give you the errors
% obtained after training on i examples.
%
% Note: You should evaluate the training error on the first i training
% examples (i.e., X(1:i, :) and y(1:i)).
%
% For the cross-validation error, you should instead evaluate on
% the _entire_ cross validation set (Xval and yval).
%
% Note: If you are using your cost function (linearRegCostFunction)
% to compute the training and cross validation error, you should
% call the function with the lambda argument set to 0.
% Do note that you will still need to use lambda when running
% the training to obtain the theta parameters.
%
% Hint: You can loop over the examples with the following:
%
% for i = 1:m
% % Compute train/cross validation errors using training examples
% % X(1:i, :) and y(1:i), storing the result in
% % error_train(i) and error_val(i)
% ....
%
% end
%
% ---------------------- Sample Solution ----------------------
for i = 1:m
theta = trainLinearReg(X(1:i,:),y(1:i),lambda);
[error_train(i), ~] = linearRegCostFunction(X(1:i,:),y(1:i),theta,0);
[error_val(i), ~] = linearRegCostFunction(Xval,yval,theta,0);
end
% -------------------------------------------------------------
% =========================================================================
end
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
图示(高偏差):
多项式回归
特征扩张(将一次线性的特征扩张到p次)
Code(polyFeatures.m):
function [X_poly] = polyFeatures(X, p)
%POLYFEATURES Maps X (1D vector) into the p-th power
% [X_poly] = POLYFEATURES(X, p) takes a data matrix X (size m x 1) and
% maps each example into its polynomial features where
% X_poly(i, :) = [X(i) X(i).^2 X(i).^3 ... X(i).^p];
%
% You need to return the following variables correctly.
X_poly = zeros(numel(X), p);
% ====================== YOUR CODE HERE ======================
% Instructions: Given a vector X, return a matrix X_poly where the p-th
% column of X contains the values of X to the p-th power.
%
%
for i = 1:p
X_poly(:,i) = X.^i;
end
% =========================================================================
end
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
效果查看
这里说明一下接下来运行ex5
的效果。因为我们把特征扩张到了8次方(本例中),所以最后一个特征的数值特别特别大,所以我们这里需要特征归一化。
程序是这么调用的(实际上我们可以自己写这个部分,也不是很麻烦)(featureNormalize.m):
function [X_norm, mu, sigma] = featureNormalize(X)
%FEATURENORMALIZE Normalizes the features in X
% FEATURENORMALIZE(X) returns a normalized version of X where
% the mean value of each feature is 0 and the standard deviation
% is 1. This is often a good preprocessing step to do when
% working with learning algorithms.
mu = mean(X);
X_norm = bsxfun(@minus, X, mu);
sigma = std(X_norm);
X_norm = bsxfun(@rdivide, X_norm, sigma);
% ============================================================
end
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
然后就是利用我们之前写好的函数计算代价啦:
效果图如下:
从图上看来是高方差了(过拟合)
然后我们看一下不同的λ对结果有什么影响吧(这一步不计分,只是帮助我们来理解)~
Code(直接在控制台运行)(只需对第一行进行修改):
lambda = 1;
[theta] = trainLinearReg(X_poly, y, lambda);
% Plot training data and fit
figure(1);
plot(X, y, 'rx', 'MarkerSize', 10, 'LineWidth', 1.5);
plotFit(min(X), max(X), mu, sigma, theta, p);
xlabel('Change in water level (x)');
ylabel('Water flowing out of the dam (y)');
title (sprintf('Polynomial Regression Fit (lambda = %f)', lambda));
figure(2);
[error_train, error_val] = ...
learningCurve(X_poly, y, X_poly_val, yval, lambda);
plot(1:m, error_train, 1:m, error_val);
title(sprintf('Polynomial Regression Learning Curve (lambda = %f)', lambda));
xlabel('Number of training examples')
ylabel('Error')
axis([0 13 0 100])
legend('Train', 'Cross Validation')
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
λ=1时(拟合的不错)
λ=32时(欠拟合啦)
那么我们改小一点:λ=0.1时(惩罚的力度不够,还是有点过拟合)
使用交叉验证集选择合适的λ(画出λ-Error曲线)
效果图:
发现λ大概在3的位置上比较好