训练集：\[\left\{ {\left( {{x^{\left( 1 \right)}},{y^{\left( 1 \right)}}} \right),\left( {{x^{\left( 2 \right)}},{y^{\left( 2 \right)}}} \right),...,\left( {{x^{\left( m \right)}},{y^{\left( m \right)}}} \right)} \right\}\]
\[x \in \left[ {\begin{array}{*{20}{c}}
{{x_0}}\\
{{x_2}}\\
\begin{array}{l}
.\\
.
\end{array}\\
{{x_n}}
\end{array}} \right]{x_0} = 1,y \in \left\{ {0,1} \right\}\]
\[{h_\theta }(x) = \frac{1}{{1 + {e^{ - {\theta ^T}x}}}}\]
如何选取参数θ？
损失函数
对于线性回归：\[J\left( \theta \right) = \frac{1}{m}\sum\limits_{i = 1}^m {\frac{1}{2}{{\left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right) - {y^{\left( i \right)}}} \right)}^2}} \]
令 \[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = \frac{1}{2}{\left( {{h_\theta }\left( x \right) - y} \right)^2}\]
对于逻辑回归而言，这样的损失函数是非凸面函数，应该对损失函数做处理
\[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = \left\{ {\begin{array}{*{20}{c}}
{\begin{array}{*{20}{c}}
{ - \log \left( {{h_\theta }\left( x \right)} \right)}&{\begin{array}{*{20}{c}}
{if}&{y = 1}
\end{array}}
\end{array}}\\
{\begin{array}{*{20}{c}}
{\begin{array}{*{20}{c}}
{ - \log \left( {1 - {h_\theta }\left( x \right)} \right)}&{if}
\end{array}}&{y = 0}
\end{array}}
\end{array}} \right.\]
当y=1时，函数的图形如下图
当y=0时，函数的图形如下图
代价函数可以写成 \[{\mathop{\rm Cos}\nolimits} t\left( {{h_\theta }\left( x \right),y} \right) = - y\log \left( {{h_\theta }\left( x \right)} \right) - \left( {1 - y} \right)\log \left( {1 - {h_\theta }\left( x \right)} \right)\]
这时 \[J\left( \theta \right) = - \frac{1}{m}[\sum\limits_{i = 1}^m {{y^{\left( i \right)}}\log \left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right) + \left( {1 - {y^{\left( i \right)}}} \right)\log \left( {1 - {h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right)]} \]
我们需要最小化代价函数J(θ)来找到θ
找到θ后运用h(x)预测新的输入x