1) 逻辑回归(Logistic Regression, Logistic Function, Sigmoid Function)的损失函数为:
J(θ)=−1m∑i=1m[y(i)loghθ(x(i))+(1−y(i))log(1−hθ(x(i)))]
其中
hθ(x(i))=11+exp(−θTx(i))
,
θTx(i)=θ0x0+θ1x1+θ2x2+...
2) 对其中一个参数,比如
θ1
对其求偏导:
∂J(θ)∂θ1=−1m∑i=1m[y(i)hθ(x(i))lne∂hθ(x(i))∂θ1+(1−y(i))(1−hθ(x(i)))lne−∂hθ(x(i))∂θ1]
3) 提取公因式
∂hθ(x(i))∂θ1
:
=−1m∑i=1m[∂hθ(x(i))∂θ1(y(i)hθ(x(i))+(1−y(i))(1−hθ(x(i))))]
=−1m∑i=1m[∂hθ(x(i))∂θ1y(i)−hθ(x(i))hθ(x(i))(1−hθ(x(i)))]
4) 其中假设函数
hθ(x(i))
对
θ1
求偏导:
∂hθ(x(i))∂θ1=−1(1+e−θTx(i))2e−θTx(i)(−1)x(i)1=hθ(x(i))(1−hθ(x(i)))x(i)1
5) 所以
θ1
偏导:
扫描二维码关注公众号,回复:
2179384 查看本文章
=−1m∑i=1mhθ(x)(1−hθ(x))x(i)1y(i)−hθ(x(i))hθ(x(i))(1−hθ(x(i)))=−1m∑i=1m[(y(i)−hθ(x(i)))x(i)1]
6) 所以对任意的参数
θj
求偏导:
∂J(θ)∂θj=−1m∑i=1m[(y(i)−hθ(x(i)))x(i)j]