sigmoid(x) function definition:
\[
\begin{align*}
\sigma(x) &= \frac{1}{1+e^{-x}} \\
\frac{d(\sigma(x))}{dx} &= \sigma(x)(1-\sigma(x))
\end{align*}
\]
The logistic regression classification model is defined as follows:
\[
\begin{align*}
P(Y=1|x) &= \frac{e^{w \cdot x}}{1+e^{w \cdot x}}\\
&= \frac{1}{1+e^{-w \cdot x}}\\
&=\sigma (w \cdot x)\\
P(Y=0|x) &= \frac{1}{1+e^{w \cdot x}} \\
&=1 - \sigma (w \cdot x)\\
\end{align*}
\]
It can be seen from the above that in the two-class problem, the output of the sigmoid function is the predicted value corresponding to the sample whose label value is 1
Defined by the cross-entropy function:
\[
crossEntropy = -\sum_{i=0}^{classNum}{y^{label}_i\log {y^{pred}_i}}
\]
\(y^{label}, y^{pred}\) are probability distributions, and labels\(y^{label}\) use one-hot encoding to indicate probability distributions.
For binary classification, the cross-entropy loss is defined as follows:
\[
\begin{align*}
Loss_{crossEntropy}& = - \left[ y^{label}_0 \log y_0^{pred} + y^{label}_1 \log y_1^{pred}\right] \\
&= - \left[ y^{label}_1 \log y_1^{pred} + (1- y^{label}_1) \log(1- y_1^{pred})\right]
\end{align*}
\]
And the output of the sigmoid function is the predicted value corresponding to the sample whose label value is 1, so
\[
\begin{align*}
Loss &= - \left[ y^{label}_1 \log \sigma(x) + (1- y^{label}_1) \log (1- \sigma(x)) \right] \\
\frac{\partial{Loss}}{{\partial{\sigma(x)}}} &= - \left[ \frac {y^{label}_1}{\sigma(x)} - \frac{(1- y^{label}_1)}{(1- \sigma(x))} \right] \\
&= \frac {\sigma(x) -y^{label}_1}{\sigma(x){(1- \sigma(x))}} \\
\end{align*}
\]
Known \(x = w \cdot z\), then the inverse of the Loss function to w is
\[
\begin{align*}
\frac{\partial Loss}{\partial w} &= \frac{\partial{Loss}}{{\partial{\sigma(x)}}}\frac{\partial{\sigma(x)}}{{\partial{x}}}\frac{\partial{x}}{{\partial{w}}} \\
&= \frac {\sigma(x) -y^{label}_1}{\sigma(x){(1- \sigma(x))}} \cdot {\sigma(x)(1-\sigma(x))} \cdot z \\
&= z \cdot \left ({\sigma(x) -y^{label}_1} \right)
\end{align*}
\]