LR and Sigmod function

When using LR, often used to represent a probability function Sigmod why LR can be used Sigmod function?

First, LR assumption that only one, is characterized by two categories of unequal distribution with mean equal to the variance of the Gaussian distribution. Why is it assumed that Gaussian distribution? In one aspect, it is readily appreciated Gaussian distribution; the other hand, from the viewpoint of information theory, when the mean and variance are known, the Gaussian distribution is a distribution of maximum entropy. When the maximum entropy distribution, risk can be shared equally. As binary search method, each will look as middle point, the purpose is to be shared equally risk.

Since the definition of "risk":

$R(y=0|x) = \lambda _{00}P(y=0|x) + \lambda _{01}P(y=1|x)$

$R(y=1|x) = \lambda _{10}P(y=0|x) + \lambda _{11}P(y=1|x)$

In the formula, $R(y=0|x)$ it shows a sample prediction is zero risk, $R(y=1|x)$ expressed as a risk prediction sample 1, $\lambda _{ij}$ represents the prediction is $i$ , for the actual $j$ risk associated.

In LR algorithm, it is believed, will not pose a risk prediction is correct, that is, $\lambda _{00}$ and $\lambda _{11}$ are 0, in addition, that the tag is zero, and the forecast is 1 and think the label is 1, and the prediction is 0, both brought the risk is the same, so $\lambda _{01}$ and $\lambda _{10}$ unified with $\lambda$ to represent.

Above the "risk" reduces to:

$R(y=0|x) = \lambda P(y=1|x)$

$R(y=1|x) = \lambda P(y=0|x)$

For one sample, it should be predicted based on their category minimize the risk that compare two conditional probabilities, and the probability distribution of samples to the largest class.

Such as: $\frac{P(y=1|x)}{P(y=0|x)}$

The logarithmic formula, then Naive Bayes formula and obtain:

$lg{\frac{P(y=1|x)}{P(y=0|x)}}$

$= lg(\frac{P(x,y=1)}{P(x,y=0)})$

$Lg = {\ frac {P (x | y = 1) P (y = 1)} {P (x | y = 0) p (y = 0)}}$

$Lg = {\ frac {P (x | y = 1)} {P (x | y = 0)}} + {g \ frac {P (y = 1)} {P (y = 0)}}$

Because $P(y=1)$ and $P(y=0)$ is a constant, the constant can be used $C_1$ instead of, set into the Gauss formula,

$lg {\ frac {P (x | y = 1)} {P (x | y = 0)}} + {g \ frac {P (y = 1)} {P (y = 0)}}$

$= - \ frac {(x- \ mu _1) ^ 2} {2 \ sigma ^ 2} + \ frac {(x- \ mu _0) ^ 2} {2 \ sigma ^ 2} + C_1 = \ frac {\ it _1 - \ mu _2} {\ sigma ^ 2} x = wx + C_2$

Index taking both sides, to give:

$P(y=1|x) = \frac{1}{1+exp(-wx)}$

In summary, LR algorithm can be calculated using Sigmod function analysis.

LR and Sigmod function

Guess you like