Representative entropy in information theory is a measure of uncertainty of a random variable
1, the greater the entropy, the higher the uncertainty of the data, the more intense the random motion
2, the smaller the entropy, the lower the data uncertainties
Information entropy formula:
This equation represents samples have class k, pi represents the proportion of data in class i in the population.
Because the negative sign is between 0 and 1 pi belongs, then the log (pi) to less than zero, a minus sign.
For chestnut:
After entropy can see a smaller, more certain than the data after a previous data.
Designated points Objective:
such that the entropy reduction division
All division possibility to search, find the best way to divide, generate decision trees