Statistical Learning——Fundmental

Three elements in statistical learning:

1. model
2. strategy
3. algorithm

$method = model + strategy+algorithm$

Concept

data:

1. input space
2. output space
3. feature space

function：

1. hypothesis space

Classify

• supervised learning
• unsupervised learning
• reinforcement learning
• semi-supervised learning
• active learning

according to model

• probabilistic model / non-probabilistic model
• linear mode / non-linear model
• parametric model /non- parametric model

accoding to algorithm

• online learning
• batch learning

Loss function

1. 0-1 loss function
2. quadratic loss function
3. absolute loss function
4. logarithmic loss function

risk:

1. risk function / expected function
2. empirical risk / empirical loss
3. structural risk

Evaluation

1. training error
2. test error