4.1 Model Evaluation

4.1 Model Evaluation

Li Mu

Station B: https://space.bilibili.com/1567748478/channel/collectiondetail?sid=28144

Course homepage: https://c.d2l.ai/stanford-cs329p/

Courseware: https://c.d2l.ai/stanford-cs329p/_static/pdfs/cs329p_slides_6_1.pdf

Machine learning is not concerned with how well the trained model predicts the training data set, but more concerned with sampling a new data sample and then inputting it into the trained model, the prediction performance of the model for the new data.

1. Model indicators:

• In supervised learning, the model is trained by minimizing a loss:
  • The value of loss is a widely used metric to measure the quality of the model;
• Other metrics:
  • Model aspects: such as classification accuracy, mAP of target detection;
  • Business aspects: such as the impact of the model on revenue, reasoning time, etc.;
• When evaluating a model, use more than one metric at a time.

Example: ad display

Users will search through keywords, or on a certain page, the background will find relevant advertisements based on keywords and other information, and then the algorithm will predict the click-through rate of each advertisement, calculate the expected income and sort, and finally select the top advertisement for show.

In the binary classification problem of this ad, since most people do not click on the ad, the negative