[Data Mining] Automobile transaction price prediction detailed version {feature engineering, cross-validation, drawing learning rate curve and verification curve}

Car transaction price prediction is a typical regression problem that can be solved by machine learning. Below I will introduce in detail the steps of feature engineering, cross-validation, drawing learning rate curve and verification curve of the automobile transaction price prediction task.

1. feature engineering

Feature engineering is a very important step in machine learning. It can process and transform raw data to better express the essential characteristics of the data. In the task of car transaction price prediction, we can consider the following characteristics:

• Basic vehicle information: such as model, year, brand, mileage, etc.
• Vehicle configuration information: such as engine displacement, gearbox type, drive mode, body color, etc.
• Vehicle history and maintenance records: such as whether there are accident records, maintenance status, etc.
• Market information: the average selling price of the same model, the selling prices of other models of the same brand, etc.

Here are a few things to keep in mind when choosing features:

• Features should cover as much information as possible, but should not be too redundant, that is, there should be a certain degree of independence between different features.
• The selection of features should be carried out according to the actual situation, and the best combination of features can be obtained through professional knowledge or experiments.

2. cross-check

Cross-validation is a commonly used method in machine learning to evaluate model performance. In cross-validation, we divide the data set into several parts, each time using one part as a validation set, and the rest as a training set for training. This avoids problems such as overfitting and underfitting, and also enables more accurate model performance evaluation results.

Common cross-validation methods include K-fold cross-validation and leave-one-out cross-validation. Among them, the K-fold cross-validation divides the data set into K parts randomly, one of which is used as the verification set each time, and the remaining K-1 parts are used as the training set; the leave-one-out cross-validation divides the data set into N parts, and one of them is divided into N parts each time. One is used as a verification set, and the remaining N-1 are used as a training set.

3. Plot Learning Rate Curve vs. Validation Curve