What is Random Forest
There are many trees in a forest, and many decision trees in a random forest.
Random forest is an upgraded version of decision tree, and random refers to the growth process of the tree. No two leaves are the same, and every tree in a random forest is different. When building a decision tree, we will randomly select some samples from the training data with replacement. Similarly, we will not use all the features of the data, but randomly select some features for training. The samples used by each tree and The characteristics are different, and the training results are naturally different.
Why do you want to do this?
At the beginning of the training, we don't know which are abnormal samples, and which features have a greater impact on the classification results. The random process reduces the impact of both on the classification results.
The output of the random forest is determined by voting, and if a majority of the decision trees think the test data is a good apple, then we consider it a good apple. It's a lot like democratic decision-making among humans, although everyone has different information, reasoning processes, and conclusions. But when everyone has a vote, better decisions are often made. Because the trees are independent from each other, they can be trained simultaneously without taking too long. The stochastic process makes it less prone to overfitting.
It can handle high-dimensional data with many features, and does not need to do feature selection. After reasonable training, the accuracy is very high. When you don’t know what classification method to use, it’s a good idea to try random forest first.
In machine learning, random forest belongs to integrated learning, which is to combine multiple models to solve problems. These models will independently learn, predict, and vote for the results. Accuracy tends to be much higher than the model alone.
In addition to decision trees, other models such as neural networks can also be used.
Similarly, ensemble learning does not have to be the same model, and neural networks and decision trees can coexist in one system.