Machine learning algorithms --XGboost

XGboost algorithm

　　XGBoost GBDT algorithm is an improvement, is a common integrated supervised learning algorithm; is one kind of scalability strong, easy GradientBoosting parallel algorithm to construct the model.

　　The principle is : adding a penalty term in the objective function on the basis of GBDT, as shown green box. To reduce the complexity of the model and the number of leaf nodes a value of a leaf node of the tree model limitations, thereby preventing over-fitting, one-half of the guide for the sake of convenience. t is the number of trees of the tree, obj to loss of function

　　General steps: prevent over-fitting, second order Taylor expansion formula, given the new criteria for the classification tree, with an incremental loss of function.

　　

　　The purpose : to find the first pieces of t is how to build a tree

So our expectation is that only the loss of function and t pieces of the tree has a relationship　　

 

 

 

 

 

 

 

 

　　XGBoost official website: http: //xgboost.readthedocs.io;

　　XGBoost supported development languages: Python, R, Java, Scala, C ++ and so on.

Installation a: compiling source code on Github, reference http://xgboost.readthedocs.io/en/latest/build.html

Installation Second way: python of whl file to install, requires python version 3.5 or 3.6; download link: https: //www.lfd.uci.edu/~gohlke/pythonlibs/#xgboost; installation commands: pipinstallf: /// xgboost-0.7-cp36-cp36m-win_amd64.whl

Installation III: direct command to install pip: pip install xgboost