"Machine Learning Formula Derivation and Code Implementation" study notes, record your own learning process, please buy the author's book for detailed content.
This blog is to record the model and code of "Machine Learning Formula Derivation and Code Implementation" while learning. In some places, some changes have been made to the original author's code based on my own thinking. These blogs are mainly to implement some Model, feel the problems that each model of machine learning can solve and the effect after convergence, so I don't go too deep into the relevant theories.
1. Supervised learning model
chapter3-logarithmic probability regression
Logistic algorithm (logarithmic probability regression) numpy implementation
chapter4-LASSO regression and Ridge regression
Numpy implements lasso regression and ridge regression
chapter5-Linear Discriminant Analysis LDA
Linear Discriminant Analysis LDA Derivation and Manual Implementation
chapter6-k nearest neighbor algorithm
K nearest neighbor algorithm numpy implementation
Machine learning decision tree formula derivation and implementation: ID3, CART
2. Supervised learning ensemble model
Implementation of integrated learning Boosting algorithm AdaBoost based on numpy and sklearn
Manually implement GBDT classification tree and GBRT regression tree
XGBoost classification tree numpy implementation
LightGBM Introduction and Examples
A brief introduction to CatBoost and examples of using native libraries
Random forest numpy implementation
chapter16-integrated learning comparison and parameter adjustment
Comparison and parameter adjustment of the three integrated learning models XGBoost, LightGBM and CatBoost
3. Unsupervised learning model
kmeans implemented manually
chapter18-Principal Component Analysis PCA
numpy implementation of principal component analysis PCA
chapter19-Singular value decomposition SVD
numpy implementation of singular value decomposition svd and image compression
4. Probability Model
chapter21-Bayesian probability model
Derivation and Implementation of Naive Bayesian and Bayesian Networks
Introduction to EM algorithm, numpy programming EM algorithm to realize the three-coin problem
To be continued! Machine learning is broad and profound, and everyone is welcome to discuss it together!