Summary of all notes: "Machine Learning" Watermelon Book-Summary of reading notes
1. K-nearest neighbor learning
Relatively simple, the following article has been written, so I won’t repeat it.
Statistical learning method reading notes (14)-clustering method
Two, low-dimensional embedding
Problems such as sparse data samples and difficulty in distance calculation in high-dimensional situations? It is a serious obstacle faced by all machine learning methods and is called the "curse of dimensionality". An important way to alleviate the disaster of dimensionality is dimensionality reduction, also known as "dimensionality reduction".
- MDS algorithm
Third, principal component analysis
- Recent reconstruction
- Maximum separability
Fourth, nuclear linear dimensionality reduction
The linear dimensionality reduction method is cored based on the core technique.
- Nuclear Principal Component Analysis KPCA
Five, manifold learning
Manifold learning is a type of dimensionality reduction method that draws on the concept of topological manifolds. "Manifold" is a space that is locally homeomorphic to Euclidean space. In other words, it has the properties of Euclidean space locally and can be calculated using Euclidean distance.
- Isomap algorithm
- LLE algorithm
Six, metric learning
In machine learning, the main purpose of dimensionality reduction on high-dimensional data is to find a suitable low-dimensional space where learning can perform better than the original space. In fact, each space corresponds to a distance metric defined on the sample attributes, and searching for a suitable space is essentially looking for a suitable distance metric.
Linear model of metric learning
Linear metric learning problems are also called Mahalanobis metric learning problems, which can be divided into supervised and unsupervised learning algorithms.
Supervised global metric learning
- Information-theoretic metric learning(ITML)
- Mahalanobis Metric Learning for Clustering(MMC)
- Maximally Collapsing Metric Learning (MCML)
Supervised local metric learning
- Neighbourhood Components Analysis (NCA)
- Large-Margin Nearest Neighbors (LMNN)
- Relevant Component Analysis(RCA)
- Local Linear Discriminative Analysis(Local LDA)
Unsupervised metric learning
- Pricipal Components Analysis (PCA)
- Multi-dimensional Scaling (MDS)
- Non-negative Matrix Factorization (NMF)
- Independent components analysis (ICA)
- Neighborhood Preserving Embedding (NPE)
- Locality Preserving Projections. LPP
Nonlinear model of metric learning
Non-linear dimensionality reduction algorithm (non-linear metric learning):
- Isometric Mapping (ISOMAP)
- Locally Linear Embedding (LLE)
- Laplacian Eigenmap (Laplacian Eigenmap, LE)
Use the kernel method to extend the linear mapping:
- Non-Mahalanobis Local Distance Functions
- Mahalanobis Local Distance Functions
- Metric Learning with Neural Networks
Related papers
- Distance metric learning with application to clustering with side-information
- Information-theoretic metric learning(关于ITML)
- Distance metric learning for large margin nearest neighbor classification(关于LMNN)
- Learning the parts of objects by non-negative matrix factorization（Nature关于RCA的文章）
- Neighbourhood components analysis (About NCA)
- Metric Learning by Collapsing Classes(关于MCML)
- Distance metric learning a comprehensive survey (a classic review)
The next chapter portal: Watermelon book reading notes (11)-feature selection and sparse learning