Machine Learning course system

Written down on the number of public, want to learn the knowledge and then back again combing these elements, as a guide their learning

The first stage of the algorithm and machine learning basic
[core knowledge]
. The time complexity, space complexity analysis
. Master's Theorem, analysis recursive complexity
. Dynamic Programming and Dynamic Time Warpping
. Earth Mover's Distance
. Viterbi algorithm
. LR, tree , random forests, XGBoost
. gradient descent method, stochastic gradient descent method, Newton method
. the Projected gradient descent
. the L0, Ll, L2 of, L-Infinity Norm
. the Grid Search, the Bayesian Optimization
. convex function, convex set, Duality, KKT conditions
. Linear the SVM, Dual of the SVM
. Kernel the Tick, Mercer apos Theorem is
. Kernelized Linear Regression, Kernelized the KNN
. Linear / for a Quadratic Programming
. Integer / Semi-definite Programming
. the NP-Completeness / the NP-Hard / P / the NP
. Constrained Relaxation, Approximate Algorithm
. Convergence Analysis of Iterative Algorithm

[Part of the case to explain]
. Optimized write strategy based on the equity portfolio Sparse Quadratic Programming in
. Similarity calculation based on short text Earth Mover's Distance is
based on Projected Gradient Descent and NMF word vector learning
. Airfare pricing system based on Linear Programming of
based on text similarity analysis DTW

The second phase language model and serial label

[Core] knowledge
. Text pretreatment technology (tf-idf, Stemming, etc.)
. Characteristics engineering text field
. Inverted table, information retrieval technology
. Noisy Channel Model
. N-Gram model, vector word introduction
. Common Smoothing Techniques
. Learning to Rank
. Latent Variable the model
. the EM algorithm and the Local Optimality
. Convergence of the EM
. the EM and K-Means, the GMM
. Variational Autoencoder and the Text Disentangling
. digraph and undirected model
. Conditional Indepence, D-separation, Markov Blanket
. models, and the HMM parameter estimation
. the Viterbi, Baum Welch
. with the Log-Linear the model parameter estimation
. CRF model of CRF-Linear
. CRF and parameter estimation of Viterbi Decoding

[Part of the case to explain]
. Structures based Q & A system unsupervised learning methods
. Build a supervised learning of Aspect-Based sentiment analysis system
. NER-based applications CRF, LSTM-CRF, BERT- CRF is
based on a language model and Noisy Channel Model of spelling correction

The third phase information extraction, word vector map and knowledge

[Core knowledge points]
. NER technical
information extraction technology
. Snowball, KnowItAll, RunnerText
. Distant Supervision, unsupervised learning methods
. Entity unified entity disambiguation, anaphora resolution
. Mapping knowledge, entities and relationships
. Word vector, the Gram-Skip, the Sampling Negative
. matrix decomposition, CBOW with Glove vector
. Contexualized embedding and ELMO
. embedding KL Divergence and Gaussian
. embedding the non-Euclidean space with Pointcare
. Riemann space gradient descent method
. knowledge Mapping embedding technique
. TransE, NTN the Detailed
. Node2Vec explain
. Adversial Learning and KBGAN

[Section] Case explain
With unstructured data and information extraction technology to build knowledge map
. Build task-oriented robots chat
. Intent contains the Entity Extraction of NLU module implementation
. Implementation (Airbnb paper) SkipGram recommendation system based on

The fourth stage of deep learning and NLP

[Core] knowledge
. Pytorch and Tensorflow Detailed Indicates learning, distributed representation techniques
. Text field Disentangling
. Depth and BP neural network algorithm explain
. RNN and Vanishing / Exploding Gradient
. LSTM and GRU
. Seq2Seq and attention mechanisms
. Greedy Decoding and Beam Search
. BI-LSTM-CRF model
. Neural Turing Machine
. Memory Network
. Self the Attention, the Transformer and the Transformer-XL.
. Detailed Bert's
. BERT-BiLSTM-CRF
. GPT, MASS, XLNet
. Low-Resource Learning
visualization depth learning
. laywer-wise Relevance Propagation

[Part of the case explain]
Using pure Python implementation BP algorithm
based Seq2Seq + attentional mechanisms, Transformer machine translation system based on
based Transformer chat type chatbot
based BI-LSTM-CRF and BERT-BiLSTM-CRF in a named entity comparison
With laywer-wise RP end visualization of machine translation system

The fifth stage Bayesian models with NLP

[Core] knowledge
independent probabilistic graphical models and conditions
. Markov Blanket
. Dirichlet distribution, Multinomial distribution
. Beta distribution, Conjugate Prior Review
. Balance the Detail
. Detailed theme model
. MCMC and Gibbs sampling
. Gibbs Sampling topic model and Collapsed
. Hasting the Metropolis, the Sampling Rejection
. Dyamics with the Langevin SGLD
. SGLD distributed with a topic model
. topic the Dynamic the model
. Supervised topic the model
. KL Divergence and ELBO
. Variantional Inference, Stochastic Vl
. variational method and a topic model
. Nonparametric models
. the Dirichlet process
. Restarant Process chinese
. Deep Bayesian Neural Network
. Trick VAE and Reparametrization
. Bayesian RNN / LSTM
. Bayesian Word2Vec
. MMSB

[Section] Case explain
With Collapsed Gibbs Sampler and SGLD on topic models do Inference
. Named entity recognition based on Bayesian-LSTM
With the theme text classification model to do
. Modify and build unsupervised sentiment analysis model on the basis of LDA

The sixth stage of open items (Optional)

[Projects]
open project, also known as capstone project course. As an important part of the course, you can choose work on a challenging project. Through this project, you can go in-depth understanding of a particular field, quickly became an expert in this field, and let the results of the project to become a bright spot in the resume.

[Project] Process
Step 1: Team
Step 2: project and submit Proposal
the Step 3: Short Survey Paper
the Step 4: medium-term projects Review
the Step 5: PPT final project and submit the code
Step 6: final Presentation
the Step 7: Technical Report / blog

[Output]
complete PPT, the final stage of the code and the Conference-Style Technical Report most projects, we will organize a presentation of the participants shared the General Assembly. Whereby we will invite a number of industry experts, practitioners, corporate recruiters, headhunters high-quality resources to participate in the General Assembly to share.

Q system
from scratch to build a complete system of questions and answers. Given a corpus (questions and answers), for user input need to return the most appropriate answer. It relates to a module:

1. For the user's input needs to do spelling correction, this chapter uses language model
2. After pre-made text input, such as filtration.
3. Converting the text into a vector form, there will need related art tf-idf, word2vec like.
4. For in the corpus, in order to enhance the efficiency of the need to create inverted table.
5. Calculated based on similarity to obtain the optimal answer.

Sentiment analysis system
based on the given data, to build a complete sentiment analysis system. Projects related to the module:

1. Data preprocessing
2. Feature project, which is the core of this project.
3. Supervised learning model selection and parameter adjustment. Parameter adjustment process need to try different optimization strategies.

Knowledge Mapping systems
utilize unstructured data to build knowledge map. Projects related to the module:

1. Entity extraction, and unstructured data from the dictionary database construct
2. Extraction (relationship specified) relationship
3. Unified entity and an entity disambiguation.
4. Building knowledge map, and query

NLU dialogue system
based on a given conversation data to construct NLU identification section, and the results for the bot. Projects related to the module:

1. Extract text feature
2. CRF build models to identify keywords
3. Build LSTM-CRF model to identify keywords.

Machine translation system
based on the given data, to build a complete sentiment analysis system. Projects related to the module:

1. Data preprocessing
2. Feature project, which is the core of this project.
3. Supervised learning model selection and parameter adjustment. Parameter adjustment process need to try different optimization strategies.

Task-oriented chat robot
to build a complete robot chat, search for restaurant service. Projects related to the module:

1. Text preprocessing
2. Intended to identify and extract key information
3. For each session management state machine design intent
4. The method of design process context
5. Dialogue generation module
6. Deal with some common boundary case.