Transfer from: https://blog.csdn.net/koala_tree/article/details/79913655
Author : Mr. tree
blog : http://blog.csdn.net/koala_tree
know almost : https://www.zhihu.com/people/dashuxiansheng
GitHub : https://github.com/KoalaTree
2018 April 5, day
This article published in the know almost column, for the convenience of users accustomed to using CSDN, change the linear notes to the following article in the CSDN.
At the same time, I also welcome everyone to pay attention to my knowledge: Mr. Dashu, there will be new dry goods updates from time to time. Learn together and make progress together! ^_^
Introduction to DeepLearning.ai
deepLearning.ai is a series of courses on teaching deep learning launched by Wu Enda on Coursera. The whole topic includes five courses: 01. Neural Networks and Deep Learning; 02. Improving Deep Neural Networks-Hyperparameter Tuning, Regularization and Optimization; 03. Structured Machine Learning Projects; 04. Convolutional Neural Networks; 05. Sequences model.
Course Description:
Allow me to quote the introduction of the official website:
If you want to get into artificial intelligence, this course topic will help you. Deep learning is one of the most sought-after skills in the field of technology, and we will help you learn this knowledge.
In the five courses, you will learn the basics of deep learning, understand how to build neural networks, and learn how to practice machine learning projects, learn convolutional networks, RNN, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, etc. . You will work on case studies in medical treatment, autonomous driving, sign language reading, music creation and natural language processing. By then, you will not only master the basic theories of deep learning, but also see its applications in industry. The above ideas will be implemented in Python and TensorFlow exercises. In addition, you will also hear many senior leaders in deep learning who will share their personal stories with you and provide you with career advice.
AI is having a huge impact on all walks of life. After completing the courses on this topic, you may find creative ways to apply it to your work. We will help you master deep learning, understand how to use it, and help you build a career in AI.
Course content:
- Coursera : Official course schedule (English subtitles). Paying users can get homework grades in coursework, and get a certificate of completion for each course; they can take classes and do homework for free without paying, but they can’t get a course certificate if they don’t have homework grades.
- NetEase Cloud Classroom : Genuine license introduced by NetEase (Chinese and English subtitles). The courses are completely free, but there are no homework and no course certificates.
Recommended:
4.5 stars (personal opinion) a
rare good course among the existing deep learning courses.
Collection of personal refining notes and programming assignments
The following are the main notes extracted from the individual in the course of class, as well as the programming homework completed by himself. Courses are the main course, exercises are supplemented, and notes are used to consolidate . So I suggest that you use this core idea to study this course. Not much nonsense, take notes!
01. Neural Networks and Deep Learning
- Introduction to Neural Networks
- Main introduction: the concept of neural network, the reason for the rise of deep learning, course content, etc.;
- Notes: Introductory course, no corresponding notes were made.
- Programming assignment: None
- Neural network basics
- Main introduction: logistic regression, loss function, gradient descent, calculation vectorization, cost function, etc.;
- Notes: DeepLearning.ai course refining notes (1-2) Neural Network and Deep Learning-Neural Network Foundation
- Programming assignment: basic Python using Numpy, logistic regression
- Shallow neural network
- Main introduction: neural network, activation function, gradient descent method, back propagation, random initialization, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (1-3) Neural Network and Deep Learning-Shallow Neural Network
- Programming assignment: Use shallow neural network to realize flat data classification
- Deep neural network
- Main introduction: deep neural network, DNN forward and backward propagation, parameters and hyperparameters, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (1-4) Neural Network and Deep Learning — Deep Neural Network
- Programming assignment: construct DNN and DNN for image classification
02. Improve deep neural network: hyperparameter debugging, regularization and optimization
- The practical aspects of deep learning
- Main introduction: training and test set division, bias and variance, regularization, dropout, input normalization, gradient disappearance and gradient explosion, weight initialization, gradient inspection, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (2-1) Improving Deep Neural Networks—Practical aspects of deep learning
- Programming tasks: initialization, regularization, gradient check
- optimization
- Main introduction: Mini-batch gradient descent, exponential weighted average, Momentum gradient descent, RMSprop, Adam optimization algorithm, attenuation learning rate, local optimization, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (2-2) Improve Deep Neural Network-Optimization Algorithm
- Programming tasks: multiple optimization algorithms
- Hyperparameter debugging and Batch Norm and framework
- Main introduction: Super parameter debugging, Batch Normalization, Softmax, TensorFlow program framework, etc.;
- Notes: DeepLearning.ai course refining notes (2-3) to improve deep neural networks-hyperparameter tuning and Batch Norm
- Programming assignment: TensorFlow simple tutorial
03. Structured Machine Learning Project
- Machine learning strategy (1)
- Main introduction: orthogonalization, single-digit evaluation index, training/development/test set, deviation and variance, improvement of model performance, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (3-1) Structured Machine Learning Project — Machine Learning Strategy (1)
- Programming assignment: None
- Machine learning strategy (2)
- Main introduction: error analysis, error sample removal, data distribution mismatch problem, transfer learning, multi-task learning, end-to-end deep learning, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (3-2) Structured Machine Learning Project — Machine Learning Strategy (2)
- Programming assignment: None
04. Convolutional Neural Network
- Convolutional neural network basics
- Main introduction: computer vision, edge detection, convolutional neural network, padding, convolution, pooling, etc.;
- Notes: DeepLearning.ai course refining notes (4-1) Convolutional Neural Network-Convolutional Neural Network Foundation
- Programming assignments: building convolutional neural networks, gesture recognition applications
- Convolutional neural network example model
- Main introduction: AlexNet, LeNet, VGG, ResNet, Inception Network, 1 by 1 convolution, migration learning, data expansion, etc.;
- Notes: DeepLearning.ai course refining notes (4-2) Convolutional Neural Network-Deep Convolution Model
- Programming assignment: Keras tutorial-the Happy House, Residual Networks
- Target Detection
- Main introduction: target positioning, target detection, Bounding Box prediction, intersection and ratio, non-maximum suppression NMS, Anchor box, YOLO algorithm, candidate region region proposals, etc.;
- Notes: DeepLearning.ai course refining notes (4-3) Convolutional Neural Network-Target Detection
- Programming task: Autonomous driving-car inspection
- Special application: face recognition and neural style transfer
- Main introduction: face recognition, one-shot learning, Siamese network, Triplet loss, style transfer, content loss, style loss, 1D-3D convolution, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (4-4) Convolutional Neural Network-Face Recognition and Neural Style Transfer
- Programming assignment: face recognition-Happy House, neural style transfer-deep learning and art
05. Sequence Model
- Recurrent neural network
- Main introduction: Recurrent neural networks, different types of RNNs, language models, new sequence sampling, RNN gradient disappearance, GRU, LSTM, bidirectional RNN, deep RNNs, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-1) Sequence Model-Recurrent Neural Network
- Programming assignment: build RNN , alphabet-level language model-Dinosaurus land , improvise Jazz with LSTM
- Natural language processing and word embedding
- Main introduction: vocabulary representation, Word Embedding, embedding matrix, Word2Vec, negative sampling, GloVe word vector, sentiment classification, word embedding to eliminate bias, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-2) Sequence Model-NLP and Word Embedding
- Programming assignments: word vector operations , Emojify
- Sequence model and attention mechanism
- Main introduction: sequence-to-sequence model, beam search, beam search error analysis, Bleu score, attention model, attention weight, speech recognition, trigger word detection, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-3) Sequence Model-Sequence Model and Attention Mechanism
- Programming tasks: machine translation , trigger word detection
to sum up
The learning span of the whole thematic course is relatively long, and the process of constantly thinking and taking notes during the class is indeed slow and hard, but it does have a lot of gains along the way. At the beginning, I just made a note for later review, but later I felt that the recorded notes were still neat, so I put it on Zhihu and shared it with everyone. I hope that these of mine can be given to more students and friends with the same needs. Bring a little help.
At last
Note is a refinement of the course. Although it is generally more comprehensive, it is limited to the individual's ability and energy. Omissions or errors will inevitably appear in the notes. If you find the wrong place while reading the notes and the content that I think is more important but I did not record, then you are welcome to leave a comment or private message to me below, I will make corrections and supplements in time, thank you for your support.
Finally, I would like to thank every friend who liked it. At the same time, other platforms are also welcome to reprint and share, and make progress together ^_^!