Andrew Ng DeepLearning.ai "Deep Learning" Course Notes Catalog Collection

Reprinted: https://blog.csdn.net/koala_tree/article/details/79913655

Introduction to DeepLearning.ai

deepLearning.ai is a series of special courses on deep learning launched by Andrew Ng on Coursera. The whole topic includes five courses: 01. Neural Networks and Deep Learning; 02. Improving Deep Neural Networks - Hyperparameter Debugging, Regularization and Optimization; 03. Structured Machine Learning Projects; 04. Convolutional Neural Networks; 05. Sequences Model.

Course Description:

Please allow me to quote the introduction from the official website:

If you want to get into artificial intelligence, this course topic will help you. Deep learning is one of the most sought-after skills in tech, and we're here to help you learn it.
In five courses, you'll learn the basics of deep learning, learn how to build neural networks, and learn how to practice machine learning projects, learn about convolutional networks, RNNs, LSTMs, Adam, Dropout, BatchNorm, Xavier/He initialization, and more . You will work on case studies in healthcare, autonomous driving, sign language reading, music composition and natural language processing. By then, you will not only master the basic theory of deep learning, but also see its application in industry. All of the above ideas will be implemented in Python and TensorFlow exercises. Plus, you'll hear from many of the top leaders in deep learning who will share their personal stories with you and offer you career advice.
AI is having a huge impact on all walks of life, and after completing a course on this topic, you may find creative ways to apply it to your work. We'll help you master deep learning, understand how to use it, and help you build a career in AI.

Course content:

Coursera : Official course schedule (with English subtitles). Paid users can get homework grades in coursework, and a course completion certificate can be obtained after completing each course; free class and homework can be done without payment, but without homework grades, the course certificate cannot be obtained after completion of the course.
NetEase Cloud Classroom : A genuine license introduced by NetEase (with Chinese and English subtitles). The course is completely free, but there is no homework and no course certificate.

Collection of personal refining notes and programming assignments

The following are the key notes extracted from the individual in the course of the class, as well as the after-school programming assignments completed by themselves. The course is the main, the practice is supplemented, and the notes are consolidated . Therefore, it is recommended that you study this course with this core idea. Without further ado, take notes!

01. Neural Networks and Deep Learning

Introduction to Neural Networks
- Main introduction: the concept of neural network, the reasons for the rise of deep learning, course content, etc.;
- Notes: Introductory class, no corresponding notes are made.
- Programming assignments: none
Neural Network Basics
- Main introduction: logistic regression, loss function, gradient descent, computational vectorization, cost function, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (1-2) Neural Networks and Deep Learning - Neural Network Basics
- Programming assignments: Basic Python, logistic regression using Numpy
shallow neural network
- Main introduction: neural network, activation function, gradient descent method, back propagation, random initialization, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (1-3) Neural Networks and Deep Learning - Shallow Neural Networks
- Programming Assignment: Flat Data Classification Using Shallow Neural Networks
deep neural network
- Main introduction: deep neural network, forward and back propagation of DNN, parameters and hyperparameters, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (1-4) Neural Networks and Deep Learning - Deep Neural Networks
- Programming assignment: building DNN, DNN for image classification

02. Improving Deep Neural Networks: Hyperparameter Tuning, Regularization, and Optimization

Practical Aspects of Deep Learning
- Main introduction: training test set division, bias and variance, regularization, dropout, input normalization, gradient disappearance and gradient explosion, weight initialization, gradient test, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (2-1) Improving Deep Neural Networks - Practical Aspects of Deep Learning
- Programming assignments: initialization, regularization, gradient checking
optimization
- Main introduction: Mini-batch gradient descent, exponential weighted average, Momentum gradient descent, RMSprop, Adam optimization algorithm, decay learning rate, local optimum, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (2-2) Improving Deep Neural Networks - Optimization Algorithms
- Programming Assignments: Multiple Optimization Algorithms
Hyperparameter Tuning and Batch Norm and Framework
- Main introduction: debugging of hyperparameters, Batch Normalization, Softmax, TensorFlow program framework, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (2-3) Improving Deep Neural Networks - Hyperparameter Tuning and Batch Norm
- Programming Homework: TensorFlow Simple Tutorial

03. Structured Machine Learning Project

Machine Learning Strategies (1)
- Main introductions: orthogonalization, single-digit evaluation metrics, training/dev/test sets, bias and variance, improving model performance, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (3-1) Structured Machine Learning Project - Machine Learning Strategies (1)
- Programming assignments: none
Machine Learning Strategies (2)
- Main introduction: error analysis, error sample removal, data distribution mismatch problem, transfer learning, multi-task learning, end-to-end deep learning, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (3-2) Structured Machine Learning Project - Machine Learning Strategies (2)
- Programming assignments: none

04. Convolutional Neural Networks

Convolutional Neural Network Basics
- Main introduction: computer vision, edge detection, convolutional neural network, padding, convolution, pooling, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (4-1) Convolutional Neural Networks - Basics of Convolutional Neural Networks
- Programming Assignments: Building Convolutional Neural Networks, Gesture Recognition Applications
Convolutional Neural Network Instance Model
- Main introduction: AlexNet, LeNet, VGG, ResNet, Inception Network, 1 by 1 convolution, transfer learning, data expansion, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (4-2) Convolutional Neural Network - Deep Convolution Model
- Programming assignments: Keras Tutorial - the Happy House, Residual Networks
Target Detection
- Main introduction: target positioning, target detection, Bounding Box prediction, intersection ratio, non-maximum suppression NMS, Anchor box, YOLO algorithm, candidate region region proposals, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (4-3) Convolutional Neural Network - Target Detection
- Programming Assignment: Autonomous Driving - Vehicle Detection
Special Applications: Face Recognition and Neural Style Transfer
- Main introduction: face recognition, one-shot learning, Siamese network, Triplet loss, style transfer, content loss, style loss, 1D-3D convolution, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (4-4) Convolutional Neural Networks - Face Recognition and Neural Style Transfer
- Programming Assignments: Face Recognition - Happy House, Neural Style Transfer - Deep Learning and Art

05. Sequence model

Recurrent Neural Network
- Main introductions: Recurrent Neural Networks, Different Types of RNNs, Language Models, New Sequence Sampling, RNN Gradient Vanishing, GRU, LSTM, Bidirectional RNNs, Deep RNNs, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-1) Sequence Model - Recurrent Neural Network
- Programming assignments: building RNNs , letter-level language models - Dinosaurus land , improvising Jazz with LSTMs
Natural Language Processing and Word Embeddings
- Main introduction: vocabulary representation, Word Embedding, embedding matrix, Word2Vec, negative sampling, GloVe word vector, sentiment classification, word embedding to eliminate bias, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-2) Sequence Models - NLP and Word Embeddings
- Programming assignments: word vector operations , Emojify
Sequence Models and Attention Mechanisms
- Main introduction: sequence-to-sequence model, beam search, beam search error analysis, Bleu score, attention model, attention weight, speech recognition, trigger word detection, etc.;
- Notes: DeepLearning.ai Course Refinement Notes (5-3) Sequence Model - Sequence Model and Attention Mechanism
- Programming assignments: machine translation , trigger word detection

Summarize

The learning span of the entire special course is relatively long, and the process of constantly thinking and taking notes in the course of the class is indeed slow and hard, but I have indeed gained a lot along the way. At the beginning of the period, I just wanted to make a note for my later review, but later I felt that the notes I recorded were quite neat, so I put it on Zhihu and shared it with everyone. I hope these can be given to more students and friends who have the same needs. Bring a little help.

finally

Notes belong to the refinement of the course. Although generally speaking, it is relatively comprehensive, but limited to personal ability and energy, there will inevitably be omissions or mistakes in the notes. If you find wrong places or things that you think are important but I didn't record when you read the notes, you are welcome to leave a comment below or send me a private message, and I will make corrections and additions in a timely manner. Thank you for your support.

Finally, thank you to everyone who liked it. At the same time, other platforms are also welcome to reprint and share, and make progress together ^_^!