Hands-on deep learning - (Li Mu) PyTorch version

3.1 Linear regression
3.2 Implementation of linear regression from scratch
3.3 Simple implementation of linear regression
3.4 Softmax regression
3.5 Image classification data set (Fashion-MNIST)
3.6 Implementation of softmax regression from scratch
3.7 Simple implementation of softmax regression
3.8 Multilayer perceptron
3.9 Implementation of multilayer perceptron from scratch
3.10 Simple implementation of multilayer perceptron
3.11 Model selection, underfitting and overfitting
3.12 Weight decay
3.13 Dropout method
3.14 Forward propagation, backpropagation and calculation Figure
3.15 Numerical stability and Model Initialization
3.16 Actual Kaggle Competition: House Price Prediction

4. Deep Learning Computing

4.1 Model Construction
4.2 Access, Initialization and Sharing of Model Parameters
4.3 Delayed Initialization of Model Parameters
4.4 Custom Layer
4.5 Reading and Storage
4.6 GPU Computing

5. Convolutional Neural Networks

5.1 2D Convolutional Layers
5.2 Padding and Striding
5.3 Multiple Input Channels and Multiple Output Channels
5.4 Pooling Layers
5.5 Convolutional Neural Networks (LeNet)
5.6 Deep Convolutional Neural Networks (AlexNet)
5.7 Networks Using Repeating Elements (VGG)
5.8 Networks in Networks (NiN)
5.9 Networks with Parallel Connections (GoogLeNet)
5.10 Batch Normalization
5.11 Residual Networks (ResNet)
5.12 Densely Connected Networks (DenseNet)

6. Recurrent Neural Networks

6.1 Language Model
6.2 Recurrent Neural Network
6.3 Language Model Dataset (Jay Chou Album Lyrics)
6.4 Scratch Implementation of Recurrent Neural Network
6.5 Compact Implementation of Recurrent Neural Network
6.6 Backpropagation Through Time
6.7 Gated Recurrent Unit (GRU)
6.8 Long and Short Term Memory (LSTM)
6.9 Deep Recurrent Neural Network
6.10 Bidirectional Recurrent Neural Network

7. Optimization algorithm

7.1 Optimization and Deep Learning
7.2 Gradient Descent and Stochastic Gradient Descent
7.3 Small Batch Stochastic Gradient Descent
7.4 Momentum Method
7.5 AdaGrad Algorithm
7.6 RMSProp Algorithm
7.7 AdaDelta Algorithm
7.8 Adam Algorithm

8. Computing performance

8.1 Imperative and symbolic mixed programming
8.2 Asynchronous computing
8.3 Automatic parallel computing
8.4 Multi-GPU computing

9. Computer Vision

9.1 Image augmentation
9.2 Fine-tuning
9.3 Object detection and bounding boxes
9.4 Anchor boxes
9.5 Multi-scale object detection
9.6 Object detection dataset (Pikachu)
9.7 Single-shot multi-box detection (SSD)
9.8 Region Convolutional neural network (R-CNN) series
9.9 Semantic Segmentation and Datasets
9.10 Fully Convolutional Network (FCN)
9.11 Style Transfer
9.12 Practical Kaggle Competition: Image Classification (CIFAR-10)
9.13 Practical Kaggle Competition: Dog Breed Recognition (ImageNet Dogs)

10. Natural Language Processing

10.1 Word embedding (word2vec)
10.2 Approximate training
10.3 Implementation of word2vec
10.4 Subword embedding (fastText)
10.5 Word embedding of global vector (GloVe)
10.6 Finding synonyms and analogs
10.7 Text sentiment classification: using cyclic neural network
10.8 Text sentiment classification: using Convolutional Neural Network (textCNN)
10.9 Encoder-Decoder (seq2seq)
10.10 Beam Search
10.11 Attention Mechanism
10.12 Machine Translation

environment

matplotlib==3.3.2
torch==1.1.0
torchvision==0.3.0
torchtext==0.4.0
CUDA Version==11.0

Reference (you can download the code here)

Implementation of PyTorch in this book: Dive-into-DL-PyTorch
Implementation of TendorFlow2.0 in this book: Dive-into-DL-TensorFlow2.0

Original book address (you can read the electronic PDF content here)

Chinese Version: Dive into Deep Learning | Github Warehouse
English Version: Dive into Deep Learning | Github Repo

quote

Please cite the original book if you use this project in your research:

@book{zhang2019dive,
    title={Dive into Deep Learning},
    author={Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola},
    note={\url{http://www.d2l.ai}},
    year={2020}
}

read the guide

Like the original book, the content of docs can be roughly divided into 3 parts:

The first part (chapters 1 to 3) covers preparation and basic knowledge. Chapter 1 introduces the background of deep learning. Chapter 2 provides the prerequisites for hands-on deep learning. Chapter 3 covers the most fundamental concepts and techniques of deep learning, such as multi-layer perceptrons and model regularization. If the reader has limited time and only wants to understand the most basic concepts and techniques of deep learning, then only read the first part.
The second part (Chapters 4 through 6) focuses on modern deep learning techniques. Chapter 4 describes the various important components of deep learning computing and lays the groundwork for implementing subsequent, more complex models. Chapter 5 explains the convolutional neural networks that have made deep learning so successful in computer vision in recent years. Chapter 6 describes the recurrent neural network that is commonly used to process sequence data in recent years. Reading Part II will help you master modern deep learning techniques.
Part III (Chapters 7 through 10) discusses computing performance and applications. Chapter 7 evaluates various optimization algorithms for training deep learning models. Chapter 8 examines several important factors affecting the computational performance of deep learning. Chapters 9 and 10 list important applications of deep learning in computer vision and natural language processing, respectively. Readers can choose to read this part according to their interests.
The diagram below depicts the structure of Hands-On Deep Learning.

insert image description here

The arrow pointing from Chapter A to Chapter B in the figure above indicates that the knowledge of Chapter A is helpful for understanding the content of Chapter B.

If readers want to understand the most basic concepts and techniques of deep learning in a short time, just read Chapters 1 to 3;

If readers want to master modern deep learning techniques, they also need to read Chapters 4 to 6.

From Chapter 7 to Chapter 10, readers can choose to read according to their interests.