Li Hongyi's "Machine Learning" (DL direction) staged summary

Datawhale202211——Li Hongyi's "Machine Learning" DL Summary

---

---

foreword

Li Hongyi's team learning of machine learning has come to an end. Next, we will make a summary of the recent content, linking the past and the future, and continue to work hard! This article provides an application example, the data set used is fashion-mnist.

---

1. A series of articles guiding the way

original

Introduction to ML/DL (underlying understanding + basic concepts)
Introduction to ML/DL - Regression Error
and Gradient Descent (Li Hongyi "Machine Learning" ) What to do" (Li Hongyi "Machine Learning" <Deep Learning Direction>) A comprehensive introduction to CNN entry (Li Hongyi "Machine Learning" <Deep Learning Direction>)

Teammate Chuang (@晨哥is a good actor)

Regression model introduction
Neural network cannot be trained, what should I do?
convolutional neural network

2. Important knowledge points

gradient descent

loss function

backpropagation

CNN

3. Example combat (fashion-mnist demonstration)

1. Knowledge review

1) The Module class is the model construction class provided in torch.nn. It is used to construct the accumulation of all neural network modules. Other models can be built by inheriting it.
2) It provides three definition methods, namely Sequential, ModuleList, and ModuleDict.

2. Three definition methods + code examples

(Before the code demonstration, you need to introduce the required library)

import os
import numpy as np
import collections
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision

1) The Sequential(nn.Sequential())
model simply concatenates each layer when performing forward calculations, and Sequential defines the model in a simpler way. Receive a module's OrderedDict or a series of modules as parameters to pay attention to the example of adding Module. The forward calculation of the model is to calculate these examples in order. The code demonstration is as follows:

direct alignment

import torch.nn as nn
net1=nn.Sequential(
   nn.Linear(784,256),
   nn.ReLU(),    
   nn.Linear(256,10),
      )
   print(net1)
 #结果
 Sequential(
  (0): Linear(in_features=784, out_features=256, bias=True)
  (1): ReLU()
  (2): Linear(in_features=256, out_features=10, bias=True))
  #利用OrderedDict
  import collections
  import torch.nn as nn
  net2=nn.Sequential(collections.OrderedDict([
   ('fcl',nn.Linear(784,256)),('relu1',nn.ReLU()),       ('fc2',nn.Linear(256,10))
   ]))
   print(net2)
  #结果
  Sequential(
  (fcl): Linear(in_features=784, out_features=256, bias=True)
  (relu1): ReLU()
  (fc2): Linear(in_features=256, out_features=10, bias=True)
  )

2）ModuleList(nn.ModuleList())

ModuleList accepts a list of submodules as input, so append and extend operations can be performed, and at the same time, the weights of submodules or layers are automatically added to the network.

net3=nn.ModuleList([nn.Linear(784,256),nn.ReLU()])
net3.append(nn.Linear(256,10))
print(net3[-1])
print(net3)
Linear(in_features=256, out_features=10, bias=True)
ModuleList(
 (0): Linear(in_features=784, out_features=256, bias=True)
 (1): ReLU()
 (2): Linear(in_features=256, out_features=10, bias=True)
 )

It should be noted that ModuleList does not provide examples, but simply combines the modules together. Using it to instantiate the model requires the following operations, otherwise an error will be reported

（NotImplementedError）
class Net3(nn.Module):
   def __init__(self):
          super().__init__()
                 self.modulelist=nn.ModuleList([nn.Linear(784,256),nn.ReLU()])      self.modulelist.append(nn.Linear(256,10))    
   def forward(self,x):   
           for layer in self.modulelist:    
             x=layer(x)     
            return xnet3_=Net3()
            out3_=net3_(a)
            print(out3_.shape)

3）ModuleDict（nn.ModuleDict())

ModuleDict is similar to ModuleList, and the former can better add names to each layer to facilitate the realization of large models.

net4=nn.ModuleDict({
    
    
'linear':nn.Linear(784,256), 'act':nn.ReLU(),
})
net4['output']=nn.Linear(256,10)print(net4['linear'])
print(net4.output)

4. Reference documents

Feeding from Datawhale

Li Hongyi's "Machine Learning" open source content 1:
https://linklearner.com/datawhale-homepage/#/learn/detail/93
Li Hongyi's "Machine Learning" open source content 2:
https://github.com/datawhalechina/leeml -notes
Li Hongyi's "Machine Learning" open source content 3:
https://gitee.com/datawhalechina/leeml-notes

Feeding from the official

Li Hongyi's "Machine Learning" official address
http://speech.ee.ntu.edu.tw/~tlkagk/courses.html
Li Mu's "Hands-On Deep Learning" official address
https://zh-v2.d2l.ai/

---

Summarize

1. The team study this time is actually an introduction, which made me see the charm of classic courses. There is a long way to go and continuous improvement! ! !
2. Thanks to the organizers of Datawhale, still a great learning experience! ! ! I also want to thank the members of the get it team for their mutual help and support. It is not easy to persevere.