Write directory title here
note reference
1. [Dry goods] Deep learning experiment process and solutions provided by PyTorch
Common Research workflow
One day, you are sitting in your lab chair and suddenly:
An idea pops up in your mind.
You read an article about a certain theory and want to try: What if you add xx to it?
Your boss suddenly gives you a piece of paper and says: Who, come and realize this thing So
, you designed the experimental process, and selected a suitable data set and operating environment for this idea, and then you realized the model without sleep or food. After a long period of training and testing, you found that:
This idea doesn't work --> Forget it or adjust again.
This idea works --> You can write paper
We can use the following figure to represent the above process:
the common process consists of the following items:
1 Once the data is selected set, you have to write some functions to load the data set, then pre-process the data set, and normalize the data set. It can be said that this is the most important part of an experiment, because: a. The format of each data set is
different
b The methods of preprocessing and regularization are also different
c need a fast dataloader to feed data, the faster the better
2 Then, you have to implement your own model . If you are in the CV direction, you may want to implement a ResNet. If you are related to NLP, you may want to implement a Seq2Seq
3 Next, you need to implement training steps, divide into batches, and cycle epoch
4 After several rounds of training, it is always safest to checkpoint
5 You also need to build some baselines to verify the effectiveness of your idea
6 If you are implementing a neural network model, of course you cannot do without the support of GPU
7 Many deep learning frameworks provide common loss functions, but most of the time, loss functions must be combined with specific tasks and then re-implemented
8 Use the optimization method to optimize the constructed model and dynamically adjust the learning rate
swin as a backbone
For semantic segmentation tasks, the Swin Transformer task exists as a backbone. The model pspnet_r50-d8_512x1024_40k_cityscapes given in the demo uses PSPNet as the detection method and ResNet-d8 as the backbone to extract feature maps . In order to experience the effect of Swin Transformer in semantic segmentation tasks, I also configured UPerNet as the detection method and Swin Transformer as the backbone detection model . However, there are some details about the support of the Swin Transformer framework in Swin-Transformer-Semantic-Segmentation, and some adjustments are required. Hereby, the stepping process is listed in detail.
experimental method
How to start with image semantic segmentation? How is the algorithm implemented?
After the model is built, use the nodes of the backbone network (lightweight backbone, complex backbone, and multi-task backbone to continuously update) to lead out its own segmentation branch, use the auxiliary model container to build the class of the segmentation head, and complete the process of building the segmentation network.
Start with mature projects, and make necessary modifications according to your own topics or tasks. It is recommended to use MMSegmentation, one of the OpenMMLab series, which is currently the most model-rich semantic segmentation code library in open source projects. If the graphics card in the laboratory is not sufficient, you can finetune with the training model provided in modelzoo, which can not only obtain higher performance, but also save training time~ Specifically, you can start with the configuration environment first. Run other people's open source code in your own environment. After completing this step, the basic setup is complete. Then it's time to play freely according to the topic. According to your project requirements, first clarify the problem you want to solve or prove, and then choose a suitable data set (appropriate size and domain). If you want to improve the effect of a certain aspect, first do some baseline experiments as a comparison, and then Apply your own proposed method that may solve the problems in the baseline experiment, and iterate the method step by step~