The way Colab unzips the compressed package && deletes the non-empty folder

Colaboratory, referred to as "Colab", was developed by the Google Research team. Anyone can write and execute arbitrary Python code through a browser, especially suitable for machine learning, data analysis, and educational purposes. Colab is a managed Jupyter notebook service that users can use directly without setting up, and can use GPU/TPU computing resources for free. Select "Upload Notebook" from the "File" menu to import an existing Jupyter/IPython notebook. Colab notebooks are stored on Google Drive or loaded from GitHub. For more details, see the documentation: colaboratory .

Question 1: Colab decompresses the CIFAR-10 dataset

When using the GPU in Colab for deep learning training, it is easy to train with the official data set provided by CIFAR-10. There are only a few files, and it can be easily run without decompression tools. If there are 60,000 pictures with a size of 55M, it may take a day to decompress (the personal test uses the method shown in the figure below, and it took about 8 hours to decompress 30,000 pictures).

Question 2: Colab does not support deleting non-empty folders

What if you want to delete the data set you just decompressed? One picture one picture delete? Then you will be slow when you sit.

General solution: Python script

In the final analysis, Colab can provide us with a Python environment, and we can execute Python scripts, and using Python scripts can often achieve the functions we want. In addition to the above two problems, we can consider other problems from this perspective.

First load the Google hard drive, and Baidu if you don’t know it.

Solution to problem one: unzip command

!unzip "/content/drive/MyDrive/Colab Notebooks/ResNet18/data.zip" -d "/content/drive/MyDrive/Colab Notebooks/ResNet18/"

Replace the directory with your own directory (be sure to check the path before executing the command).

This method is so fast that it takes off, and the 6w pictures are decompressed in just a few minutes!

Solution to Problem 2: Recursively delete non-empty folders

Just upload the code directly, everyone who engages in deep learning should understand these basic knowledge!

import os
# os.listdir('/content/drive/MyDrive/Colab Notebooks/ResNet18/data')
# os.remove(f'/content/drive/MyDrive/Colab Notebooks/ResNet18/data/')

def rmdir(dir):
    #判断是否是文件夹，如果是，递归调用rmdir()函数
    if(os.path.isdir(dir)):
        #遍历地址下的所有文件及文件夹
        for file in os.listdir(dir):
            #进入下一个文件夹中进行删除
            rmdir(os.path.join(dir,file))
        #如果是空文件夹，直接删除
        if (os.path.exists(dir)):
            os.rmdir(dir)
            print(dir,"文件夹删除成功!")
    #如果是文件，直接删除
    else:
        if(os.path.exists(dir)):
            os.remove(dir)
            print(dir,"文件删除成功!")
#调用定义函数（路径换成自己的，最好自己先找个废目录测试一下！）
rmdir("/content/drive/MyDrive/Colab Notebooks/ResNet18/data")

Delete non-empty directories in seconds!

---