PyTorch/torchvision.datasets自带常用数据集【总结】

一、PyTorch环境

@PyTorch 1.0
@安装torchvision及报错的解决办法如下:
@下载命令1：pip3 install torchvision #可能会报错
@下载命令2：pip install --no-deps torchvision
@Linux下，可能需要sudo

二、PyTorch自带常用数据集列表

• MNIST
  #一个手写数字数据集集，提供了60000+训练用例和10000个测试用例

  The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples.

• COCO
  #通常认为COCO是更倾向于图像分割的数据集

  COCO-Text is a new large scale dataset for text detection and recognition in natural images.
  Version 1.3 of the dataset is out!
  63,686 images, 145,859 text instances, 3 fine-grained text attributes.
  This dataset is based on the MSCOCO dataset.
  •Text localizations as bounding boxes
  •Text transcriptions for legible text
  •Multiple text instances per image
  •More than 63,000 images
  •More than 145,000 text instances
  •Text instances categorized into machine printed and handwritten text
  •Text instances categorized into legible and illegilbe text
  •Text instances categorized into English script and non-English script

• Captions
  #望文生义，主要是标题测试用例，通常算在COCO中

• Detection

• #感知数据集，通常也包含在COCO中

• LSUN
  #场景感知数据集（感觉很酷

  While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models.
  详细信息点击：LSUN

• ImageFolder
  #一个通用的数据加载器，数据集中的数据以以下方式组织

  root/dog/xxx.png
  root/dog/xxy.png
  root/dog/xxz.png
  root/cat/123.png
  root/cat/nsdf3.png
  root/cat/asd932_.png

• Imagenet-12
  #一个层次结构图像库，通常通过ImageFolder 实现
  #示例如下

  # Data loading code
  transform = transforms.Compose([
      transforms.RandomSizedCrop(224),
      transforms.RandomHorizontalFlip(),
      transforms.ToTensor(),
      transforms.Normalize(mean = [ 0.485, 0.456, 0.406 ],
                           std = [ 0.229, 0.224, 0.225 ]),
  ])
  
  traindir = os.path.join(args.data, 'train')
  valdir = os.path.join(args.data, 'val')
  train = datasets.ImageFolder(traindir, transform)
  val = datasets.ImageFolder(valdir, transform)
  train_loader = torch.utils.data.DataLoader(
      train, batch_size=args.batchSize, shuffle=True, num_workers=args.nThreads)
  

• CIFAR
  #一个常用的微型图像数据子集

  The CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

• STL10
  #STL-10数据集是一个用于开发无监督特征学习、深度学习、自学学习算法的图像识别数据集。

  The STL-10 dataset is an image recognition dataset for developing unsupervised feature learning, deep learning, self-taught learning algorithms. It is inspired by the CIFAR-10 dataset but with some modifications. In particular, each class has fewer labeled training examples than in CIFAR-10, but a very large set of unlabeled examples is provided to learn image models prior to supervised training. The primary challenge is to make use of the unlabeled data (which comes from a similar but different distribution from the labeled data) to build a useful prior. We also expect that the higher resolution of this dataset (96x96) will make it a challenging benchmark for developing more scalable unsupervised learning methods.

• SVHN
  #Street View House Numbers
  #SVHN是一个真实世界的图像数据集，用于开发机器学习和对象识别算法，对数据预处理和格式要求很低。

  SVHN is a real-world image dataset for developing machine learning and object recognition algorithms with minimal requirement on data preprocessing and formatting. It can be seen as similar in flavor to MNIST (e.g., the images are of small cropped digits), but incorporates an order of magnitude more labeled data (over 600,000 digit images) and comes from a significantly harder, unsolved, real world problem (recognizing digits and numbers in natural scene images). SVHN is obtained from house numbers in Google Street View images.

• PhotoTour
  #如题，旅游照片数据集，现在这个项目好像是一个子项目，有兴趣的朋友可以查一查。

三、调用方式

import torchvision.datasets as datasets
trainset = datasets.MNIST(root='./data', # 表示 MNIST 数据的加载的目录
                                      train=True,  # 表示是否加载数据库的训练集，false的时候加载测试集
                                      download=True, # 表示是否自动下载 MNIST 数据集
                                      transform=None) # 表示是否需要对数据进行预处理，none为不进行预处理

水平有限，如有错误还请指正。