Boosting Few-Shot Visual Learning with Self-Supervision（代码理解）

GitHub - valeoai/BF3S: Boosting Few-Shot Visual Learning with Self-Supervision

Preparation

Pre-requisites

• Python 3.7
• Pytorch >= 1.0.0
• CUDA 10.0 or higher

Installation

(1) Clone the repo:

$ git clone https://github.com/valeoai/BF3S

(2) Install this repository and the dependencies using pip:

$ pip install -e ./BF3S

With this, you can edit the BF3S code on the fly and import function and classes of BF3S in other projects as well.

(3) Optional. To uninstall this package, run:

$ pip uninstall BF3S

(4) Create dataset and experiment directories:

$ cd BF3S
$ mkdir ./datasets
$ mkdir ./experiments

You can take a look at the Dockerfile if you are uncertain the about steps to install this project.

Necessary datasets.

(1) MiniImagenet To download the MiniImagenet dataset go to this github page and follow the instructions there. Then, set in bf3s/datasets/mini_imagenet_dataset.py the path to where the dataset resides in your machine.

(2) tiered-MiniImagenet To download the tiered-MiniImagenet dataset go to this github page and follow the instructions there. Then, set in bf3s/datasets/tiered_mini_imagenet_dataset.py the path to where the dataset resides in your machine.

(3) ImageNet-FS Download the ImageNet dataset and set in bf3s/datasets/imagenet_dataset.py the path to where the dataset resides in your machine.

(4) CIFAR-FS The dataset will be automatically downloaded when you run the code. Set in bf3s/datasets/cifar100_fewshot_dataset.py the path to where the dataset should be downloaded.

这里我下载的数据集是“MiniImagenet”（因为readme中的MiniImagenet 下载地址无效），它的链接为：https://drive.google.com/file/d/12V7qi-AjrYi6OoJdYcN_k502BM_jcP8D/view

提供该链接的网址为：GitHub - kjunelee/MetaOptNet: Meta-Learning with Differentiable Convex Optimization (CVPR 2019 Oral)

Download pre-trained models (optional).

(1) Download the models trained on the MiniImageNet dataset.

# Run from the BF3S directory
$ mkdir ./experiments/miniImageNet
$ cd ./experiments/miniImageNet

# WRN-28-10-based Cosine Classifier (CC) with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip

# WRN-28-10-based CC with rotation prediction self-supervision model trained with extra unlabeled images from tiered-MiniImageNet.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip

# WRN-28-10-based CC with location prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierLocSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierLocSelfsupervision.zip

$ cd ../../

(2) Download the model trained on the CIFAR-FS dataset.

# Run from the BF3S directory
$ mkdir ./experiments/cifar
$ cd ./experiments/cifar

# WRN-28-10-based CC with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.2.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip

$ cd ../../

(3) Download the model trained on the ImageNet-FS dataset.

# Run from the BF3S directory
$ mkdir ./experiments/ImageNet
$ cd ./experiments/ImageNet

# ResNet10-based CC with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.3.0/ResNet10CosineClassifierRotSelfsupervision.zip
$ unzip ResNet10CosineClassifierRotSelfsupervision.zip

$ cd ../../

Train and test Cosine Classifier (CC) based few-shot models [3] with self-supervision.

MiniImageNet: CC model with rotation prediction self-supervision.

To train and test the WRN-28-10 based Cosine Classifier model with rotation prediction self-supervision run (you can skip the training step if you have downloaded the pre-trained model):

# Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 62.81% with confidence interval +/- 0.46%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 80.00% with confidence interval +/- 0.34%

Note that the configuration file (of the above experiment) specified by the config variable is located here: ./config/miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision.py. All the experiment configuration files are placed in the ./config/ directory.

(imagemol) D:\pycharm_workspace\2\BF3S>python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
Loading experiment miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
Generated logs and/or snapshots will be stored on D:\pycharm_workspace\2\BF3S\experiments\miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
==> Download MiniImageNet dataset at ./datasets/MiniImagenet
Loading mini ImageNet dataset - phase train
==> Download MiniImageNet dataset at ./datasets/MiniImagenet
Loading mini ImageNet dataset - phase val
2022-08-24 20:34:00,187 - bf3s.algorithms.algorithm - INFO   - ==> Tensorboard file location: D:\pycharm_workspace\2\BF3S\experiments\miniImageNet\WRNd28w10CosineClassifierRotAugRotSelfsupervision\logs\TENSORBOARD
2022-08-24 20:34:00,187 - bf3s.algorithms.algorithm - INFO   - Algorithm options {'data_train_opt': {'dataset_name': 'MiniImageNet80x80', 'nKnovel': 0, 'nKbase': 64, 'n_exemplars': 0, 'n_test_novel': 0, 'n_test_base': 64, 'batch_size': 1, 'epoch_size': 1000, 'phase': 'train'}, 'data_test_opt': {'dataset_name': 'MiniImageNet80x80', 'nKnovel': 5, 'nKbase': 64, 'n_exemplars': 1, 'n_test_novel': 75, 'n_test_base': 75, 'batch_size': 1, 'epoch_size': 500}, 'max_num_epochs': 26, 'networks': {'feature_extractor': {'def_file': 'feature_extractors.wide_resnet', 'pretrained': None, 'opt': {'depth': 28, 'widen_Factor': 10, 'drop_rate': 0.0, 'pool': 'none'}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}, 'classifier': {'def_file': 'classifiers.cosine_classifier_with_weight_generator', 'pretrained': None, 'opt': {'num_classes': 64, 'num_features': 640, 'scale_cls': 10, 'learn_scale': True, 'global_pooling': True}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}, 'classifier_aux': {'def_file': 'classifiers.convnet_plus_classifier', 'pretrained': None, 'opt': {'convnet_type': 'wrn_block', 'convnet_opt': {'num_channels_in': 640, 'num_channels_out': 640, 'num_layers': 4, 'stride': 2}, 'classifier_opt': {'classifier_type': 'cosine', 'num_channels': 640, 'scale_cls': 10.0, 'learn_scale': True, 'num_classes': 4, 'global_pooling': True}}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}}, 'criterions': {'loss': {'ctype': 'CrossEntropyLoss', 'opt': None}}, 'algorithm_type': 'selfsupervision.fewshot_selfsupervision_rotation', 'auxiliary_rotation_task_coef': 1.0, 'rotation_invariant_classifier': True, 'random_rotation': False, 'exp_dir': 'D:\\pycharm_workspace\\2\\BF3S\\experiments\\miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision', 'disp_step': 200}
2022-08-24 20:34:00,188 - bf3s.algorithms.algorithm - INFO   - Set network feature_extractor
2022-08-24 20:34:00,188 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network feature_extractor from file feature_extractors.wide_resnet with opts: {'depth': 28, 'widen_Factor': 10, 'drop_rate': 0.0, 'pool': 'none'}
2022-08-24 20:34:00,537 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
2022-08-24 20:34:00,538 - bf3s.algorithms.algorithm - INFO   - WideResnet(
  (_feature_blocks): ModuleList(
    (0): Sequential(
      (Conv): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (BN): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (ReLU): ReLU(inplace=True)
    )
    (1): Sequential(
      (Block): NetworkBlock(
        (layer): Sequential(
          (0): BasicBlock(
            (convResidual): Sequential(
              (conv1): Conv2d(16, 160, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Conv2d(16, 160, kernel_size=(1, 1), stride=(2, 2), bias=False)
          )
          (1): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (2): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (3): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
        )
      )
      (BN): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (ReLU): ReLU(inplace=True)
    )
    (2): Sequential(
      (Block): NetworkBlock(
        (layer): Sequential(
          (0): BasicBlock(
            (convResidual): Sequential(
              (conv1): Conv2d(160, 320, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Conv2d(160, 320, kernel_size=(1, 1), stride=(2, 2), bias=False)
          )
          (1): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (2): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (3): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
        )
      )
      (BN): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (ReLU): ReLU(inplace=True)
    )
    (3): Sequential(
      (Block): NetworkBlock(
        (layer): Sequential(
          (0): BasicBlock(
            (convResidual): Sequential(
              (conv1): Conv2d(320, 640, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Conv2d(320, 640, kernel_size=(1, 1), stride=(2, 2), bias=False)
          )
          (1): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (2): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
          (3): BasicBlock(
            (convResidual): Sequential(
              (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu1): ReLU(inplace=True)
              (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              (relu2): ReLU(inplace=True)
              (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            )
            (convShortcut): Sequential()
          )
        )
      )
      (BN): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (ReLU): ReLU(inplace=True)
    )
  )
)
2022-08-24 20:34:00,540 - bf3s.algorithms.algorithm - INFO   - Set network classifier
2022-08-24 20:34:00,540 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network classifier from file classifiers.cosine_classifier_with_weight_generator with opts: {'num_classes': 64, 'num_features': 640, 'scale_cls': 10, 'learn_scale': True, 'global_pooling': True}
2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - CosineClassifierWithWeightGeneration()
2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - Set network classifier_aux
2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network classifier_aux from file classifiers.convnet_plus_classifier with opts: {'convnet_type': 'wrn_block', 'convnet_opt': {'num_channels_in': 640, 'num_channels_out': 640, 'num_layers': 4, 'stride': 2}, 'classifier_opt': {'classifier_type': 'cosine', 'num_channels': 640, 'scale_cls': 10.0, 'learn_scale': True, 'num_classes': 4, 'global_pooling': True}}
2022-08-24 20:34:00,816 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
2022-08-24 20:34:00,816 - bf3s.algorithms.algorithm - INFO   - ConvnetPlusClassifier(
  (layers): Sequential(
    (0): WideResnetBlock(
      (block): Sequential(
        (Block): NetworkBlock(
          (layer): Sequential(
            (0): BasicBlock(
              (convResidual): Sequential(
                (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu2): ReLU(inplace=True)
                (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              )
              (convShortcut): Conv2d(640, 640, kernel_size=(1, 1), stride=(2, 2), bias=False)
            )
            (1): BasicBlock(
              (convResidual): Sequential(
                (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): ReLU(inplace=True)
                (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu2): ReLU(inplace=True)
                (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              )
              (convShortcut): Sequential()
            )
            (2): BasicBlock(
              (convResidual): Sequential(
                (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): ReLU(inplace=True)
                (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu2): ReLU(inplace=True)
                (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              )
              (convShortcut): Sequential()
            )
            (3): BasicBlock(
              (convResidual): Sequential(
                (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): ReLU(inplace=True)
                (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu2): ReLU(inplace=True)
                (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
              )
              (convShortcut): Sequential()
            )
          )
        )
        (BN): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (ReLU): ReLU(inplace=True)
      )
    )
    (1): Classifier(
      (layers): CosineClassifier(num_channels=640, num_classes=4, scale_cls=10.0 (learnable=True), normalize_x=True, normalize_w=True, bias=False)
    )
  )
)
2022-08-24 20:34:00,821 - bf3s.algorithms.algorithm - INFO   - Initialize criterion[loss]: CrossEntropyLoss with options: None
2022-08-24 20:34:00,822 - bf3s.algorithms.algorithm - INFO   - ==> Loading criterions to device: cuda
2022-08-24 20:34:00,822 - bf3s.algorithms.algorithm - INFO   - ==> Loading tensors to device: cuda
2022-08-24 20:34:00,962 - bf3s.algorithms.algorithm - INFO   - ==> Loading networks to device: cuda
2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: feature_extractor
2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: classifier
2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: classifier_aux
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - Training epoch [  1 /  26]
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▎                                                             | 199/1000 [35:37<1:20:28,  6.03s/it]2022-08-24 21:17:56,249 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.2066', 'loss_total': '5.3655', 'Accuracy': '1.6797', 'AccuracyRot': '40.5254'}
 40%|███████████████████████████████▌                                               | 399/1000 [55:29<58:49,  5.87s/it]2022-08-24 21:37:48,318 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1953', 'loss_total': '5.3542', 'Accuracy': '1.6172', 'AccuracyRot': '41.2666'}
 60%|██████████████████████████████████████████████                               | 599/1000 [1:15:21<40:07,  6.00s/it]2022-08-24 21:57:40,442 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1900', 'loss_total': '5.3489', 'Accuracy': '1.6198', 'AccuracyRot': '41.5859'}
 80%|█████████████████████████████████████████████████████████████▌               | 799/1000 [1:34:52<19:36,  5.85s/it]2022-08-24 22:17:11,494 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1859', 'loss_total': '5.3448', 'Accuracy': '1.5781', 'AccuracyRot': '41.8672'}
100%|████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:54:23<00:05,  5.85s/it]2022-08-24 22:36:42,316 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1828', 'loss_total': '5.3417', 'Accuracy': '1.5391', 'AccuracyRot': '42.0699'}
100%|████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:54:28<00:00,  6.87s/it]
2022-08-24 22:36:42,321 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '1.1828', 'loss_total': '5.3417', 'Accuracy': '1.5391', 'AccuracyRot': '42.0699'}
2022-08-24 22:36:45,884 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-24 22:36:45,884 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:44<00:00,  1.05s/it]
2022-08-24 22:45:30,847 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-24 22:45:30,848 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - Training epoch [  2 /  26]
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-24 22:45:31,122 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:21<1:18:10,  5.86s/it]2022-08-24 23:04:58,476 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8979', 'loss_total': '5.0568', 'Accuracy': '1.8203', 'AccuracyRot': '49.8711'}
 40%|████████████████████████████████▋                                                 | 399/1000 [38:52<58:36,  5.85s/it]2022-08-24 23:24:29,101 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8896', 'loss_total': '5.0485', 'Accuracy': '1.7109', 'AccuracyRot': '50.1729'}
 60%|█████████████████████████████████████████████████                                 | 599/1000 [58:22<39:06,  5.85s/it]2022-08-24 23:43:59,678 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8866', 'loss_total': '5.0454', 'Accuracy': '1.6276', 'AccuracyRot': '50.0618'}
 80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:17:53<19:36,  5.85s/it]2022-08-25 00:03:30,281 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8848', 'loss_total': '5.0437', 'Accuracy': '1.6133', 'AccuracyRot': '50.1372'}
100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:24<00:05,  5.86s/it]2022-08-25 00:23:00,991 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8839', 'loss_total': '5.0428', 'Accuracy': '1.6062', 'AccuracyRot': '50.1906'}
100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:29<00:00,  5.85s/it]
2022-08-25 00:23:00,992 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8839', 'loss_total': '5.0428', 'Accuracy': '1.6062', 'AccuracyRot': '50.1906'}
2022-08-25 00:23:04,586 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 00:23:04,586 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:45<00:00,  1.05s/it]
2022-08-25 00:31:49,689 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '5.5160', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
NaN or Inf found in input tensor.
2022-08-25 00:31:49,690 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '5.5160', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 00:31:49,690 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - Training epoch [  3 /  26]
2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 00:31:49,692 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:27<1:18:32,  5.88s/it]2022-08-25 00:51:23,095 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8803', 'loss_total': '5.0392', 'Accuracy': '1.5703', 'AccuracyRot': '50.6738'}
 40%|████████████████████████████████▋                                                 | 399/1000 [39:04<58:56,  5.88s/it]2022-08-25 01:11:00,168 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0387', 'Accuracy': '1.5781', 'AccuracyRot': '50.5850'}
 60%|█████████████████████████████████████████████████                                 | 599/1000 [58:41<39:20,  5.89s/it]2022-08-25 01:30:37,293 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0387', 'Accuracy': '1.5599', 'AccuracyRot': '50.4466'}
 80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:18<19:43,  5.89s/it]2022-08-25 01:50:14,362 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0386', 'Accuracy': '1.5391', 'AccuracyRot': '50.4712'}
100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:55<00:05,  5.89s/it]2022-08-25 02:09:51,432 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.5266', 'AccuracyRot': '50.4121'}
100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:38:01<00:00,  5.88s/it]
2022-08-25 02:09:51,433 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.5266', 'AccuracyRot': '50.4121'}
2022-08-25 02:09:55,060 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 02:09:55,060 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:43<00:00,  1.05s/it]
2022-08-25 02:18:38,811 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
NaN or Inf found in input tensor.
2022-08-25 02:18:38,812 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 02:18:38,812 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - Training epoch [  4 /  26]
2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 02:18:38,814 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:25<1:18:24,  5.87s/it]2022-08-25 02:38:09,951 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8806', 'loss_total': '5.0395', 'Accuracy': '1.6406', 'AccuracyRot': '50.6211'}
 40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:47,  5.87s/it]2022-08-25 02:57:44,382 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8802', 'loss_total': '5.0391', 'Accuracy': '1.6211', 'AccuracyRot': '50.7100'}
 60%|█████████████████████████████████████████████████                                 | 599/1000 [58:34<39:14,  5.87s/it]2022-08-25 03:17:18,876 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8802', 'loss_total': '5.0391', 'Accuracy': '1.5547', 'AccuracyRot': '50.6224'}
 80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:08<19:39,  5.87s/it]2022-08-25 03:36:53,327 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.6055', 'AccuracyRot': '50.5508'}
100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:43<00:05,  5.87s/it]2022-08-25 03:56:27,810 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.5609', 'AccuracyRot': '50.4820'}
100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:48<00:00,  5.87s/it]
2022-08-25 03:56:27,811 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.5609', 'AccuracyRot': '50.4820'}
2022-08-25 03:56:31,390 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 03:56:31,390 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:43<00:00,  1.05s/it]
2022-08-25 04:05:15,021 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
NaN or Inf found in input tensor.
2022-08-25 04:05:15,022 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - Training epoch [  5 /  26]
2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-25 04:05:15,024 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 04:05:15,024 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:25<1:18:23,  5.87s/it]2022-08-25 04:24:46,183 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8810', 'loss_total': '5.0398', 'Accuracy': '1.4141', 'AccuracyRot': '50.5742'}
 40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:50,  5.87s/it]2022-08-25 04:44:20,688 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8805', 'loss_total': '5.0393', 'Accuracy': '1.4414', 'AccuracyRot': '50.4531'}
 60%|█████████████████████████████████████████████████                                 | 599/1000 [58:34<39:14,  5.87s/it]2022-08-25 05:03:55,117 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8801', 'loss_total': '5.0390', 'Accuracy': '1.4661', 'AccuracyRot': '50.4648'}
 80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:08<19:41,  5.88s/it]2022-08-25 05:23:29,643 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0388', 'Accuracy': '1.5000', 'AccuracyRot': '50.4761'}
100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:43<00:05,  5.87s/it]2022-08-25 05:43:04,090 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0387', 'Accuracy': '1.4969', 'AccuracyRot': '50.5203'}
100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:49<00:00,  5.87s/it]
2022-08-25 05:43:04,091 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0387', 'Accuracy': '1.4969', 'AccuracyRot': '50.5203'}
2022-08-25 05:43:07,710 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 05:43:07,710 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:46<00:00,  1.05s/it]
2022-08-25 05:51:53,727 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
NaN or Inf found in input tensor.
2022-08-25 05:51:53,728 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - Training epoch [  6 /  26]
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 05:51:53,730 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:25<1:18:26,  5.88s/it]2022-08-25 06:11:24,909 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8808', 'loss_total': '5.0397', 'Accuracy': '1.6875', 'AccuracyRot': '50.5273'}
 40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:48,  5.87s/it]2022-08-25 06:30:59,362 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.6445', 'AccuracyRot': '50.4277'}
 60%|█████████████████████████████████████████████████                                 | 599/1000 [58:32<39:15,  5.87s/it]2022-08-25 06:50:32,443 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8795', 'loss_total': '5.0384', 'Accuracy': '1.5911', 'AccuracyRot': '50.5195'}
 80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:07<19:40,  5.87s/it]2022-08-25 07:10:06,814 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8794', 'loss_total': '5.0382', 'Accuracy': '1.5820', 'AccuracyRot': '50.5464'}
100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:41<00:05,  5.87s/it]2022-08-25 07:29:41,270 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8792', 'loss_total': '5.0381', 'Accuracy': '1.5641', 'AccuracyRot': '50.5418'}
100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:47<00:00,  5.87s/it]
2022-08-25 07:29:41,271 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8792', 'loss_total': '5.0381', 'Accuracy': '1.5641', 'AccuracyRot': '50.5418'}
2022-08-25 07:29:44,885 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 07:29:44,885 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:45<00:00,  1.05s/it]
2022-08-25 07:38:30,437 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
NaN or Inf found in input tensor.
2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - Training epoch [  7 /  26]
2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
2022-08-25 07:38:30,440 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
 20%|███████████████▉                                                                | 199/1000 [19:23<1:18:18,  5.87s/it]2022-08-25 07:58:00,266 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  7][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8805', 'loss_total': '5.0394', 'Accuracy': '1.6953', 'AccuracyRot': '50.3262'}
 34%|███████████████████████████▌             

MiniImageNet: CC model with rotation prediction self-supervision and exploiting unlabeled images from tiered-MiniImageNet.

To train and test the WRN-28-10 based CC few-shot model with rotation prediction self-supervision which also exploits (with the semi-supervised learning setting) unlabeled images from tiered-MiniImageNet, run (you can skip the training step if you have downloaded the pre-trained model):

# Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 64.03% with confidence interval +/- 0.46%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 80.68% with confidence interval +/- 0.33%

MiniImageNet: CC model with relative patch location prediction self-supervision.

To train and test the WRN-28-10 based Cosine Classifier few-shot model with relative patch location prediction self-supervision run (you can skip the training step if you have downloaded the pre-trained model):

# Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 60.70% with confidence interval +/- 0.47%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 77.61% with confidence interval +/- 0.33%

CIFAR-FS: CC model with rotation prediction self-supervision.

To train and test the WRN-28-10 based Cosine Classifier few-shot model with rotation prediction self-supervision and rotation augmentations run (you can skip the training step if you have downloaded the pre-trained model):

# Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=1 --num_episodes=5000
# Expected 5-way classification accuracy: 75.38% with confidence interval +/- 0.31%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=5 --num_episodes=5000
# Expected 5-way classification accuracy: 87.25% with confidence interval +/- 0.21%

ImageNet-FS: CC model with rotation prediction self-supervision.

Instructions for training and testing the CC few-shot model with rotation prediction self-supervision on the ImageNet based few-shot benchmark [1, 2].

(1) To train the ResNet10 based Cosine Classifier few-shot model with rotation prediction self-supervision and rotation augmentations run:

# Run from the BF3S directory
$ python scripts/train_fewshot.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --num_workers=8

You can skip the above training step if you have downloaded the pre-trained model.

(2) Extract and save the ResNet10 features (with the above model) from images of the ImageNet dataset:

# Run from the BF3S directory
# Extract features from the validation image split of the Imagenet.
$ python scripts/save_features_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --split='val'
# Extract features from the training image split of the Imagenet.
$ python scripts/save_features_imagenet.py with config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --split='train'

The features will be saved on ./datasets/feature_datasets/ImageNet/ResNet10CosineClassifierRotSelfsupervision.

(4) Test the model:

# Run from the BF3S directory
# Test the CC+Rot model on the 1-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=1 --bias_novel=0.8
# ==> Top 5 Accuracies:      [Novel: 46.43 | Base: 93.52 | All 57.88]

# Test the CC+Rot model on the 2-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=2 --bias_novel=0.75
# ==> Top 5 Accuracies:      [Novel: 57.80 | Base: 93.52 | All 64.76]

# Test the CC+Rot model on the 5-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=5 --bias_novel=0.7
# ==> Top 5 Accuracies:      [Novel: 69.67 | Base: 93.52 | All 72.29]

# Test the CC+Rot model on the 10-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=10 --bias_novel=0.65
# ==> Top 5 Accuracies:      [Novel: 74.64 | Base: 93.52 | All 75.63]

# Test the CC+Rot model on the 20-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=20 --bias_novel=0.6
# ==> Top 5 Accuracies:      [Novel: 77.31 | Base: 93.52 | All 77.40]

Note that here, to evaluate the model trained with the ./config/ImageNet/ResNet10CosineClassifierRotSelfsupervision.py config file, we used a different config file named ./config/ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval.py. Also, the --bias_novel term specifies a multiplicative bias for the classification scores of the novel classes. Its purpose is to balance the classification scores of the base and novel classes (necessary since the classifiers for those two different classes are trained in different ways and different stages). It only affects the All classification accuracy metrics. The used bias values were tuned on the validation split.