Skip to content

albumentations-team/autoalbument-benchmarks

Repository files navigation

Benchmarks for AutoAlbument - AutoML for Image Augmentation.

Results

CIFAR-10 (Classification)

Augmentation strategy Top-1 Accuracy Top-5 Accuracy
Baseline 91.79 99.63
AutoAlbument 96.02 99.91

SVHN (Classification)

Augmentation strategy Top-1 Accuracy Top-5 Accuracy
Baseline 98.31 99.68
AutoAlbument 98.48 99.72

ImageNet (Classification)

  • Model: ResNet-50.
  • Baseline augmentation strategy:
    • Resize an image to 256x256 pixels.
    • Crop a random 224x224 pixels patch.
    • Apply Horizontal Flip with probability 0.5.
  • AutoAlbument augmentation strategy:
    • Resize an image to 256x256 pixels.
    • Crop a random 224x224 pixels patch.
    • Apply AutoAlbument augmentation policies.
  • Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy Top-1 Accuracy Top-5 Accuracy
Baseline 73.27 91.64
AutoAlbument 75.17 92.57

Pascal VOC (Semantic segmentation)

  • Model: DeepLab-v3-plus.
  • Baseline augmentation strategy:
    • Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
    • If required, pad an image to the size 256x256 pixels.
    • Apply Horizontal Flip with probability 0.5.
  • AutoAlbument augmentation strategy:
    • Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
    • If required, pad an image to the size 256x256 pixels.
    • Apply AutoAlbument augmentation policies.
  • Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy mIOU
Baseline 73.34
AutoAlbument 75.55

Cityscapes

  • Model: DeepLab-v3-plus.
  • Baseline augmentation strategy:
    • Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
    • If required, pad an image to the size 256x256 pixels.
    • Apply Horizontal Flip with probability 0.5.
  • AutoAlbument augmentation strategy:
    • Resize an image preserving its aspect ratio, so the longest size is 256 pixels.
    • If required, pad an image to the size 256x256 pixels.
    • Apply AutoAlbument augmentation policies.
  • Configs: AutoAlbument augmentation search | Baseline training | AutoAlbument training.
Augmentation strategy mIOU
Baseline 79.47
AutoAlbument 79.92

How to run the benchmarks

  1. Download datasets and put them in the following directory structure:
  2. Clone this repository.
  3. Run the run.sh script that will build a Docker image and train models using the following command:

./run.sh </path/to/data/directory> </path/to/outputs/directory>

e.g. ./run.sh ~/data ~/outputs

where

  • </path/to/data/directory> is a path to a directory that contains datasets (e.g., a directory that contains folders imagenet, pascal_voc, etc)
  • </path/to/outputs/directory> is a path to a directory that should contain outputs from a training pipeline, such as a CSV log with metrics and a checkpoint with the best model.

About

Benchmarks for AutoAlbument - AutoML for Image Augmentation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published