Image Annotation Tasks

This category contains annotation task designs for computer vision research, including image classification, object detection, semantic segmentation, and medical and remote-sensing imagery.

Run these designs in Potato. See the image annotation documentation to configure bounding boxes, keypoints, and classification.

Subcategories

Aerial

Design	Description	Reference
bigearth-net	Multi-label land cover classification from Sentinel-2 imagery	Sumbul et al., IGARSS 2019
dota-aerial	Oriented bounding box detection in aerial images	Xia et al., CVPR 2018
xview	Large-scale overhead imagery object detection	Lam et al., arXiv 2018

Classification

Design	Description	Reference
cub-200-birds	CUB-200-2011 is a fine-grained bird image dataset of 11,788 photos across 200 species, each labeled with…	Wah et al., 2011
imagenet	Large-scale image classification following the ImageNet dataset	Deng et al., CVPR 2009
ms-coco	Object detection annotation following the MS COCO format	Lin et al., ECCV 2014
open-images	Large-scale image-level labeling following the Open Images dataset	Kuznetsova et al., IJCV 2020
pascal-voc	Multi-label object classification following the PASCAL Visual Object Classes challenge	Everingham et al., IJCV 2010
places365	Scene recognition and classification following the Places365 dataset	Zhou et al., IEEE TPAMI 2017

DocLayNet Document Layout Analysis

Design	Description	Reference
doclaynet-document-layout	Document layout analysis with bounding box annotations	Pfitzmann et al., KDD 2022

Driving

Design	Description	Reference
bdd100k	BDD100K is a 100,000-video driving dataset from UC Berkeley spanning detection, segmentation, lane…	Yu et al., CVPR 2020
kitti	Autonomous driving benchmark for object detection	Geiger et al., CVPR 2012

FLAIR: French Land Cover from Aerospace Imagery

Design	Description	Reference
flair-aerial-land-use	Land use and land cover classification from high-resolution aerial imagery	Garioud et al., NeurIPS 2023

Generation Eval

Design	Description	Reference
t2i-compbench	T2I-CompBench is a 6,000-prompt benchmark for compositional text-to-image generation	Huang et al., NeurIPS 2023

Human Pose

Design	Description	Reference
vitpose-keypoint-annotation	Annotate human body keypoints and classify pose categories in images	Template

Medical

Design	Description	Reference
camelyon-pathology	Pathology slide annotation for breast cancer metastasis detection	Ehteshami Bejnordi et al., JAMA 2017
chexpert	Multi-label classification of chest radiographs for 14 observations	Irvin et al., AAAI 2019
mimic-cxr	Large-scale chest radiograph classification based on MIMIC-CXR. Multi-label classification with 14…	Johnson et al., Scientific Data 2019

OmniDocBench Comprehensive Document Parsing

Design	Description	Reference
omnidocbench-document-parsing	Comprehensive document parsing annotation covering layout detection, text recognition, table structure,…	Ouyang et al., CVPR 2025

Segment Anything (SA-1B) Interactive Segmentation

Design	Description	Reference
sa1b-segment-anything	Interactive image segmentation annotation	Kirillov et al., ICCV 2023

Segmentation

Design	Description	Reference
ade20k	ADE20K is a scene parsing dataset from MIT CSAIL with dense pixel labels for objects, stuff, and parts	Zhou et al., CVPR 2017
cityscapes	Urban scene understanding with instance-level semantic labeling	Cordts et al., CVPR 2016
lip-human-parsing	Pixel-level human body part segmentation	Gong, Liang, Zhang, Shen & Lin, CVPR 2017

Specialized

Design	Description	Reference
celeba	CelebA is a face dataset of 202,599 celebrity images, each labeled with 40 binary attributes, identity,…	Liu et al., ICCV 2015
deepfashion	DeepFashion is a clothes recognition and retrieval dataset of 800,000+ images labeled with 50 categories,…	Liu et al., CVPR 2016
docbank	Document layout analysis benchmark	Li et al., COLING 2020
iwildcam	Camera trap image classification for wildlife monitoring	Beery et al., CVPR 2019
mvtec-ad	Anomaly detection and localization in industrial images	Bergmann, Fauser, Sattlegger & Steger, CVPR 2019
wikiart	Art classification by style, genre, and artist (Saleh & Elgammal, 2015)	Saleh & Elgammal, 2015

Visual Grounding

Design	Description	Reference
refcoco-expression	Visual grounding task where annotators draw bounding boxes around objects referred to by natural language…	Yu et al., ECCV 2016

Visual QA

Design	Description	Reference
textvqa-reading-in-images	TextVQA is a visual question answering dataset where reading scene text in the image is needed to answer	Singh et al., CVPR 2019
vqav2-visual-question-answering	VQA v2.0 is a balanced visual question answering benchmark of open-ended questions over COCO images, with…	Goyal et al., CVPR 2017

WTW Wired Table Structure Annotation

Design	Description	Reference
wtw-table-structure-annotation	Cell-grid structure annotation for photographed and scanned wired tables, based on the WTW (Wired Table in…	Long et al., ICCV 2021

xBD: Building Damage Assessment from Satellite Imagery

Design	Description	Reference
xbd-building-damage	Building damage assessment from pre- and post-disaster satellite imagery	Gupta et al., CVPR Workshops 2019

Quick Start

# Navigate to a specific task
cd image/classification/ms-coco

# Run with Potato
potato start config.yaml

Task Count

Total: 34 image annotation tasks across 16 subcategories