Berzelius Common Datasets

To avoid data duplication and save hard drive space, we provide access to a selection of public datasets frequently used in AI/ML research. The datasets are available read-only under COMMON_DATASETS=/proj/common-datasets.

Please refer to the List of Common Datasets on Berzelius for the information of version control and license.

Users are encouraged to contact us to request corrections, updates, or the addition of new datasets.

• AlphaFold Genetic Databases
• COCO
• ImageNet
• MNIST
• SMHI IFCB plankton
• SYKE-plankton_IFCB_2022
• SYKE-plankton_IFCB_Utö_2021
• Waymo Open Dataset
• WHOI-Plankton
• List of Common Datasets on Berzelius

AlphaFold Genetic Databases

AlphaFold needs multiple genetic (sequence) databases to run:

• BFD,
• MGnify,
• PDB70,
• PDB (structures in the mmCIF format),
• PDB seqres – only for AlphaFold-Multimer,
• UniRef30 (FKA UniClust30),
• UniProt – only for AlphaFold-Multimer,
• UniRef90.

The dataset is available under $COMMON_DATASETS/AlphaFold.

COCO

The COCO (Common Objects in Context) dataset is a large-scale object detection, segmentation, and captioning dataset.

The dataset is available under $COMMON_DATASETS/COCO.

ImageNet

ImageNet is a large and widely used dataset in the field of computer vision, particularly in tasks involving image classification, object detection, and other types of visual recognition tasks. We provide the datasets for ImageNet Large-scale Visual Recognition Challenge (ILSVRC) 2012, including

• train: 1,281,167 training images in 1,000 categories
• val: 50,000 validation images

We also provide the training and validation images in both LMDB and TFRecord formats.

Please open the ImageNet site, find the terms of use (http://image-net.org/download), copy them, replace the needed parts with your name, send us an email including the terms with your name - thereby confirming you agree to the these terms. Once you do this, we can grant you access to the copy of the dataset on the cluster.

The dataset is available under $COMMON_DATASETS/ImageNet.

MNIST

MNIST is a handwritten digit database used for image processing and machine learning algorithms.

Four files are available:

• train-images-idx3-ubyte: training set images
• train-labels-idx1-ubyte: training set labels
• t10k-images-idx3-ubyte: test set images
• t10k-labels-idx1-ubyte: test set labels

The dataset is available under $COMMON_DATASETS/MNIST.

SMHI IFCB plankton

This repository includes three datasets of manually annotated plankton images by phytoplankton experts at the Swedish Meteorological and Hydrological Institute (SMHI).

The dataset is available under $COMMON_DATASETS/SMHI-IFCB-Plankton.

SYKE-plankton_IFCB_2022

The SYKE-plankton_IFCB_2022 dataset consists of approximately 63,000 images representing 50 different classes of phytoplankton, collected using the Imaging FlowCytobot (IFCB) from various locations in the Baltic Sea. These images were manually annotated by expert taxonomists and are used to develop and evaluate classification methods for phytoplankton recognition.

The dataset is available under $COMMON_DATASETS/SYKE-plankton_IFCB_2022.

SYKE-plankton_IFCB_Utö_2021

The SYKE-plankton_IFCB_Utö_2021 dataset is a collection of approximately 150,000 images of phytoplankton, classified into 50 distinct categories, with an additional set of about 94,000 unclassifiable images. The dataset was collected using an Imaging FlowCytobot (IFCB) at the Utö Atmospheric and Marine Research Station in the Baltic Sea during 2021.

The dataset is available under $COMMON_DATASETS/SYKE-plankton_IFCB_Utö_2021.

Waymo Open Dataset

The Waymo Open Dataset is a publicly available dataset provided by Waymo, focused on autonomous driving technology. This dataset is designed to advance research and development in the field of autonomous driving by providing high-quality, diverse, and large-scale data collected from Waymo's fleet of autonomous vehicles.

To get access to the dataset, you need to:

• Register at waymo.com/open and agree to the terms.
• Agree not to prepare, initiate, assert, or otherwise support any claim against Waymo or any authorized licensee of the dataset for violation of any of your rights protecting the distributed work.

Once you do this, send us an email and we can grant you access to the copy of the dataset on the cluster.

The dataset is available under $COMMON_DATASETS/Waymo.

WHOI-Plankton

WHOI-Plankton is a comprehensive dataset of annotated plankton images developed by researchers at the Woods Hole Oceanographic Institution (WHOI). The dataset contains over 3.5 million images of microscopic marine plankton, categorized into 103 classes. These images are used primarily for developing and evaluating visual recognition models in plankton classification.

The dataset is available under $COMMON_DATASETS/WHOI-Plankton.

List of Common Datasets on Berzelius