AutoQC

Zixuan Pan, Justin Sonneck, Dennis Nagel, Anja Hasenberg, Matthias Gunzer, Yiyu Shi, Jianxu Chen

Introduction

AutoQC is a benchmarking framework for automatic quality control of high-throughput microscopy images. It provides a benchmark dataset and a set of baseline methods for evaluating the performance of quality control algorithms.

Fig. 1: Overview of the benchmark dataset.

Benchmark Dataset Preparation

All datasets curated in this paper and split files are available at BioStudies. We also provide the pre-trained models for all baselines at Google Drive. Once downloaded, please unzip the files and place them in the data folder of this repository.

Installation

Before starting, we recommend creating a new conda environment and installing the required packages listed in requirements.txt. We test our methods on Python 3.9.7 and CUDA 11.8.

Create a Conda Environment

You can create and activate a new conda environment with the following commands:

conda create -n autoqc python=3.9.7
conda activate autoqc
pip install -r requirements.txt

Quick Start

We provide a jupyter notebook in the root directory of this repository, which can be used to run inference on the example data.

To download the benchmark dataset from BioStudies:

python download_autoqc_data.py --out ./data

Using the flags --only_train, --only_test, or --only_splits, you can download individual subsets.

Training and Evaluation

Configuration

1. Edit the configuration files in the configs folder to match your experiment setup:

• configs/config.yaml: Main configuration for training and evaluation.
• configs/datamodule/DNA.yaml: Set the correct paths for dataset splits.
• configs/experiment/Benchmark_Methods/*.yaml: Select and configure the baseline method you wish to use.
• pc_environment.env: Set the paths for logs and data storage.

To reproduce the baseline results from the paper, you only need to update the split paths in datamodule/DNA.yaml and set the log/data paths in pc_environment.env.

Training

Run the training script:

bash train.sh

This will start the training process using the selected configuration. After training completes, the model will automatically evaluate on the test set and save the results.

Evaluation

If you want to directly perform evaluation using a pre-trained model, follow these steps:

1. Download the desired pre-trained model checkpoint from the provided Google Drive link and place it in your workspace.
2. In configs/config.yaml, set the following options:

• onlyEval: True # This will skip training and run evaluation only
• load_checkpoint: <path_to_your_checkpoint.ckpt> # Specify the path to your downloaded checkpoint

Example snippet for config.yaml:

onlyEval: True
load_checkpoint: data/checkpoints/your_model.ckpt

Then run:

bash train.sh

The script will load the pre-trained model and perform evaluation on the test set, saving results and logs as configured.