Histopathology Slide Pre-processing Pipeline

HS2P is an open-source project largely based on CLAM tissue segmentation and patching code.

🛠️ Installation

System requirements: Linux-based OS (e.g., Ubuntu 22.04) with Python 3.11+ and Docker installed.

We recommend running the script inside a container using the latest hs2p image from Docker Hub:

docker pull waticlems/hs2p:latest
docker run --rm -it \
    -v /path/to/your/data:/data \
    waticlems/hs2p:latest

Replace /path/to/your/data with your local data directory.

Alternatively, you can install hs2p via pip:

pip install hs2p

Slide tiling

1. Create a .csv file containing paths to the desired slides. Optionally, you can provide paths to pre-computed tissue masks under the 'mask_path' column

   wsi_path,mask_path
   /path/to/slide1.tif,/path/to/mask1.tif
   /path/to/slide2.tif,/path/to/mask2.tif
   ...
   

2. Create a configuration file

   A good starting point is to look at the default configuration file under hs2p/configs/default.yaml where parameters are documented.

3. Kick off slide tiling

   python3 -m hs2p.tiling --config-file </path/to/config.yaml>

Tile sampling

1. Create a .csv file containing paths to the desired slides & associated annotation masks:

   wsi_path,mask_path
   /path/to/slide1.tif,/path/to/mask1.tif
   /path/to/slide2.tif,/path/to/mask2.tif
   ...
   

2. Create a configuration file

   A good starting point is to look at the default configuration file under hs2p/configs/default.yaml where parameters are documented.

3. Kick off tile sampling

   python3 -m hs2p.sampling --config-file </path/to/config.yaml>

Output structure

Both tiling.py and sampling.py produce a similar output structure in the specified output directory.

Coordinates

The coordinates/ folder contains a .npy file for each successfully processed slide.
This file stores a numpy array of shape (num_tiles, 8) containing the following information for each tile:

1. x: x-coordinate of the tile at level 0
2. y: y-coordinate of the tile at level 0
3. contour_index: index of the contour containing the tile (useful for masking non-tissue content)
4. target_tile_size: requested tile size (in pixels)
5. target_spacing: spacing at which the user requested the tile (in microns per pixel)
6. tile_level: pyramid level at which the tile was extracted
7. resize_factor: ratio between tile_size_resized and the requested tile size (target_tile_size), useful for resizing when loading the tile
8. tile_size_resized: size of the tile at the extraction level (tile_level), which may differ from the requested tile size (target_tile_size) if the target spacing was not available
9. tile_size_lv0: tile size scaled to the slide's level 0

Visualization (optional)

If visualize is set to true, a visualization/ folder is created containing low-resolution images to verify the results:

• mask/: visualizations of the provided tissue (or annotation) mask
• tiling/ (for tiling.py) or sampling/ (for sampling.py): visualizations of the extracted or sampled tiles overlaid on the slide. For sampling.py, this includes subfolders for each category defined in the sampling parameters (e.g., tumor, stroma, etc.)

These visualizations are useful for double-checking that the tiling or sampling process ran as expected.

Process summary

• process_list.csv: a summary file listing each processed slide, indicating whether processing was successful or failed. If a failure occurred, the traceback is provided to help diagnose the issue.