Pipeline: (1) patch tiling & feature extraction, (2) prototype learning by K-Means clustering, (3) prototype graph construction, (4) inference with a trained checkpoint. Utility scripts for H&E-mIF slide registration and attention mapping to single-cell spatial data are also included.
conda env create -f environment.yml
conda activate biopredictYou also need to provide:
- Encoder weights (e.g. UNI) for Step 1.
- Your trained model checkpoint (e.g.
checkpoint.pth) for Step 4. We provided a checkpoint file for ESCC biopsy.
Expected input/output layout when running the pipeline:
project/
├── annotation.csv # input: sample annotation
├── he/ # input: WSIs (mrxs)
├── weights/pytorch_model.bin # input: encoder weights
│
├── settings.json, datasets.json # Step 1: slideflow project config
├── tfrecords/ # Step 1: normalized patches + coordinates
├── pt_files/ # Step 1: feature extraction output
│ ├── <slide>.pt # Step 1: patch features
│ ├── <slide>.index.npz # Step 1: patch coordinate index
│ ├── bags_config.json
│ └── geojson/<slide>.geojson # Step 1: patch-coordinate GeoJSONs
│
├── prototype/
│ ├── weights_<n_proto>.pkl # Step 2: learned prototypes
│ ├── Phenotypes_GRAPH_DIRECTORY/ # Step 3: full spatial graphs
│ ├── Phenotypes_LargestComponent_Graph_DIRECTORY/ # Step 3: largest connected components
│ └── PP_LargestComponent_Graph_DIRECTORY/ # Step 3: graphs with prototype assignments
│
└── results/ # Step 4: inference output
├── predictions.csv # Step 4: per-slide predicted probability
└── cohort_patch_attention.csv # Step 4: per-patch coordinates + attention scores
python 1_patch_encoder.py \
--project_root /path/to/project \
--slides /path/to/WSIs \
--annotation /path/to/annotation.csv \
--tile_px 224 \
--mpp 0.5 \
--encoder uni \
--weights /path/to/uni/pytorch_model.bin \
--output_dir /path/to/pt_filesTile size in microns is computed as round(tile_px * mpp) (defaults: 224 px, 0.5 mpp); set --tile_um directly to override.
The annotation file may contain any number of columns, but it must contain the following headers at minimum:
- patient: patient identifiers.
- slide: slide identifiers (without the file extension).
- category: slide label.
- set: dataset split designation ('train', 'val', or 'test').
This step creates tfrecords and pt_files directories under --output_dir. The tfrecords directory contains normalized patches and their spatial coordinates, while pt_files contains patch feature embeddings.
Per-slide patch coordinates are also exported as square-polygon GeoJSON files into --geojson_dir (default: <output_dir>/geojson).
Learn prototypes by K-Means clustering over the patch features of training slides:
python 2_prototype_extractor.py \
--annotation /path/to/annotation.csv \
--feature_dir /path/to/pt_files \
--output_dir /path/to/prototype \
--n_proto 16This step creates a weights_<n_proto>.pkl under --output_dir. Only slides marked as train in the set column of the annotation CSV are used for clustering.
Build a spatial graph per slide from the patch features, keep the largest connected component, and assign each patch to its nearest prototype learned in Step 2 (runs on all slides, not only training ones):
python 3_phenotype_graph.py \
--feature_dir /path/to/pt_files \
--output_dir /path/to/prototype \
--prototypes /path/to/prototype/weights_<n_proto>.pklThis step creates following directories under --output_dir:
- Phenotypes_GRAPH_DIRECTORY: full spatial patch graphs.
- Phenotypes_LargestComponent_Graph_DIRECTORY: largest connected component of each graph.
- PP_LargestComponent_Graph_DIRECTORY: largest-component graphs with prototype assignments.
Run inference on the prototype graphs from Step 3. Mount your own trained weights via --checkpoint:
python 4_infer.py \
--annotation /path/to/annotation.csv \
--graph_dir /path/to/PP_LargestComponent_Graph_DIRECTORY \
--checkpoint /path/to/checkpoint.pth \
--prototypes /path/to/weights_<n_proto>.pkl \
--output_dir /path/to/results--clinical_cols is used to specify feature coloumns in annotation CSV (default: cT cN Grade Age Sex).
This step outputs following files in --output_dir:
- predictions.csv: per-slide predicted probability and label
- cohort_patch_attention.csv: per-patch coordinates, prototype assignment, global attention score, and per-clinical-feature attention scores
Cross-modal registration with VALIS, plus warping of an H&E slide and its patch coordinate GeoJSON onto the mIF reference slide:
python 5_he_mihc_registration.py \
--slides /path/to/WSIs \
--reference_img /path/to/reference_img \
--warp_img /path/to/warp_img \
--geojson /path/to/annotations.geojson \
--attention_csv /path/to/cohort_patch_attention.csv \
--output_dir /path/to/resultsThe warped image is saved as <output_dir>/registered_slides/registered.ome.tif, and the warped GeoJSON as <output_dir>/registered_slides/registered.geojson.
--attention_csv is optional. When provided (the cohort_patch_attention.csv from step 4), each GeoJSON polygon is matched to its corresponding patch by coordinates, and the attention attributes (prototype, global_attention, per-clinical-feature attention) are attached to the polygon properties before warping, so the registered GeoJSON carries them.
Assign each cell from single-cell spatial transcriptomics (e.g. CosMx) to its nearest registered patch tile, and transfer the patch-level attention attributes onto cells:
Rscript 6_smi_attention.R \
--tiles /path/to/registered.geojson \
--meta /path/to/meta.csv \
--output_dir /path/to/meta_patch.csv--meta is the meta data CSV file from spatial data. Cell coordinates are read from --x_col / --y_col (default: x_FOV_px / y_FOV_px).