dinov2

Source code on GitHub.

README

Low-resource finetuning of foundation models beats state-of-the-art in histopathology

This is the repository of Low-resource finetuning of foundation models beats state-of-the-art in histopathology which was accepted at ISBI 2024. It is a slightly adapted version of the original DINOv2, GitHub repository.

Finetuning can be compute efficient

We propose finetuning a DINOv2 ViT-S, which yields at least equal performance compared to CTransPath and RetCCL but in a fraction of domain specific training time. Performance is measured on three datasets: TCGA & CPTAC (WSI-level classification) and NCT-CRC (patch-level classification).

Loss and performance over time


Performance over time of finetuning a ViT-s with DINOv2: a) on NCT-CRC and evaluating on the external NCT- CRC testset on patch-level classification and b) on TCGA and testing on TCGA (5-fold cross-validation) and CPTAC (external testset) on WSI-level classification.

Data

For the finetuning process, we utilized histopathological data from two primary datasets:

TCGA (The Cancer Genome Atlas): Specifically, colorectal cancer (CRC) data from the cohorts COAD and READ were used. This dataset includes annotations of microsatellite instability (MSI).

- TCGA Dataset: The Cancer Genome Atlas Pan-Cancer analysis project

• NCT-CRC-100K: This dataset consists of 100,000 histological images of human colorectal cancer and healthy tissue.
- NCT-CRC-100K Dataset: 100,000 histological images of human colorectal cancer and healthy tissue

For testing purposes, we incorporated two additional external datasets:

• CPTAC (Clinical Proteomic Tumor Analysis Consortium): For more details, visit the CPTAC Data Portal. (Accessed: 10.11.2023)
- CPTAC Dataset: CPTAC Data Portal
• CRC-VAL-HE-7K: This dataset, similar to NCT-CRC-100K, was employed for testing purposes.
- CRC-VAL-HE-7K Dataset: 7180 histological images of human colorectal cancer and healthy tissue

We used the following testing pipeline for TCGA and CPTAC:

• Testing Pipeline:...