September 23rd-27th 2025, meet us at  Daejeon for MICCAI 2025!

Introduction

In medical imaging, Deep Learning (DL) models are often tasked with delineating structures or abnormalities within complex anatomical structures, such as tumors, blood vessels, or organs. Uncertainty arises from the inherent complexity and variability of these structures, leading to challenges in precisely defining their boundaries. This uncertainty is further compounded by interrater variability, as different medical experts may have varying opinions on where the true boundaries lie. DL models must grapple with these discrepancies, leading to inconsistencies in segmentation results across different annotators and potentially impacting diagnosis and treatment decisions.

Addressing interrater variability in DL for medical segmentation involves the development of robust algorithms capable of capturing and quantifying uncertainty, as well as standardizing annotation practices and promoting collaboration among medical experts to reduce variability and improve the reliability of DL-based medical image analysis. Interrater variability poses significant challenges in the field of DL for medical image segmentation.

This challenge is designed to promote awareness of the impact uncertainty has on clinical applications of medical image analysis. In our last-year edition, we proposed a competition based on modeling the uncertainty of segmenting three abdominal organs, namely kidney, liver and pancreas, focusing on organ volume as a clinical quantity of interest. This year, we go one step further and propose to segment pancreatic pathological structures, namely Pancreatic Ductal Adenocarcinoma (PDAC), with the clinical goal of understanding vascular involvement, a key measure of tumor resectability. In this above context, uncertainty quantification is a much more challenging task, given the wildly varying contours that different PDAC instances show.

This year, we will provide a richer dataset, in which we start from an already existing dataset of clinically verified contrast-enhanced abdominal CT scans with a single set of manual annotations (provided by the PANORAMA organization, which we invite to be part of our challenge), and make an effort to construct four extra manual annotations per PDAC case. In this way, we will assemble a unique dataset that creates a notable opportunity to analyze the impact of multi-rater annotations in several dimensions, e.g. different annotation protocols or different annotator experiences, to name a few.



Timeline















Dataset

The challenge cohort consists of 125 CT scans selected from the PANORAMA challenge dataset. Selection prioritized scans with manually generated labels, excluding those with automated annotations. Preference was also given to cases with conclusive diagnostic tests (e.g., pathology, cytology, histopathology). To ensure real-world representativeness, lesion sizes were assessed to cover a broad range of cases, while patient demographics, including sex and age, were considered to minimize bias.

The target age distribution is as follows: below 50 years (5%), 50–59 years (15%), 60–69 years (20%), 70–79 years (30%), and 80–89 years (30%). Sex distribution aims for 40-50% females and 50-60% males. For tumor location, approximately 60-70% of cases involve the head of the pancreas, 15-25% the body, and 10-15% the tail.

A preliminary visual review was conducted before submitting the scans for segmentation to ensure the tumor's location, size, and clinical relevance, maintaining the dataset's representativeness. Each CT will receive four additional annotations from radiologists at Universitätsklinikum Erlangen, Hospital de Sant Pau, and Hospital de Mataró. Additionally, existing labels from the PANORAMA dataset (veins, arteries, and pancreas) will be utilized for comprehensive evaluation in the challenge.

Training Phase cohort:
40 CT scans the respective annotations will be given. It is encouraged to leverage publicly available external data annotated by multiple raters. The idea of giving a small amount of data for the training set and giving the opportunity of using a public dataset for training is to make the challenge more inclusive, giving the option to develop a method by using data that is in anyone's hands. Furthermore, by using this data to train and using other data to evaluate, it makes it more robust to shifts and other sources of variability between datasets.

  The data coming from the PANORAMA Batch 1 (https://zenodo.org/records/13715870), Batch 2 (https://zenodo.org/records/13742336), and Batch 3 (https://zenodo.org/records/11034011), cannot be used. Batch 4 (https://zenodo.org/records/10999754) may be used.

The training dataset will be uploaded uploaded in Zenodo.

Find our evaluation and baseline code from previous edition in our Github: https://github.com/SYCAI-Technologies/curvas-challenge

Validation Phase cohort:
5 CT scans will be used for the validation phase.

Test Phase cohort:
80 CT scans will be used for evaluation.

Both validation and testing CT scans cohorts will not be published until the end of the challenge.



Ranking and Prices

Top five performing methods will be announced publicly. Winners will be invited to present their methods and results in the challenge event hosted in MICCAI 2025.

Two members of the participating team can be qualified as author (one must be the person that submits the results). The participating teams may publish their own results separately only after the organizer has published a challenge paper and always mentioning the organizer's challenge paper.

Training Set Release

MICCAI 2025