A study of cross-dataset generalization on liver vessel segmentation improved by a topological loss

Abstract: Our research addresses the critical task of localizing liver vascularity for medical applications such as surgical planning and intraoperative navigation. We conduct a comprehensive and unbiased evaluation of contemporary segmentation architectures for liver vessel segmentation, comparing UNet-based, mask-based, and foundational models. Our analysis emphasizes cross-dataset generalization by assessing model performance on multiple datasets. Notably, we achieve strong generalization by integrating region-based and topologically based Dice loss functions. This approach substantially improves cross-domain generalization, yielding clDice scores of 0.7132 on the IRCAD dataset and 0.6704 on the MSD dataset, even when these datasets were not included in the training set. Additionally, training with a mixed dataset further increases the MSD Dice score to 0.7472.