Multi-Scale Dual-Stream CNN–Mamba Hybrid Framework for Accurate Nuclei Segmentation in H&E Images

Abstract

Accurate nuclei segmentation in H&E-stained histopathology images is critical for quantitative tissue analysis and downstream computational pathology tasks. We propose MS-DS Mamba-Seg, a multi-scale dual-stream framework that integrates CNN-based local feature extraction with Mamba-based global context modeling to enhance nuclei segmentation. The Local Stream captures fine-grained morphological details such as nuclear boundaries and texture, while the Global Stream models long-range spatial dependencies across tissue regions. A novel Fusion Gate adaptively combines these complementary features, producing high-resolution and morphologically precise segmentation masks. Evaluated on the NuInsSeg dataset, MS-DS Mamba-Seg achieves a Dice score of 95.41%, outperforming baseline methods including Attention U-Net, UNETR, and SegMamba, while maintaining structural consistency and boundary accuracy. These results demonstrate the framework’s effectiveness in leveraging complementary local and global features for robust nuclei segmentation.