Hybrid 3D CNN–transformer model for early brain tumor detection with multi-modal magnetic resonance imaging

Abstract

Accurate and early diagnosis of brain tumors using multi-modal magnetic resonance imaging (MRI) remains a critical challenge due to tumor heterogeneity and complex spatial representation. This study proposes a novel hybrid deep learning framework that integrates a 3D convolutional neural network (3D CNN) with swin transformer blocks and an attention-based feature fusion module (ABFFM). The model leverages multi-modal MRI inputs—T1, T1Gd, T2, and fluid-attenuated inversion recovery (FLAIR)—and features a dual-branch classification head for binary tumor detection and multi-label tumor sub-region classification: enhancing tumor (ET), tumor core (TC), and whole tumor (WT). Experiments conducted on the BraTS2023-GLI dataset demonstrate that the proposed model achieves a superior classification accuracy of 96.51%, with precision of 97.98%, recall of 97.04%, and F1-score of 97.61%, outperforming state-of-the-art methods. Furthermore, intrinsic attention weights offer interpretability by highlighting modality-specific contributions. The proposed model establishes a clinically promising approach for brain tumor analysis, with strong implications for early diagnosis and treatment planning.