Papers › Transformer-Based Multimodal Fusion for Early Diagnosis of Alzheimer's Disease Using...
Transformer-Based Multimodal Fusion for Early Diagnosis of Alzheimer's Disease Using Structural MRI And PET
Yuanwang Zhang, Kaicong Sun, Yuxiao Liu, Dinggang Shen
Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are both the widely used imaging modalities for early diagnosis of Alzheimer's disease (AD). Combining these two modalities allow using both anatomical and metabolic information for evaluating brain status. However, the commonly-used multimodal fusion strategy, i.e., through channel concatenation, cannot effectively exploit complementary information among these two modalities. To encourage effective information exchange between structural MRI (sMRI) and FDG-PET as used in our study for early AD diagnosis, we propose a novel transformer-based multimodal fusion framework. Specially, our proposed model composes of three parts: 1) Feature extraction based on adversarial training; 2) Feature fusion based on multimodal transformer through cross-attention mechanism; 3) Classification head based on full connection. By resorting to adversarial learning, the feature gap between two modalities becomes smaller, thus easing the cross-attention operation to achieve more effective fusion. In the experiment, we show that our model outperforms other representative models by a large margin.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections