Papers › A Negative-Control Protocol for Clinical EEG Foundation-Model Benchmarks: Dataset...
A Negative-Control Protocol for Clinical EEG Foundation-Model Benchmarks: Dataset Identity and External-Cohort Stress Testing
Marzieh Zare
Title, abstract, authors and date from arXiv's metadata (CC0); this paper is not in the Papers with Code archive (frozen 2025-07-28).
EEG foundation-model gains may depend on cohort, montage, or probe design. We evaluated five models on five tasks across four benchmark datasets plus Korean CAUEEG, using subject-disjoint validation where identifiers exist. CAUEEG is recording-level with an annotated no-overlap held-out sensitivity. On matched CAUEEG normal/mild cognitive impairment/dementia classification (1,187 recordings), classical features reached 0.734 macro-AUROC (enhanced sensitivity: 0.736), versus BIOT-bipolar16 0.677, CBraMod 0.669, and REVE 0.568. The annotated no-overlap held-out subset preserved the classical-over-REVE ordering (0.717 versus 0.565). All five encoders decoded dataset identity at 1.000 before and after in-fold PCA-50; label permutations collapsed to chance and balanced subsamples remained at 1.000. This establishes dataset membership, not a causal site, geography, or population effect. A matched fully randomly initialized encoder was descriptively higher than pretrained REVE on CAUEEG (0.667 versus 0.570), and correct- versus scrambled-source-label LoRA runs yielded numerically similar AUROCs in unmatched descriptive sensitivities, not label-effect estimates or equivalence tests. On CHB-MIT cross-subject ictal detection, REVE reached 0.793 AUROC, versus 0.739 for the best tested enhanced nonlinear comparator, 0.691 for fully random initialization, and 0.505 for raw-signal random features. The paired REVE-minus-enhanced-comparator difference was +5.38 percentage points (95% CI -0.36 to +11.22), so comparator superiority remains unresolved; an amplitude-aware comparator also cannot be reconstructed from the retained normalized inputs. We distill montage matching, patient-overlap checks, stronger comparators, and representation controls into a reporting protocol for clinical EEG foundation-model studies.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Syntology holds the repository link but has not harvested or run code from it.
Results from the paper
The Papers with Code archive ends with its 2025-07-28 snapshot. This paper's arXiv identifier, 2607.24519, was issued in July 2026, after that date, so the archive has no leaderboard rows for it.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections