EEG Feature Robustness Under Cross-Dataset Distribution Shift Using Classical Models

EEG-based brain-computer interfaces (BCIs) frequently report high within-dataset classification accuracy, yet performance collapses when models are applied to new datasets recorded under different conditions. This paper investigates whether frequency-domain feature representations are more robust than time-domain statistical features under cross-dataset distribution shift. Using two publicly available motor imagery datasets—PhysioNet EEG Motor Imagery (109 subjects) and BCI Competition IV Dataset 2a (9 subjects)—we evaluate 9 feature-model combinations under strict zero-shot transfer in both directions. FFT magnitude features showed the smallest mean normalised generalisation gap (0.065), followed by band power (0.070) and time-domain features (0.071), though differences were modest. Critically, within-dataset accuracy was near chance for PhysioNet (53–56%), indicating that commonly reported high within-dataset accuracies do not necessarily reflect genuine cross-dataset generalisation. A 9× amplitude difference between recording systems was empirically confirmed as the primary barrier to cross-dataset transfer; log-transformed frequency features partially mitigate this shift.