From Long Contexts to Short Windows: Fine-tuning Transformer on Short EEG Windows to Enable Real-time Neonatal Seizure Detection

Neonatal seizures are a global challenge and represent a significant neurological condition. Identifying and monitoring abnormal electrical activity in the brain is crucial for determining when a seizure occurs and for detecting the ongoing repetition or periodicity in electroencephalography (EEG), which is often performed by clinical experts and is time-consuming. The use of automated algorithms and deep learning convolutional neural networks (CNNs) has demonstrated significant progress and improved performance. However, CNNs frequently struggle to capture long-range dependencies, prompting the development of hybrid solutions that combine both local and global temporal modeling. Furthermore, despite using various window lengths, epoch sizes, and sampling frequencies in existing neonatal seizure detection models, several challenges limit the feasibility and effectiveness of detecting seizures with short EEG windows. These challenges include variability in EEG patterns, the non-stationary nature of EEG signals, artifacts and noise, subtle or non-convulsive seizures, imbalanced data, the absence of standardized EEG features, limited labeled data, and the necessity for generalization across different EEG systems. Additionally, reproducibility remains a significant obstacle, as many studies lack transparency and provide incomplete details. In this study, we adapt the hybrid Conformer and transformer through fine-tuning on neonatal EEG datasets to utilize their integrated local-global feature learning capabilities in short neonatal seizure EEG windows from multi-channel EEG recordings, thereby bridging the gap between long-context neural models and low-latency clinical applications. To promote open science and reproducibility of the neonatal EEG seizure detection model, we have released the full codebase, pre-trained models, and experiment configurations on GitHub under an open-source license. The findings from each model experiment, using the original dataset, the synthetic minority over-sampling technique (SMOTE), and cost-sensitive learning, yielded promising results, with the highest AUC achieved using SMOTE. Overall, our work represents a first step towards fine-tuning the transformer model using short EEG windows to enable real-time monitoring and detection. However, the model’s performance is limited to the neonatal intensive care unit dataset from Helsinki University Hospital, highlighting the need for comprehensive experiments using various datasets in future research.