Search Papers | Poster Sessions | All Posters
Poster A59 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Repeated Exemplar Leakage in EEG Category Decoding
Jack Kilgallen1 (), Barak Pearlmutter1, Jeffrey Siskind2; 1Maynooth University, 2Purdue University
Within neuroimaging research, it is a common practice to perform multiple trials using a single stimulus when working with noisy modalities such as electroencephalography (EEG). For many types of analyses, this practice is unproblematic. However, when attempting to decode object category information from EEG signals (category decoding), we show that this practice can lead to a form of leakage that can inflate a model's performance when exemplars are shared across the training and test sets. We demonstrate this phenomenon by training several existing EEG decoding models on a dataset of EEG recordings from human subjects where multiple trials were recorded for each object within a category. We also develop a statistical framework to quantify the extent of this leakage. Our results reveal that per 1% increase above chance in the category decoding accuracy of a model trained on a dataset with repeated exemplars, the model's true generalization accuracy only increases by approximately 0.66%. This raises concerns about the validity of several EEG category decoding studies, and may have implications for brain computer interface (BCI) applications being developed on the basis of these studies.
Keywords: EEG machine learning leakage decoding