Imagined speech decoding by learning consensus graph from RKHS-based multi-view EEG features

Abstract

Electroencephalogram (EEG)-based speech imagery has emerged as a novel brain–computer interface (BCI) paradigm, which holds promise for aiding individuals with speech disorders to achieve more intuitive communication; however, the imagined speech decoding performance is still far from the level required for real applications. Among the possible reasons, one is that there is still no agreement on which domain features are more closely related to imagined speech and the other is that EEG features in the original space may be not be sufficiently discriminative for decoding. To both ends, we propose to learn a Reproducible Kernel Hilbert Space (RKHS)-based consensus Graph from Multi-View (KGMV) EEG features for imagined speech decoding. Specifically, the complementary information from multiple-domain EEG features are sufficiently explored to avoid the information loss caused by single-domain features; besides, the nonlinear modeling capability of KGMV in terms of EEG data is enhanced by implicitly mapping multi-view EEG features into RKHS. Accordingly, a consensus graph is adaptively learned in the kernel space to capture the semantic connections of EEG samples. Experiments are conducted on a speech imagery EEG data set and the results depict that the proposed KGMV model demonstrates superior decoding performance compared to single-view models and several other state-of-the-art approaches.