Spatiotemporal Analysis of EEG Signal for Emotion Recognition
Loading...
Date
2025-11-04
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Electronics & Instrumentation Engineering, NITS
Abstract
Emotion plays an important role in the everyday life of a human being, starting from
cognitive abilities to having a general interaction. Therefore, understanding emotion has
significant implications for healthcare, including the treatment of depression, anxiety, and
mood disorders. Moreover, it aids in the assessment of an individual's behavioral and
cognitive abilities. As technology advancements progress, human-machine interaction has
also intensified. Consequently, an emotion-adaptive Brain Computer Interface (BCI) can
enhance general productivity and improve user experience in human-machine interactions,
facilitating various applications such as cognitive ability assessment, interactive gaming
design, experience-based multimedia content delivery, interactive robotics development,
healthcare service provision, and real-time patient monitoring. The basic premise of an
adaptive BCI is to utilize audio-visual or physiological sensors to detect signals sent by the
brain or body. Common instances of audio-visual sensors include human speech and facial
emotions. Although facial expression and speech are effective ways of communicating an
emotional state, it is important to validate the data to prevent misinterpretation, duplication,
or fraudulent manipulation, which may be effectively addressed by the use of physiological
sensors. While there are various physiological sensors available for emotion identification,
such as Galvanic Skin Response (GSR), Electrooculography (EOG), Electromyography
(EMG), Electrocardiogram (ECG), Respiration (RSP), and Electroencephalogram (EEG),
the EEG has attracted significant interest in the field of BCI emotion recognition due to its
non-invasive nature, and superior temporal resolution.
Hence, the thesis concentrates specifically on the EEG-based emotion recognition task for
establishing an emotion-adaptive BCI system and attempts to address some of the existing
challenges, such as subjective experience, high computation cost to deal with large data
volume, and channel overload for mobile practical applications. Moreover, the thesis
evaluates the effectiveness of several affect representation methodologies usually employed
in the EEG emotion recognition framework. Two experiments were performed to compare
the observed emotion clusters based on subjective ratings with the clustering of
corresponding physiological responses based on EEG characteristics (time, frequency, and
time-frequency features). The study aims to investigate whether distinct structures may be
identified in physiological signal characteristics, which are often categorized using
supervised approaches. Furthermore, the study provides insights about the existing current
methods for representing affect and evaluates the efficacy of discrete (assumes that there
are only a few basic emotions, and all the other emotions are a combination of these) and
dimensional models (quantitatively defines emotion and projects emotion in 2D or 3D space)
of affect representation.
Next, the effectiveness of various conventional feature extraction techniques utilized in EEG
based emotion identification systems has been evaluated. The study includes an extensive
range of EEG features from the time, frequency, time-frequency, and spatial domains. The
efficacy of the feature extraction methods has been assessed using three independent
machine learning algorithms and three different EEG emotion datasets. The study offers a
unique and distinct examination of various EEG features that will assist researchers in
selecting EEG features for emotion recognition.
The thesis explores a temporal window localization technique, which uses EEG band power
to identify and locate emotion-rich time periods in EEG data to enhance emotion
classification performance compared to prior studies where the entire data segment is
considered for feature extraction and further emotion classification. The study examines the
impact of the relative power ratio among all EEG bands on the selection of significant EEG
temporal windows. The study furthermore introduces a novel framework for feature
extraction using temporal segmentation. This framework is specifically applied to the
temporally localized EEG segments of importance. The proposed methodologies have been
evaluated via extensive tests on three benchmark datasets across four classification
scenarios, and over three machine learning approaches. The proposed temporal window
localization approach reduces the data volume typically required for feature extraction and
subsequent classification, hence providing a tool to reduce the computation cost.
Furthermore, the thesis presents an unsupervised spatiotemporal emotion localization
framework for selecting emotion-centric temporal windows and channels. A smaller EEG
data segment is used to construct the channel selection framework for faster and more
efficient computing. Additionally, an effective channel-feature extraction technique has been
presented to boost the efficiency of the emotion identification framework. The optimum
channel-feature extraction approach was tested for practical applicability using an EEG
emotion dataset collected over a wireless, portable, and commercially available EEG
headset. Results show great performance with few frontal and occipital lobe channels. The
outcome of the research highlights the proposed methodology's potential to advance the
development of robust, portable, and efficient EEG sensor devices for practical emotion
recognition BCI applications.