On-line recognition of emotions via electroencephalography
Keywords:
Emotion Recognition, Brain-Computer Interface, Electroencephalography, On-line ProcessingAbstract
Automated pattern recognition of brain signals can bring about new experiences, enhancing applications in a wide array of areas. One of its fields of study is the recognition of emotions via electroencephalography (EEG), which shows exclusive advantages compared to other methods. However, research with brain-computer interfaces (BCI) is usually structured in two sequential stages: data collection and data analysis. These stages leave a gap in the perspective of a functional system in a production environment since the practitioner needs to wait a considerable length of time until they can see the results of the current activity. An on-line classification system of emotions (positive, neutral, and negative) was developed using open resources in this work. Five machine learning models were trained with the SEED IV dataset, which is labeled with different emotions. The models were trained and tested using nested cross-validation and grid search to obtain the best hyperparameters. The algorithm implementation in Python was integrated with the OpenBCI software to capture the EEG signals, process them, and command the simulations. The best average accuracy obtained for a single subject was 76.19%, and the average accuracy for all subjects was 57.07%. The average execution time for signal processing and prediction, combined, was around one millisecond, which demonstrates the potential for applications with real-time characteristics.
Downloads
References
R. P. Rao, Brain-computer interfacing: an introduction. Cambridge University Press, 2013.
L. R. Wilson, “Ethical Issues of Brain Computer Interfaces (BCI),” in Proceedings of the 2013 IACAP, pp. 1–18, 2013.
M. L. Oldoni, T. Catecati, E. J. Legal, M. G. G. Ferreira, and A. R. G. Ramirez, “Measuring working memory usage on aphasic individuals and analysis of emotional valence through electroencephalography,” Computer Methods and Programs in Biomedicine Update, vol. 1, 2021.
A. R. García Ramírez, J. F. Da Silva, A. C. R. Savall, T. Catecati, and M. G. G. Ferreira, “User’s emotions and usability study of a brain-computer interface applied to people with cerebral palsy,” Technologies, vol. 6, no. 1, 2018.
W.-L. Zheng, W. Liu, Y. Lu, B.-L. Lu, and A. Cichocki, “EmotionMeter: A Multimodal Framework for Recognizing Human Emotions,” IEEE Transactions on Cybernetics, vol. 49, pp. 1110–1122, mar 2019.
G. Liberati, S. Federici, and E. Pasqualotto, “Extracting neurophysiological signals reflecting users’ emotional and affective responses to BCI use: A systematic literature review,” NeuroRehabilitation, vol. 37, p. 341–358, Oct. 2015.
E. P. Torres, E. A. Torres, M. Hernández-Álvarez, and S. G. Yoo, “EEG-Based BCI Emotion Recognition: A Survey,” Sensors, vol. 20, no. 18, 2020.
Z. Lan, Y. Liu, O. Sourina, and L. Wang, “Real-time EEG-based user’s valence monitoring,” in 2015 10th International Conference on Information, Communications and Signal Processing (ICICS), pp. 1–5, IEEE, dec 2015.
X. Hou, Y. Liu, O. Sourina, and W. Mueller-Wittig, “CogniMeter: EEG-based Emotion, Mental Workload and Stress Visual Monitoring,” in 2015 International Conference on Cyberworlds (CW), pp. 153–160, IEEE, oct 2015.
Y.-J. Liu, M. Yu, G. Zhao, J. Song, Y. Ge, and Y. Shi, “Real-Time Movie-Induced Discrete Emotion Recognition from EEG Signals,” IEEE Transactions on Affective Computing, vol. 9, pp. 550–562, oct 2018.
D. Iacoviello, A. Petracca, M. Spezialetti, and G. Placidi, “A real-time classification algorithm for EEG-based BCI driven by self-induced emotions,” Computer Methods and Programs in Biomedicine, vol. 122, no. 3, pp. 293–303, 2015.
S. Taran and V. Bajaj, “Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method,” Computer Methods and Programs in Biomedicine, vol. 173, pp. 157–165, 2019.
M. Zangeneh Soroush, K. Maghooli, S. K. Setarehdan, and A. M. Nasrabadi, “Emotion recognition through EEG phase space dynamics and Dempster-Shafer theory,” Medical Hypotheses, vol. 127, pp. 34–45, 2019.
S. Hwang, M. Ki, K. Hong, and H. Byun, “Subject-Independent EEG-based Emotion Recognition using Adversarial Learning,” in 2020 8th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–4, IEEE, feb 2020.
W.-C. L. Lew, D. Wang, K. Shylouskaya, Z. Zhang, J.-H. Lim, K. K. Ang, and A.-H. Tan, “EEG-based Emotion Recognition Using Spatial-Temporal Representation via Bi-GRU,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 116–119, IEEE, jul 2020.
P. Zhong, D. Wang, and C. Miao, “EEG-Based Emotion Recognition Using Regularized Graph Neural Networks,” IEEE Transactions on Affective Computing, pp. 1–1, 2020.
E. Lozano-Monasor, M. T. López, A. Fernández-Caballero, and F. Vigo-Bustos, “Facial expression recognition from webcam based on active shape models and support vector machines,” in Ambient Assisted Living and Daily Activities, pp. 147–154, Springer International Publishing, 2014.
G. Trigeorgis, F. Ringeval, R. Brueckner, E. Marchi, M. A. Nicolaou, B. Schuller, and S. Zafeiriou, “Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5200–5204, IEEE, mar 2016.
T. S. Rached and A. Perkusich, “Emotion recognition based on brain-computer interface systems,” Brain-computer interface systems-Recent progress and future prospects, pp. 253–270, 2013.
S. Koelstra, C. Muhl, M. Soleymani, Jong-Seok Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “DEAP: A Database for Emotion Analysis Using Physiological Signals,” IEEE Transactions on Affective Computing, vol. 3, pp. 18–31, jan 2012.
A. Mehrabian, “Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament,” Current Psychology, vol. 14, pp. 261–292, dec 1996.
R. Plutchik, “The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice,” American Scientist, vol. 89, no. 4, pp. 344–350, 2001.
M. Christ, N. Braun, J. Neuffer, and A. W. Kempa-Liehr, “Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh – A Python package),” Neurocomputing, vol. 307, pp. 72–77, sep 2018.
OpenBCI, “OpenBCI,” 2013.
M. K. Islam, A. Rastegarnia, and Z. Yang, “Methods for artifact detection and removal from scalp EEG: A review,” Clinical Neurophysiology, vol. 46, pp. 287–305, nov 2016.
