{"title":"Electroencephalography-Based Intention Recognition and Consensus Assessment during Emergency Response","authors":"Siyao Zhu, Yifang Xu","volume":197,"journal":"International Journal of Computer and Systems Engineering","pagesStart":318,"pagesEnd":328,"ISSN":"1307-6892","URL":"https:\/\/publications.waset.org\/pdf\/10013100","abstract":"
After natural and man-made disasters, robots can bypass the danger, expedite the search, and acquire unprecedented situational awareness to design rescue plans. Brain-computer interface is a promising option to overcome the limitations of tedious manual control and operation of robots in the urgent search-and-rescue tasks. This study aims to test the feasibility of using electroencephalography (EEG) signals to decode human intentions and detect the level of consensus on robot-provided information. EEG signals were classified using machine-learning and deep-learning methods to discriminate search intentions and agreement perceptions. The results show that the average classification accuracy for intention recognition and consensus assessment is 67% and 72%, respectively, proving the potential of incorporating recognizable users\u2019 bioelectrical responses into advanced robot-assisted systems for emergency response.<\/p>","references":"[1]\tJ. Ayers, J. L. Davis, and A. Rudolph, \u201cNeurotechnology for biomimetic robots,\u201d p. 636, 2002.\r\n[2]\tN. Pezeshkian, H. G. Nguyen, and A. Burmeister, \u201cUnmanned ground vehicle radio relay deployment system for non-line-of-sight operations,\u201d undefined, Jan. 2007, doi: 10.21236\/ADA475525.\r\n[3]\tE. C. Leuthardt, G. Schalk, J. Roland, A. Rouse, and D. W. Moran, \u201cEvolution of brain-computer interfaces: Going beyond classic motor physiology,\u201d Neurosurg. Focus, vol. 27, no. 1, 2009, doi: 10.3171\/2009.4.FOCUS0979.\r\n[4]\tR. Richer, N. Zhao, J. Amores, B. M. Eskofier, and J. A. Paradiso, \u201cReal-time Mental State Recognition using a Wearable EEG,\u201d undefined, vol. 2018, pp. 5495\u20135498, Jul. 2018, doi: 10.1109\/EMBC.2018.8513653.\r\n[5]\tH. Adeli and S. Ghosh-Dastidar, \u201cAutomated EEG-based diagnosis of neurological disorders: inventing the future of neurology,\u201d p. 387, 2010.\r\n[6]\tJ. M. Fuster, \u201cFrontal lobe and cognitive development,\u201d J. Neurocytol., vol. 31, no. 3\u20135, pp. 373\u2013385, Mar. 2002, doi: 10.1023\/A:1024190429920.\r\n[7]\tD. R. Euston, A. J. Gruber, and B. L. McNaughton, \u201cThe role of medial prefrontal cortex in memory and decision making,\u201d Neuron, vol. 76, no. 6, pp. 1057\u20131070, Dec. 2012, doi: 10.1016\/J.NEURON.2012.12.002.\r\n[8]\tA. Rehman and Y. Al Khalili, \u201cNeuroanatomy, Occipital Lobe,\u201d StatPearls, Jul. 2021, Accessed: Nov. 11, 2021. (Online). Available: https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK544320\/.\r\n[9]\tR. Lindenberg and L. Scheef, \u201cSupramodal language comprehension: role of the left temporal lobe for listening and reading,\u201d Neuropsychologia, vol. 45, no. 10, pp. 2407\u20132415, 2007, doi: 10.1016\/J.NEUROPSYCHOLOGIA.2007.02.008.\r\n[10]\tL. Fogassi, P. F. Ferrari, B. Gesierich, S. Rozzi, F. Chersi, and G. Rizzolotti, \u201cNeuroscience: Parietal lobe: From action organization to intention understanding,\u201d Science (80-.)., vol. 308, no. 5722, pp. 662\u2013667, Apr. 2005, doi: 10.1126\/SCIENCE.1106138\/SUPPL_FILE\/FOGASSI_SOM.PDF.\r\n[11]\tD. Bzdok, G. Hartwigsen, A. Reid, A. R. Laird, P. T. Fox, and S. B. Eickhoff, \u201cLeft inferior parietal lobe engagement in social cognition and language,\u201d Neurosci. Biobehav. Rev., vol. 68, pp. 319\u2013334, Sep. 2016, doi: 10.1016\/J.NEUBIOREV.2016.02.024.\r\n[12]\tJ. M. Spielberg, J. L. Stewart, R. L. Levin, G. A. Miller, and W. Heller, \u201cPrefrontal Cortex, Emotion, and Approach\/Withdrawal Motivation,\u201d Soc. Personal. Psychol. Compass, vol. 2, no. 1, pp. 135\u2013153, Jan. 2008, doi: 10.1111\/J.1751-9004.2007.00064.X.\r\n[13]\tA. Maydeu-olivares, \u201cQuantitative Methods in Psychology Quantitative Methods in Psychology,\u201d Psychol. Bull., vol. 112, no. 1, pp. 155\u2013159, 1992.\r\n[14]\tM. Toscani, T. Marzi, S. Righi, M. P. Viggiano, and S. Baldassi, \u201cAlpha waves: a neural signature of visual suppression,\u201d Exp. brain Res., vol. 207, no. 3\u20134, pp. 213\u2013219, Dec. 2010, doi: 10.1007\/S00221-010-2444-7.\r\n[15]\tM. Zaepffel, R. Trachel, B. E. Kilavik, and T. Brochier, \u201cModulations of EEG Beta Power during Planning and Execution of Grasping Movements,\u201d PLoS One, vol. 8, no. 3, p. e60060, Mar. 2013, doi: 10.1371\/JOURNAL.PONE.0060060.\r\n[16]\tZ. Yaple, M. Martinez-Saito, N. Novikov, D. Altukhov, A. Shestakova, and V. Klucharev, \u201cPower of feedback-induced beta oscillations reflect omission of rewards: Evidence from an eeg gambling study,\u201d Front. Neurosci., vol. 12, Oct. 2018, doi: 10.3389\/FNINS.2018.00776\/FULL.\r\n[17]\tM. Gola, M. Magnuski, I. Szumska, and A. Wr\u00f3bel, \u201cEEG beta band activity is related to attention and attentional deficits in the visual performance of elderly subjects,\u201d Int. J. Psychophysiol., vol. 89, no. 3, pp. 334\u2013341, Sep. 2013, doi: 10.1016\/J.IJPSYCHO.2013.05.007.\r\n[18]\tX. Jia and A. Kohn, \u201cGamma Rhythms in the Brain,\u201d PLoS Biol., vol. 9, no. 4, Apr. 2011, doi: 10.1371\/JOURNAL.PBIO.1001045.\r\n[19]\tG. Chanel, J. Kronegg, D. Grandjean, and T. Pun, \u201cEmotion Assessment: Arousal Evaluation Using EEG\u2019s and Peripheral Physiological Signals,\u201d Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 4105 LNCS, pp. 530\u2013537, 2006, doi: 10.1007\/11848035_70.\r\n[20]\tE. Lew, R. Chavarriaga, S. Silvoni, and J. del R. Mill\u00e1n, \u201cDetection of self-paced reaching movement intention from EEGsignals,\u201d Front. Neuroeng., vol. 5, no. July, Jul. 2012, doi: 10.3389\/FNENG.2012.00013.\r\n[21]\tY. Wang and S. Makeig, \u201cPredicting Intended Movement Direction Using EEG from Human Posterior Parietal Cortex,\u201d Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5638 LNAI, pp. 437\u2013446, 2009, doi: 10.1007\/978-3-642-02812-0_52.\r\n[22]\tC. Babiloni et al., \u201cHuman movement-related potentials vs desynchronization of EEG alpha rhythm: a high-resolution EEG study,\u201d Neuroimage, vol. 10, no. 6, pp. 658\u2013665, 1999, doi: 10.1006\/NIMG.1999.0504.\r\n[23]\tI. H. Kim, J. W. Kim, S. Haufe, and S. W. Lee, \u201cDetection of braking intention in diverse situations during simulated driving based on EEG feature combination,\u201d J. Neural Eng., vol. 12, no. 1, p. 016001, Nov. 2014, doi: 10.1088\/1741-2560\/12\/1\/016001.\r\n[24]\tS. Ge et al., \u201cTemporal-Spatial Features of Intention Understanding Based on EEG-fNIRS Bimodal Measurement,\u201d IEEE Access, vol. 5, pp. 14245\u201314258, 2017, doi: 10.1109\/ACCESS.2017.2723428.\r\n[25]\tS.-H. Oh, \u201cSubject Independent Classification of Implicit Intention Based on EEG Signals,\u201d Int. J. Contents, vol. 12, no. 3, pp. 12\u201316, Sep. 2016, doi: 10.5392\/IJOC.2016.12.3.012.\r\n[26]\tJ. S. Kang, U. Park, V. Gonuguntla, K. C. Veluvolu, and M. Lee, \u201cHuman implicit intent recognition based on the phase synchrony of EEG signals,\u201d Pattern Recognit. Lett., vol. 66, pp. 144\u2013152, Nov. 2015, doi: 10.1016\/J.PATREC.2015.06.013.\r\n[27]\tX. Wang, Y. Huang, Q. Ma, and N. Li, \u201cEvent-related potential P2 correlates of implicit aesthetic experience,\u201d Neuroreport, vol. 23, no. 14, pp. 862\u2013866, Oct. 2012, doi: 10.1097\/WNR.0B013E3283587161.\r\n[28]\tW. Zhang, J. Jin, A. Wang, Q. Ma, and H. Yu, \u201cConsumers\u2019 Implicit Motivation of Purchasing Luxury Brands: An EEG Study,\u201d Psychol. Res. Behav. Manag., vol. 12, p. 913, 2019, doi: 10.2147\/PRBM.S215751.\r\n[29]\tM. Falkenstein, J. Hoormann, S. Christ, and J. Hohnsbein, \u201cERP components on reaction errors and their functional significance: a tutorial,\u201d Biol. Psychol., vol. 51, no. 2\u20133, pp. 87\u2013107, Jan. 2000, doi: 10.1016\/S0301-0511(99)00031-9.\r\n[30]\tA. F. Salazar-Gomez, J. Delpreto, S. Gil, F. H. Guenther, and D. Rus, \u201cCorrecting robot mistakes in real time using EEG signals,\u201d Proc. - IEEE Int. Conf. Robot. Autom., pp. 6570\u20136577, Jul. 2017, doi: 10.1109\/ICRA.2017.7989777.\r\n[31]\tC. Blais, D. M. Ellis, K. M. Wingert, A. B. Cohen, and G. A. Brewer, \u201cAlpha suppression over parietal electrode sites predicts decisions to trust,\u201d Soc. Neurosci., vol. 14, no. 2, pp. 226\u2013235, Mar. 2019, doi: 10.1080\/17470919.2018.1433717.\r\n[32]\tS. Oh, Y. Seong, S. Yi, and S. Park, \u201cNeurological Measurement of Human Trust in Automation Using Electroencephalogram,\u201d Int. J. Fuzzy Log. Intell. Syst., vol. 20, no. 4, pp. 261\u2013271, 2020, doi: 10.5391\/IJFIS.2020.20.4.261.\r\n[33]\tM. Wang, A. Hussein, R. F. Rojas, K. Shafi, and H. A. Abbass, \u201cEEG-Based Neural Correlates of Trust in Human-Autonomy Interaction,\u201d Proc. 2018 IEEE Symp. Ser. Comput. Intell. SSCI 2018, pp. 350\u2013357, Jan. 2019, doi: 10.1109\/SSCI.2018.8628649.\r\n[34]\tK. Akash, W.-L. Hu, N. Jain, and T. Reid, \u201cA Classification Model for Sensing Human Trust in Machines Using EEG and GSR,\u201d ACM Trans. Interact. Intell. Syst., vol. 8, no. 4, Mar. 2018, doi: 10.1145\/3132743.\r\n[35]\tA. Delorme and S. Makeig, \u201cEEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis,\u201d J. Neurosci. Methods, vol. 134, no. 1, pp. 9\u201321, Mar. 2004, doi: 10.1016\/J.JNEUMETH.2003.10.009.\r\n[36]\tM. Akin, \u201cComparison of Wavelet Transform and FFT Methods in the Analysis of EEG Signals,\u201d J. Med. Syst. 2002 263, vol. 26, no. 3, pp. 241\u2013247, Jun. 2002, doi: 10.1023\/A:1015075101937.\r\n[37]\tA. Yazdani, T. Ebrahimi, and U. Hoffmann, \u201cClassification of EEG signals using Dempster Shafer theory and a K-nearest neighbor classifier,\u201d 2009 4th Int. IEEE\/EMBS Conf. Neural Eng. NER \u201909, pp. 327\u2013330, 2009, doi: 10.1109\/NER.2009.5109299.\r\n[38]\tA. Subasi and E. Er\u00e7elebi, \u201cClassification of EEG signals using neural network and logistic regression,\u201d Comput. Methods Programs Biomed., vol. 78, no. 2, pp. 87\u201399, May 2005, doi: 10.1016\/J.CMPB.2004.10.009.\r\n[39]\tA. Sharmila and P. Geethanjali, \u201cDWT Based Detection of Epileptic Seizure from EEG Signals Using Naive Bayes and k-NN Classifiers,\u201d IEEE Access, vol. 4, pp. 7716\u20137727, 2016, doi: 10.1109\/ACCESS.2016.2585661.\r\n[40]\tD. Garrett, D. A. Peterson, C. W. Anderson, and M. H. Thaut, \u201cComparison of linear, nonlinear, and feature selection methods for EEG signal classification,\u201d IEEE Trans. Neural Syst. Rehabil. Eng., vol. 11, no. 2, pp. 141\u2013144, Jun. 2003, doi: 10.1109\/TNSRE.2003.814441.\r\n[41]\tA. Craik, Y. He, and J. L. Contreras-Vidal, \u201cDeep learning for electroencephalogram (EEG) classification tasks: A review,\u201d J. Neural Eng., vol. 16, no. 3, 2019, doi: 10.1088\/1741-2552\/AB0AB5.\r\n[42]\tY. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert, \u201cDeep learning-based electroencephalography analysis: a systematic review,\u201d J. Neural Eng., vol. 16, no. 5, p. 051001, Aug. 2019, doi: 10.1088\/1741-2552\/AB260C.\r\n[43]\tT. Gehr, \u201cElectroencephalogram (EEG) Classification of Known Vs Unknown Skills: Unpublished. Data from Jo Shattuck\u2019s Neural Substrates of Motor Learning the Effect of Real Time Haptic Feedback on Kinesthetic Awareness and Motor Skill Performance View project Electroencephalogram (EEG) Classification,\u201d 2017, Accessed: Nov. 12, 2021. (Online). Available: https:\/\/www.researchgate.net\/publication\/333853132.\r\n[44]\tV. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, and B. J. Lance, \u201cEEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces,\u201d J. Neural Eng., vol. 15, no. 5, Nov. 2016, doi: 10.1088\/1741-2552\/aace8c.\r\n[45]\tR. Schirrmeister, L. Gemein, K. Eggensperger, F. Hutter, and T. Ball, \u201cDeep learning with convolutional neural networks for decoding and visualization of EEG pathology,\u201d 2017 IEEE Signal Process. Med. Biol. Symp. SPMB 2017 - Proc., vol. 2018-January, pp. 1\u20137, Aug. 2017, doi: 10.1109\/SPMB.2017.8257015.","publisher":"World Academy of Science, Engineering and Technology","index":"Open Science Index 197, 2023"}