Files in this item
Synch-Graph : multisensory emotion recognition through neural synchrony via graph convolutional networks
Item metadata
dc.contributor.author | Mansouri Benssassi, Esma | |
dc.contributor.author | Ye, Juan | |
dc.date.accessioned | 2020-02-11T13:30:03Z | |
dc.date.available | 2020-02-11T13:30:03Z | |
dc.date.issued | 2020-04-03 | |
dc.identifier | 264038993 | |
dc.identifier | 4cd7621a-4a04-48fd-ae52-41aff22a1092 | |
dc.identifier | 85099878650 | |
dc.identifier | 000667722801052 | |
dc.identifier.citation | Mansouri Benssassi , E & Ye , J 2020 , Synch-Graph : multisensory emotion recognition through neural synchrony via graph convolutional networks . in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-20) . Proceedings of the AAAI Conference on Artificial Intelligence , AAAI Press , pp. 1351-1358 , Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) , New York , New York , United States , 7/02/20 . https://doi.org/10.1609/aaai.v34i02.5491 | en |
dc.identifier.citation | conference | en |
dc.identifier.isbn | 9781577358350 | |
dc.identifier.issn | 2159-5399 | |
dc.identifier.other | ORCID: /0000-0002-2838-6836/work/69029067 | |
dc.identifier.uri | https://hdl.handle.net/10023/19444 | |
dc.description.abstract | Human emotions are essentially multisensory, where emotional states are conveyed through multiple modalities such as facial expression, body language, and non-verbal and verbal signals. Therefore having multimodal or multisensory learning is crucial for recognising emotions and interpreting social signals. Existing multisensory emotion recognition approaches focus on extracting features on each modality, while ignoring the importance of constant interaction and co- learning between modalities. In this paper, we present a novel bio-inspired approach based on neural synchrony in audio- visual multisensory integration in the brain, named Synch-Graph. We model multisensory interaction using spiking neural networks (SNN) and explore the use of Graph Convolutional Networks (GCN) to represent and learn neural synchrony patterns. We hypothesise that modelling interactions between modalities will improve the accuracy of emotion recognition. We have evaluated Synch-Graph on two state- of-the-art datasets and achieved an overall accuracy of 98.3% and 96.82%, which are significantly higher than the existing techniques. | |
dc.format.extent | 8 | |
dc.format.extent | 1038578 | |
dc.language.iso | eng | |
dc.publisher | AAAI Press | |
dc.relation.ispartof | Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-20) | en |
dc.relation.ispartofseries | Proceedings of the AAAI Conference on Artificial Intelligence | en |
dc.subject | QA75 Electronic computers. Computer science | en |
dc.subject | T Technology | en |
dc.subject | 3rd-DAS | en |
dc.subject | BDC | en |
dc.subject | R2C | en |
dc.subject | ~DC~ | en |
dc.subject.lcc | QA75 | en |
dc.subject.lcc | T | en |
dc.title | Synch-Graph : multisensory emotion recognition through neural synchrony via graph convolutional networks | en |
dc.type | Conference item | en |
dc.contributor.institution | University of St Andrews. School of Computer Science | en |
dc.identifier.doi | 10.1609/aaai.v34i02.5491 | |
dc.identifier.url | https://aaai.org/Library/conferences-library.php | en |
This item appears in the following Collection(s)
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.