Decision-making content of an agent affected by emotional feedback provided by capture of human emotions through a Bimodal System
Affective computing allows for widening the view of the complex world in human-machine interaction through the comprehension of emotions, which allows an enriched coexistence of natural interactions between them. Corporal features such as facial expression, kinetics, structural components of the voice or vision, to mention just a few, provide us with valid information of how a human behaves. Among all the carriers of emotional information we may point out two, voice and facial gestures as holders of an ample potential for identifying emotions with a high degree of accuracy. This paper focuses on the development of a system that will track a humans affective state using facial expressions and speech signals with the purpose of modifying the actions of an autonomous agent. The system uses a fusion of two baseline unimodal classifiers based on bayes Net giving rise to a multi-classifier. The union of the three classifiers forms a bimodal scheme of emotion classification. The outputs from the baseline unimodal classifiers are combined together through a probability fusion framework applied in the general multi-classifier. The system classifies six universal basic emotions using audiovisual data extracted from the eNTERFACE05 audiovisual emotion database. The emotional information obtained could provide an agent with the basis for taking an affective decision. It is shown by experimental results that the proposed system can detect emotions with good accuracy achieving the change of the emotional behavior of the agent faced with a human.
Keywords: Affective Computing, Machine Learning, Adversarial Risk Analysis, Broaden and Build Theory, Facial Expression Recognition, Speech Emotion Recognition, Detection of Emotional Information, Emotional self-regulation.
Download Full-Text
ABOUT THE AUTHOR
Javier Francisco Guerrero Rázuri
is currently a researcher at the Department of Computer and System Sciences, Stockholm University. He was a researcher and Ph.D. student of Department of Statistics and Operations Research at Universidad Rey Juan Carlos and acting developer of the Affective computing department in AISoy Robotics. In 2003, he received with honors the Bachelor of electronic engineering degree at Peruvian University of Applied Sciences (UPC) in Perú. In 2007, he received a Master’s Degree in Business Management at IEDE - Business School of European University of Madrid. In 2008, he received a Master's Degree in Decision Systems Engineering at Universidad Rey Juan Carlos. From 2008, he was a researcher in several projects financed for the university and the European Union focused on Robotics and decision-making related to EU higher education. His research interests are related with, Affective computing, emotional decision making, Human-machine interaction, Robotics, Human Robotics, Robotics and Autonomous Systems, Neuroscience, Sentiment analysis, in order to reproducing behavior patterns similar to human and provide to the agents a type of emotional intelligence and improve the interaction experience making more close the loop human-robot emotional interaction.
Javier Francisco Guerrero Rázuri
is currently a researcher at the Department of Computer and System Sciences, Stockholm University. He was a researcher and Ph.D. student of Department of Statistics and Operations Research at Universidad Rey Juan Carlos and acting developer of the Affective computing department in AISoy Robotics. In 2003, he received with honors the Bachelor of electronic engineering degree at Peruvian University of Applied Sciences (UPC) in Perú. In 2007, he received a Master’s Degree in Business Management at IEDE - Business School of European University of Madrid. In 2008, he received a Master's Degree in Decision Systems Engineering at Universidad Rey Juan Carlos. From 2008, he was a researcher in several projects financed for the university and the European Union focused on Robotics and decision-making related to EU higher education. His research interests are related with, Affective computing, emotional decision making, Human-machine interaction, Robotics, Human Robotics, Robotics and Autonomous Systems, Neuroscience, Sentiment analysis, in order to reproducing behavior patterns similar to human and provide to the agents a type of emotional intelligence and improve the interaction experience making more close the loop human-robot emotional interaction.
