Towards Expressive Perception And Generation In Human Computer Conversational Interaction

The construction of a natural interactive human-computer interaction has become an integral component of intelligent system development, constituting a core subject within the field of human-computer interaction. Grounded in the utilization of human perception theories, this study proposes a robust method for speech emotion recognition and a model for inferring user emotional state changes, thereby achieving a hu-man-computer interaction experience characterized by both listening and articulating, and conveying sentiments effectively. Simultaneously, focusing on the audio-visual modalities within human-computer interaction, this research explores methods for con-structing interactive feedback in auditory and visual modalities within real hu-man-computer dialog scenarios. This aims to synchronize textual, verbal, and visual expressions in human-computer interaction, enhancing its naturalness, improving user experience, and augmenting satisfaction and pleasure during interaction. The research contributions are as follows: Introducing a method that robustly identifies user emotional states in real hu-man-computer speech dialogue scenarios through the utilization of local and global attention mechanisms. This method generates robust and distinguishable representa-tions of speech emotion, thereby enhancing the robustness and accuracy of speech emotion recognition systems in authentic dialogue scenarios.