On October 14, 2024, the 70th Zhi-Xin lecture hall was held in Room 117. This forum was jointly organized by the College of Electronic and Information Engineering, Tongji University and Shanghai Research Institute for Intelligent Autonomous Systems. Prof. Gerhard Rigoll from Technical University of Munich, Germany, was invited to give a talk on “Recent Developments in the Area of Multi-Modal Human-Machine Interaction at TU Munich”, which was moderated by Prof. Hao Zhang.
First, Prof. Gerhard Rigoll introduced general multimodal human-machine interaction channels (including speech, gestures, gaze, emotions, haptics, etc.) and some common multimodal human-machine interaction examples (such as mobile phones, vehicle driving, ticket vending machines, robots, etc.). Then, he introduced the latest research results of the team: face recognition using partial and occluded face information and low-resolution face image recognition, elaborated the design of the scheme and algorithms and demonstrated the effectiveness and advantages of the algorithms through validation. Prof. Gerhard Rigoll then discussed and demonstrated the validation of an algorithm for action recognition, including gait recognition, using graph convolutional neural networks. In addition, Prof. Gerhard Rigoll presented a multimodal recognition task in a multi-speaker activity detection scenario and pointed out that the deep learning approach in this scenario is not only used for unimodal recognition, but also for the fusion of audio-visual information in particular. Finally, Prof. Gerhard Rigoll showed that optimizing the recognition components of different modalities and learning their fusion strategies by employing machine learning methods can help to look into the future of human-machine interaction.
After the talk, Prof. Gerhard Rigollhad a cordial exchange and discussion with the participating teachers and students, and he also encouraged them to actively broaden their horizons, explore, discover, and solve new scientific problems based on his own experience. This talk further expanded the vision of our teachers and students and enhanced their knowledge and understanding of the field of human-machine interaction.
Written by Hao Zhang
Photographed by Haoyue Yang
