Multimodal Interaction

Code: INF13275M

6 ECTS

Duration: 15 weeks/156 hours

Scientific Area: Informatics

Teaching languages: Portuguese

Languages of tutoring support: Portuguese

Sustainable Development Goals

Educação de qualidade - Assegurar a educação inclusiva, e equitativa e de qualidade, e promover oportunidades de aprendizagem ao longo da vida para todos.

Learning Goals

The purpose of this UC is to study New Forms of Person-Machine Interaction, as well as the design of systems that combine multiple forms of interaction, providing the user with a more natural and intuitive experience. Students begin by learning basic concepts of interaction, studying models and techniques for implementing forms of interaction, and also for an integration of multiple interaction modalities.

Human-Machine Interaction Concepts
Interaction Modes
Speech and write as interface
Voice synthesis
Speech Recognition
Writing recognition
Tangible Interfaces
Computer vision: image processing methods (features, filters, edge), automatic detection, recognition and tracking
Architectures for Multimodal Interaction Systems and approaches to the combination of modalities
Perspectives of future development in the area of Human-Machine Interaction
Recent case studies with Deep Learning in Person-Machine or Environment-Machine interaction processes

Teaching Methods

The teaching methodology includes
lectures
laboratory classes

In the lectures students will learn theoretical concepts of multimodal interaction systems along with architectures for single modality interaction and for combining several modalities. Practical classes are dedicated to experiences with specific tools for one or more interaction methods.
The last classes are dedicated to implementing a group project where students apply techniques learned in the discipline to design a system with multimodal interaction.

The evaluation:
Project of high complexity (50%)
Written examination (50%)