Course content

In this course, we will cover state-of-the-art methods and techniques for the integration and interpretation of data within and across modalities. The course will focus on hybrid audio-visual and text models for various multimodal applications using machine learning and deep learning techniques. Actual topics related to multimodal semantic and contextual data analysis and their applications in audio-visual and text domains will be covered and may include but are not limited to:

- Multimodal architectures, frameworks, and tools

- Multimodal data representation

- Embeddings and Transformers

- Multimodal data fusion

- Semantic and Contextual understanding

-- Case study on multimodal sentiment analysis

-- Case study on clinical natural language processing

-- Case study on speaker identification and recognition

Learning outcome

On completion of this course the students will have the following skills, knowledge, and general competence:

Knowledge

- Possess advanced knowledge within the area of multimodal data analysis

- Be able to identify semantic and contextual information across modalities

- Become familiar with frameworks and tools

- Get acquainted with solutions to challenging applications and open issues

Skills

- Be able to critically review and analyze existing models

- Be able to use and apply relevant methods and techniques across modalities

- Be able to propose innovative solutions to assigned problems

General Competence

- Improved understanding of the domain

- Exposure to various tools and techniques

- Exposure to open and challenging research topics

Learning methods and activities

We will have seminars using a blended learning approach with a mix of conventional lectures and flipped Classroom and in-class activities. The students will work individually or in groups and are provided with reading material.

• There will be lectures by the course instructor and guest lectures by invited experts.
• Student/groups presentations on selected topics

Compulsory requirements:

Each student/group needs to make a short introductory presentation on the topic and provide a detailed presentation of the work at the end of the course.

Compulsory assignments

Mandatory project report

Compulsory assignments

• Project Report

Specific conditions

Recommended previous knowledge

Good knowledge of machine learning/deep learning and python programming

Required previous knowledge

Python programming and previous courses on AI/machine learning or deep learning, such as PROG2051 - Artificial Intelligence

Course materials

Research papers, online tutorial/GitHub links, and lecture slides. A selection of research papers will be presented at the start of the course. Research papers and other relevant teaching material used in the seminars will be made available electronically.