MULTICOM is the first Master’s degree programme in the data science of human multimodal communication. The programme focuses on the connection between language, thought, and the different modalities of communicative behaviour: gesture, gaze, facial expression, pose, speech articulation, prosody, and more. MULTICOM is an Erasmus Mundus joint degree, co-funded by the European Union and co-awarded by the universities of Murcia (Daedalus Lab) in Spain, Lund (Centre for Languages and Literature) in Sweden, and Friedrich-Alexander-Universität Erlangen-Nürnberg (Big Data Linguistics) in Germany. MULTICOM integrates humanities, multimodality, data science, and technology. Along with the relevant theories, methods, and statistical and computational techniques, students are trained to combine and influence the key technologies in multimodal data science, such as large language models, body pose detection, automatic speech analysis, eye tracking, motion capture, or articulography. While candidates are expected to have a strong background in language and related disciplines, only basic quantitative skills are required for admission. Capturing the intricacies of human communication requires integrating complex data into research questions about language, cognition, communication, emotion, social interaction, or culture. But profiles in those areas seldom have the necessary data-science training. They also tend to be unfamiliar with the sophisticated technologies required to analyse video and audio input from communicative exchanges. Moreover, it is quite rare for anyone, from either the sciences or the humanities, to possess language-related training beyond text or the analysis of a single modality, such as speech or gesture. MULTICOM bridges all three gaps because students are trained in a comprehensive approach to multimodal communication and are required to work in more than one modality, enabling them to engage rigorously with multiple communicative channels. Also, they will receive intensive training in statistics and computing tailored to their needs as multimodal data scientists. Finally, they get hands-on training in all key technologies for the study of multimodal communication, along with the opportunity to influence their development through collaboration with top players across industry and academia.
Semester 1: Introduction to multimodal data science September to December at the University of Murcia, Spain Welcome week, inauguration, social programme Thorough overview of multimodal data science Intensive training in basic or intermediate quantitative skills Basic professional skills Opportunities to pursue key topics in the field, participate in hands-on research, fill in any gaps on language and cognition, or develop further professional and entrepreneurial skills. Semester 2: The multimodality lab: cognition and learning January to June at Lund University, Sweden Welcome week, social programme Specialised training in laboratory-based methods and on theoretical background for research on multimodality in language, cognition, and learning Intermediate or advanced training in quantitative skills Data management, property rights, and ethics Opportunity to gain further expertise in hands-on experimental research projects, to delve into the connections between multimodality and natural and artificial intelligence, or pursue a key topic in the field Professional fair Semester 3: Machine learning and multimodal corpora September to January at FAU, Erlangen Campus, Germany Professional project (internship) of at least six weeks between June and October Specialised training in multimodal corpus linguistics, multimodal language models, and the creation, curation, analysis, and management of large-scale audiovisual data sets Advanced professional skills Opportunities to further master key technologies: machine-learning techniques, computer vision, speech analysis, and natural language understanding, or to pursue key topics in the field Conference at the end of term, co-organised by students and faculty Semester 4: Research and professional development January to August at location(s) of choice Students work on the MA thesis at one or two organisations, beneficiaries or associated partners, chosen by the student and the main supervisor, who must be from one of the three universities. Stays at non-academic partners, as well as professional training related to the thesis research, will be encouraged. Further professional training for both academic and industry careers Thesis defence (virtual) Graduation ceremony To get more detailed information about the modules taught, please visit https://multicom-em.eu/program/.