Admission requirements
The course is MSc level and intended for all MSc Computer Science Specializations. In particular BioInformatics and Computer Science. Students must be highly proficient in Python programming and writing reports and have an interest in interdisciplinary Biodiversity and Ecology topics and machine learning. Students are expected to be comfortable with the implementation, training, and testing of machine learning.
Description
This course introduces the fundamentals of multimodal models and their application in ecology for biodiversity monitoring. It focuses on leveraging heterogeneous data -such as images, audio, and biological measurements- to enhance the performance of automated monitoring systems. Key topics include the design, integration, and analysis of the components that constitute multimodal machine learning models.
Biodiversity monitoring is essential for understanding species' presence and ecosystem health. Traditional observation methods, while reliable, are time-consuming and prone to human bias. Advances in remote sensing, mobile technology, and machine learning have revolutionized data collection and analysis, enabling large-scale, multi-season monitoring. This course explores modern approaches to biodiversity assessment, focusing on the integration of diverse data modalities through machine learning to enhance ecological insights and support conservation efforts.
In a series of lectures, we analyse the theoretical aspects of multimodal models along the line of their application for biodiversity monitoring, and we introduce concepts of biodiversity, ecology, and data availability for biodiversity monitoring projects. Subjects will use the theoretical knowledge to: illustrate the innovations and implication of multimodality in research, connect to current topics in biodiversity research, and to develop their own model for multimodality.
The course consists of lectures, reading assignments using recent papers at the state of the art connected to the topics of the lectures, and practical assignment using dataset provided for image and audio, student choice code-editor software and environments and "hands-on" coding multimodal architectures from scratch or based on models at the state of the art. The practical part of the course is concluded with a report on the practical assignment and a presentation of the project and the results, where the communication skills are evaluated.
Course objectives
Explain the core principles of multimodal ML in ecological contexts;
Critically analyse research papers and communicate results effectively;
Preprocess and integrate audio and image data for biodiversity tasks;
Design, implement and evaluate basic multimodal architectures.;
Present the project implementation choices, challenges, and results in written and oral communication for an expert audience.
Timetable
The most recent timetable can be found at the Computer Science (MSc) student website.
In MyTimetable, you can find all course and programme schedules, allowing you to create your personal timetable. Activities for which you have enrolled via MyStudyMap will automatically appear in your timetable.
Additionally, you can easily link MyTimetable to a calendar app on your phone, and schedule changes will be automatically updated in your calendar. You can also choose to receive email notifications about schedule changes. You can enable notifications in Settings after logging in.
Questions? Watch the video, read the instructions, or contact the ISSC helpdesk.
Note: Joint Degree students from Leiden/Delft need to combine information from both the Leiden and Delft MyTimetables to see a complete schedule. This video explains how to do it.
Mode of instruction
Lectures
Case study sessions (paper reading)
Presentation of own work (including a written report, and oral presentation following a template )
Course load
Hours of study: 168 (= 6 EC)
Lectures: 48
Presentation and preparations: 30
At home project work: 30
Report: 60
Assessment method
The deadlines are strict and linked to the submission in Brightspace. Publication of each assignment is announced via BrightSpace. If you experience problems in retrieving these files, please call in assistance from the TA's. The grade for a non-submitted assignment is 0.
- Reading assessments - assessed as individual
- Practical assessment: 2.1. Coding of the project - assessed as team 2.2. Report - assessed as individual 2.3. Presentation - assessed as team
The assignments assessed as a team have one grade that is the same for all the member of the team.
Grading is based on:
- Presence, assignments completion and active participation in discussion (10%)
- Reading assignments positive assessment: (10%)
- Contribution to a project: code, presentation, report (80%) * annotation of the code, execution of the code and reproducibility of the results (20%) * final presentation (15%) * final report (45%)
The final mark is the weighted average of the above as indicated.
If an assignment or a presentation is not completed, the resulting grade is a 0. There will be no retakes for the assignments and the presentations. The retake for the final report is an one-time resubmission with an improved version. The final grade can only be sufficient if the weighted average grade is at least a 5.5.
Reading list
Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Association for Computing Machinery, 2024, https://doi.org/10.1145/3656580
Multimodal Machine Learning:A Survey and Taxonomy, Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency, PAMI, 2019,
https://people.ict.usc.edu/~gratch/CSCI534/Readings/Baltrusaitis-MMML-survey.pdf
Additional Materials that will be announced during the course
Registration
The course is registered as: 4343MMEBX
As a student, you are responsible for enrolling on time through MyStudyMap.
In this short video, you can see step-by-step how to enrol for courses in MyStudyMap.
Extensive information about the operation of MyStudyMap can be found here.
There are two enrolment periods per year:
Enrolment for the fall opens in July
Enrolment for the spring opens in December
See this page for more information about deadlines and enrolling for courses and exams.
Note:
It is mandatory to enrol for all activities of a course that you are going to follow.
Your enrolment is only complete when you submit your course planning in the ‘Ready for enrolment’ tab by clicking ‘Send’.
Not being enrolled for an exam/resit means that you are not allowed to participate in the exam/resit.
Contact
Lecturer: Rita Pucci
Remarks
Software
Starting from the 2024/2025 academic year, the Faculty of Science will use the software distribution platform Academic Software. Through this platform, you can access the software needed for specific courses in your studies. For some software, your laptop must meet certain system requirements, which will be specified with the software. It is important to install the software before the start of the course. More information about the laptop requirements can be found on the student website.