Thesis of Simon Forest
Subject:
Defense date: 16/09/2022
Advisor: Salima Hassas
Coadvisor: Mathieu Lefort
Summary:
This thesis focuses on multimodal merging on simulated topologies for use in an active perception context. As an example, humans receive dense information from multiple sensors and use various mechanisms to select and attend to only the relevant signals, for example by moving their eyes towards a target to see it better. Because of irregularities in sensory topologies (cf.\ fovea), actions can enhance perception, while extracting and merging data also helps choosing the best course of action. Similar needs are faced by artificial systems, e.g.\ social robots, albeit with their own set of physical constraints. This thesis proposes computational models for use in AI, taking inspiration from neuroscience case studies such as the superior colliculus, a subcortical structure involved in generating saccadic eye movements towards visual, auditory or multisensory stimuli.
When selecting information from multiple signals in a dynamic and multimodal setting, one needs a way to compute robust and reliable decisions. Decision-making in general has been tackled in either psychophysics or robotics using many different algorithms. One contribution of this thesis is to review and compare these algorithms, underlining their spatio-temporal properties, including feature merging, selective attention, etc. Among these models, dynamic neural fields (DNF) display some very interesting characteristics, including selective attention and data fusion depending on stimulus distance and precision. In another contribution, this thesis then makes use of DNF as a signal filtering and merging tool applied to multimodal fusion. This thesis shows how it can apply to model realistically occurences of the ventriloquist effect, a psychophysical effect of audio/visual stimulus localization capture. Then, in order to further study the role of topologies on these cognitive tasks, a final contribution shows that DNF retain their properties in irregular learned topological maps. In this experience, topologies are learnt via growing neural gas in order to extract intrinsic dimensions of the sensory space, but new perspectives, with deeper models, are suggested for application in active perception and embodied cognition.
Jury:
Mme Estela Bicho | Professeur(e) | Centre Algoritmi, University of Minho, Portugal | Rapporteur(e) |
Mme Salima Hassas | Professeur(e) | LIRIS, UCBL | Directeur(trice) de thèse |
Mme Stéphanie Jean-Daubias | Professeur(e) | LIRIS, UCBL | Examinateur(trice) |
M. Mathieu Lefort | Maître de conférence | LIRIS, UCBL | Co-encadrant(e) |
M. Jean-Charles Quinton | Maître de conférence | Laboratoire Jean Kuntzmann, Univ. Grenoble Alpes | Co-encadrant(e) |
Mme Marieke Rohde | Docteur | Institute for Innovation and Technology, Germany | Examinateur(trice) |
M. Denis Sheynikhovich | Maître de conférence | Institut de la Vision, Sorbonne Université | Rapporteur(e) |
M. Jochen Triesch | Professeur(e) | Frankfurt Institute for Advanced Studies, Goethe University, Germany | Examinateur(trice) |
M. Alban Laflaquière | Docteur | Sony AI | Invité(e) |