Thesis of Julia Cohen

Industrial objects detection using 3D models in egocentric images

Defense date: 13/07/2022

Advisor: Laure Tougne Rodet
Coadvisor: Carlos Crispim-Junior


Industrial manufacturing can be facilitated using innovative digital solutions such as Augmented Reality (AR). The development of new devices such as AR headsets and head-mounted devices enable operators to visualize assembly instructions while having their hands free to manipulate the physical pieces. The detection of these industrial objects through a head-mounted camera enables the virtual elements to automatically adapt to the real scene.  However, images captured with an AR headset present visual artefacts inherent to the egocentric point of view. Although object detection in images is a popular application of deep learning for its effectiveness, artificial neural networks are rarely applied to egocentric images. The task is even more complex when no real image of the objects of interest is available, and the algorithm will be embedded in a mobile computer with a real-time inference requirement.

In this thesis, we address this topic by leveraging the available 3D models of the objects of interest, in order to create a synthetic and egocentric dataset for the training of mobile and real-time neural networks. We analyze the elements of this synthetic dataset playing a key-role in removing the need for real images during training. Then, we propose to use the depth information contained in RGB-D images to improve the performance of the object detector. Hence, we tackle the issue of domain generalization from synthetic to real RGB-D images, et we propose different approaches to reduce the reality gap, that are compatible with a mobile and real-time inference.

M. Desbarats PascalProfesseur(e)Université de BordeauxRapporteur(e)
M. Mille JulienMaître de conférenceINSA Centre - Val de LoireRapporteur(e)
Mme Tougne Rodet LaureProfesseur(e)Université Lyon 2Directeur(trice) de thèse
M. Aubry MathieuMaître de conférenceEcole des Ponts ParisTechExaminateur​(trice)
Mme Fromont ElisaProfesseur(e)Université de Rennes 1Examinateur​(trice)
M. Crispim-Junior CarlosMaître de conférenceUniversité Lyon 2Co-encadrant(e)
M. Chiappa Jean-MarcGroupe DEMSInvité(e)