Thesis of Mehdi Ayadi


Subject:
A skyline based approach for mobile augmented reality

Defense date: 02/07/2019

Advisor: Serge Miguet
Coadvisor: Mihaela Scuturici

Summary:

The main objective of this thesis is to propose an approach that allows a user, moving in an urban environment, to visualize in augmented reality, using a smartphone, the impact of a construction project on the urban landscape, once the building is completed. In particular, with the current attractivity for tower construction projects in major cities (London, Paris and Lyon in particular), it is becoming a very important issue for neighborhood users to be able to visualize the impact of a construction project on the skyline and to give an objec- tive opinion on it. For geographers and neighborhood users, this innovative solution makes it possible to take the opposite view from architectural firms, which only offer very qualitative renderings of their construction projects, with a limited number of views, to the detriment of more objective visualizations. We do not necessarily propose a photo-realistic rendering, but rather a more exact geometric consistency of the scene, throughout the user’s movements. Vi- sualizations at different positions and orientations correspond better to common use cases in neighborhoods. We propose an approach in which data from smartphone’s embedded instru- ments are merged to estimate a first user’s pose : the magnetic compass estimates the viewing direction ; the GPS, gyroscope and accelerometer roughly evaluate the parameters of motion (three degrees of freedom in translation, and three degrees of freedom in rotation). This first estimated pose, associated with the 3D model of the city, allows, using the rendering supports (2D and 3D) of the devices, to generate a synthetic image of what the user theoretically vi- sualizes at this position. Nevertheless, the unique use of these instruments to insert a synthetic object into the video stream gives the user a very unrealistic impression of the viewed scene : due to instrument’s data inaccuracies, synthetic objects appear to “hover” and "float" with the user’s movements. For this, the skyline is automatically extracted from real (acquired by the camera) and virtual (generated) images. A matching step between the two skylines allows to realign the virtual skyline onto the real one, allowing thus a real time insertion of the 3D ob- ject in the real-time video stream, and an improvement according to two criteria : precision and stability, giving with a much better immersion impression. In this matching step, several similarity metrics are proposed that are used in an optimization approach based on a gradient descent.


Jury:
ADEL ALIMIProfesseur(e)Université de Sfax (ENIS)Examinateur​(trice)
ANNE RUASDirecteur(trice) de rechercheIFSTTARRapporteur(e)
LUCE MORINProfesseur(e)INSA de RennesRapporteur(e)
PATRICK SAYDIngénieur(e) de rechercheComissariat à l'Energie AtomiqueExaminateur​(trice)
MIHAELA SCUTURICIMaître de conférenceUniversité Lyon 2Co-encadrant(e)
SERGE MIGUETDirecteur(trice) de rechercheUniversité Lyon 2Directeur(trice) de thèse
CHOKRI BEN AMARProfesseur(e)Université de Sfax (ENIS)Co-directeur (trice)