Thesis of Sophie Villenave


Subject:
Toward seamless thermal displays for highly immersive virtual reality: design, integration, and evaluation

Start date: 01/09/2022
End date (estimated): 01/09/2025

Advisor: Guillaume Lavoué
Coadvisor: Pierre Raimbaud

Summary:

Virtual Reality (VR) is increasingly recognized as a powerful medium, which has broad applicability across various fields. Yet its full potential hinges on providing users with truly believable and engaging multisensory experiences. While visual and auditory feedback are extensively researched, thermal simulation remains a comparatively underdeveloped modality within the field. This thesis aims to bridge this gap, and focuses on the design, integration and evaluation of novel thermal displays for use in highly immersive VR applications.

This research is structured around three key axes: the development and psychophysical characterization of novel thermal feedback displays adapted to highly immersive 6DoF VR, the exploration of parameters of thermal rendering fidelity and how they influence the quality of experience in VR and the development of toolbox to ease multimodal data collection and analysis tool for VR user studies.

Our first thermal feedback display was designed to simulate the thermal ambiance of virtual environments and uses non-contact heat transfer via infrared lamps and fans. The second display provides a local thermal sensation via contact heat transfer powered by a thermo-electric device. Our novel compact controller for thermo-electric devices allows for precise control of thermal sensations when touching and grasping virtual objects. For both displays we developed software to ease integration within 6DoF VR applications built with the Unity engine.

Both systems underwent a psychophysical characterization process, to bridge the gap between the parameters that can be acted upon by the developer and the thermal sensations to be experienced by the user. Using our ambient display we investigated the impact of dynamism, – the responsiveness of the thermal feedback to user actions – on agency, presence and haptic experience. By leveraging our localized thermal display, we compared temperature setpoint and heat flux setpoint controls, in order to understand how heat transfer control impacts thermal sensation, latency and haptic experience.

Moreover, to facilitate the rigorous analysis of the complex interplay between user actions, physiological responses, and the VR environment, we developed the PLUME toolbox. PLUME records the VR sessions and harmonizes physiological signals (e.g., heart rate, skin conductance), and behavioral cues (e.g., gaze, hand movements). This tool enables in-situ and ex-situ analysis of user actions and physiological signals, providing additional clues for a deeper understanding of the lived experience when immersed in a virtual environment.

Finally, we present two projects that integrate our thermal displays into the implementation of their multisensory VR experience. This underscores our commitment to making this research accessible and replicable, using affordable hardware, offering open source software, and providing user documentation.

This work represents a significant contribution to the ongoing exploration of multisensory VR, laying the groundwork for future research into combining thermal feedback with other modalities – touch, taste, olfactory, and beyond – within highly immersive and compelling VR scenarios.

Keywords: Virtual Reality, Thermal Display, Multisensory, User Experience, Quality of Experience.


Jury:
Mme Maud MarchalProfesseur(e)Université de RennesRapporteur(e)
Mr Cesar PabloProfesseur(e)Centrum Wiskunde InformaticaRapporteur(e)
Mr Le Callet PatrickProfesseur(e)Polytech NantesExaminateur​(trice)
Mr Melo MiguelProfesseur(e) associé(e)Universidade de Trás-os-Montes e Alto DouroExaminateur​(trice)
Mme Wu Hui-YinChargé(e) de RechercheINRIA Université Côte d'Azur
Mr Lavoué GuillaumeProfesseur(e)Ecole Centrale de Lyon ENISEDirecteur(trice) de thèse
Mr Raimbaud PierreMaître de conférenceEcole Centrale de Lyon ENISEEncadrant(e)