Thesis of Thibault Douzon

Language Models for Document Understanding

Defense date: 24/10/2023

Advisor: Christophe Garcia
Coadvisor: Stefan Duffner


Every day, countless volumes of documents are received and processed in businesses worldwide.

This thesis focuses on automating the extraction of information from these corporate documents using machine learning models.

Transformers, with their self-supervised pre-training, demonstrate remarkable accuracy in document comprehension. Moreover, they outperform recurrent networks in information extraction through word classification, requiring less training data. Specific pre-training tasks tailored to corporate documents further enhance model performance, even with smaller models. Finally, efficient transformer-derived architectures reduce the evaluation cost for long sequences, enabling the processing of sequences composed of different modalities.

Mme Lemaitre Aurélie Maître de conférenceUniversité Rennes 2Rapporteur(e)
M. Paquet Thierry Professeur(e)Université de Rouen NormandieRapporteur(e)
M. Tabbone Salvatore-Antoine Professeur(e)Université de LorraineExaminateur​(trice)
M. Ogier Jean-MarcProfesseur(e)La Rochelle UniversitéExaminateur​(trice)
M. Garcia Christophe Directeur(trice) de rechercheLIRIS INSA LyonDirecteur(trice) de thèse
M. Duffner StefanMaître de conférenceLIRIS INSA LyonCo-directeur (trice)
M. Espinas Jérémy DocteurEskerCo-encadrant(e)
M. Bérard Jean-Jacques Directeur(trice) de rechercheEskerInvité(e)