Every day, countless volumes of documents are received and processed in businesses worldwide.

This thesis focuses on automating the extraction of information from these corporate documents using machine learning models.

Transformers, with their self-supervised pre-training, demonstrate remarkable accuracy in document comprehension. Moreover, they outperform recurrent networks in information extraction through word classification, requiring less training data. Specific pre-training tasks tailored to corporate documents further enhance model performance, even with smaller models. Finally, efficient transformer-derived architectures reduce the evaluation cost for long sequences, enabling the processing of sequences composed of different modalities.