Thesis of Lucas Foulon

Anomaly detection in data streams using indexing and approximation. Application to continuous analysis of message streams within the SNCF information system


In this thesis, we propose methods to approximate an anomaly score in order to detect abnormal parts in data streams. Two main problems are considered in this context. Firstly, the handling of the high dimensionality of the objects describing the time series extracted from the raw streams, and secondly, the low computation cost required to perform the analysis on-the-fly. To tackle the curse of dimensionality, we have selected the CFOF anomaly score, that has been proposed recently and proven to be robust to the increase of the dimensionality. Our main contribution is then the proposition of two methods to quickly approximate the CFOF score of new objects in a stream. The first one is based on safe pruning and approximation during the exploration of object neighbourhood. The second one is an approximation obtained by the aggregation of scores computed in several subspaces. Both contributions complete each other and can be combined. We show on a reference benchmark that our proposals result in important reduction of the execution times, while providing approximations that preserve the quality of anomaly detection. Then, we present our application of these approaches within the SNCF information system. In this context, we have extended the existing monitoring modules by a new tool to help to detect abnormal behaviours in the real stream of messages within the SNCF communication system.

Advisor: Serge Fenet
Codirection: Christophe Rigotti

Defense date: friday, october 16, 2020

Mr Cournier AlainProfesseur(e)Université de Picardie Jules Verne, AmiensRapporteur(e)
Mr Charnoy ThierryProfesseur(e)Université Paris 13Rapporteur(e)
Mme Calabretto SylvieProfesseur(e)INSA LyonExaminateur​(trice)
Mr Sturm PeterDirecteur(trice) de rechercheINRIA Grenoble Rhône-AlpesExaminateur​(trice)
Mr Rigotti ChristopheMaître de conférenceINSA LyonDirecteur(trice) de thèse
Mr Fenet SergeMaître de conférenceUniversité Claude Bernard Lyon 1Examinateur​(trice)
Mme Fromont ElisaProfesseur(e)Université de Rennes 1Examinateur​(trice)