Thesis of Julia Cohen

Object and action recognition to interact with an augmented reality system


The objective of the thesis is to analyze videos containing actions and activities made by a person in an assembly line of a factory to accomplish a task. We would like to propose a set of models capable of detecting visual (objects), spatio-temporal (movements) and semantic (relations between actions and objects) cues that would make it possible to recognize the tasks performed by a human operator and to measure their quality. The recognition of these elements is intended to allow an intelligent information system (IIS) to come to the aid of an operator as soon as a doubt arises, or to intervene when an action taken is not in accordance with the target task. In other words, we would like to develop modules for the detection of actions and key objects that will allow interactions between the operator and the S2I developed by the DEMS group. 

Advisor: Laure Tougne
Coadvisor: Carlos Crispim-Junior