RegressiOn with Optimal Transport, for computer graphics and vision (ROOT)

Type de projet : ANR
Dates du contrat : 2016 - 2021
Équipe(s) : GeoMod, M2DisCo, Origami
Responsable scientifique LIRIS : Nicolas Bonneel
URL du projet :

Description :
ROOT will develop numerical methods for solving important problems in computer graphics and vision via the optimal mass transport theory. These problems relate to histograms, and can be advantageously written and solved via regressions with loss functions involving optimal transport. These problems include data fitting, supervised learning or statistical inference. Histograms are frequently encountered in computer graphics (e.g., reflectance functions, color palettes, distance histograms etc.) and vision (e.g., SIFT or HoG descriptors). But in many cases, they are treated simply as Euclidean vectors to fit the existing learning machinery, or other ill-suited metrics are used to compare and manipulate them. Meanwhile, the increasingly popular optimal transport theory considers histograms as piles of sand which physical motion requires effort. Based on this principle, optimal transport offers a framework with a meaningful way of comparing histograms as the amount of work required to reshape a pile of sand into another. It also proposes a way of interpolating histograms as the intermediate histograms produced during this motion. The ROOT project explores the use of optimal transport, but, as a metric in the context of inverse problems for graphics and vision, and machine learning. However, this theory remains costly in practice, and practical optimal transport solvers will also be explored. Challenges addressed by ROOT include: - Computationally efficient optimal transport solvers between histograms. Since our regression tools will make repeated calls to optimal transport solvers during the optimization iterations, this step should be made computationally efficient. We will investigate methods based on Voronoi / Power diagrams, linear programming and entropy regularizations via Bregman projections. We will strive to offer tools for high-dimensional scattered data, as well as simpler histograms living on low-dimensional grids, considering both Eulerian as well as Lagrangian approaches. - Solving important inverse problems in graphics and vision with optimal transport. These problems will be cast as regressions within the optimal transport framework. They will be used for inference, supervised learning and fitting. Typical computer graphics applications include fitting reflectance data to analytical models, inferring missing values for captured or measured data (reflectance, 3d geometries), and learning color histogram manifolds for image or video color manipulation. Typical computer vision applications include the inference of parameters in reflectance models based on images, and object recognition given sparse databases ROOT will offer a public open-source library for efficient histogram regression.