Abstract
Scene reconstruction enables applications across visual computing, including media creation and editing, and capturing the real-world for virtual tourism and telecommunication. Advances in differentiable rendering for optimization- and learning-based reconstruction have increased quality but, as in 'forward' rendering, different methods have varying capabilities and computational costs that must be traded off against application needs. I will discuss our recent view reconstruction projects across the differentiable rendering spectrum, covering work on 6DoF video via image-based rendering for VR (visual.cs.brown.edu/matryodshka), depth reconstruction from sparse 3D points using differentiable splatting and diffusion (visual.cs.brown.edu/diffdiffdepth), and integrating time-of-flight imaging for monocular dynamic scene reconstruction (imaging.cs.cmu.edu/torf/). Finally, I will discuss how these trade-offs might inform how we can make differentiable rendering practical.

Short bio:
James Tompkin (www.jamestompkin.com) is an assistant professor of Computer Science at Brown University. His research at the intersection of computer vision, computer graphics, and human-computer interaction helps develop new visual computing tools and experiences. His doctoral work at University College London on large-scale video processing and exploration techniques led to creative exhibition work in the Museum of the Moving Image in New York City. Postdoctoral work at Max-Planck-Institute for Informatics and Harvard University helped create new methods to edit content within images and videos. Recent research has developed new multi-camera reconstruction techniques for light field, 360, and time-of-flight imagery, and has developed new image editing and generation methods through learning explicit structured representations.