Masters Thesis Defense: Alen Joy

In this thesis, we address the problem of estimating spatially varying BRDF (SVBRDF) of complex outdoor scenes. We present two novel techniques for extracting the scene's reflectance maps. The reflectance maps can be used to re-render the scene in arbitrary lighting and viewing conditions.

In the first approach, we present a neural network trained on synthetic images of spheres representing a per-triangle face reflectance map. The trained neural network is then used to predict reflectance properties for each of the reflectance maps in the object. In the second approach, we propose an end-to-end process using a differentiable path tracer and formulate inverse rendering as a two-step optimization introducing a new multi-view gradient consistency loss. The BRDF is parameterized with three reflectance maps for diffuse, specular and roughness, optimized using the differentiable path tracer. The estimated  SVBRDF can then be used to relight the scene from novel viewpoints and illumination conditions.

We have extensively tested and reported the performance of our proposed techniques. Using multi-view images, camera properties, and geometry, we show that we can successfully estimate the reflectance properties of the scene. We also present our experiments on large-scale outdoor scenes, which demonstrate the effectiveness of the approaches.