Learning Scene Illumination by Pairwise Photos from Rear and Front Mobile Cameras

Dachuan Cheng, Jian Shi, Yanyun Chen, Xiaoming Deng and Xiaopeng Zhang

PaperImportant things

     Illumination estimation is an essential problem in computer vision, graphics and augmented reality. In this paper, we propose a learning based method to recover low-frequency scene illumination represented as spherical harmonic (SH) functions by pairwise photos from rear and front cameras on mobile devices. An end-to-end deep convolutional neural network (CNN) structure is designed to process images on symmetric views and predict SH coefficients. We introduce a novel Render Loss to improve the rendering quality of the predicted illumination. A high quality high dynamic range (HDR) panoramic image dataset was developed for training and evaluation. Experiments show that our model produces visually and quantitatively superior results compared to the state-of-the-arts. Moreover, our method is practical for mobile-based applications.

Slide / Poster / Viedo


More than 1000 HDR panoramas(6912x3456). For downloading this dataset, please contact to chengdc@ios.ac.cn


We provide source code for training, testing and prediction. github