DeepBasis: Hand-Held Single-Image SVBRDF Capture
via Two-Level Basis Material Model
SIGGRAPH ASIA 2023

Li Wang, Lianghao Zhang, Fangzhou Gao, Jiawan Zhang

College of Intelligence and Computing, Tianjin University

Abstract

overview

Recovering spatial-varying bi-directional reflectance distribution function (SVBRDF) from a single hand-held captured image has been a meaningful but challenging task in computer graphics. Benefiting from the learned data priors, some previous methods can utilize the potential material correlations between image pixels to serve for SVBRDF estimation. To further reduce the ambiguity from single-image estimation, it is necessary to integrate additional explicit material correlations. Given the flexible expressive ability of basis material assumption, we propose DeepBasis, a deep-learning-based method integrated with this assumption. It jointly predicts basis materials and their blending weights. Then the estimated SVBRDF is their linear combination. To facilitate the extraction of data priors, we introduce a two-level basis model to keep the sufficient representative while using a fixed number of basis materials. Moreover, considering the absence of ground-truth basis materials and weights during network training, we propose a variance-consistency loss and adopt a joint prediction strategy, thereby enabling the existing SVBRDF dataset available for training. Additionally, due to the hand-held capture setting, the exact lighting directions are unknown. We model the lighting direction estimation as a sampling problem and propose an optimization-based algorithm to find the optimal estimation. Quantitative evaluation and qualitative analysis demonstrate that DeepBasis can produce a higher quality SVBRDF estimation than previous methods. All source codes will be publicly released.

Supplementary Video

bilibili

BibTeX 

                    @inproceedings{10.1145/3610548.3618239,
                        author = {Wang, Li and Zhang, Lianghao and Gao, Fangzhou and Zhang, Jiawan},
                        title = {DeepBasis: Hand-Held Single-Image SVBRDF Capture via Two-Level Basis Material Model},
                        year = {2023},
                        isbn = {9798400703157},
                        publisher = {Association for Computing Machinery},
                        address = {New York, NY, USA},
                        url = {https://doi.org/10.1145/3610548.3618239},
                        doi = {10.1145/3610548.3618239},
                        booktitle = {SIGGRAPH Asia 2023 Conference Papers},
                        articleno = {85},
                        numpages = {11},
                        keywords = {Material Reflectance Modeling, SVBRDF, Basis Maiterials, Deep Learning, Rendering},
                        location = {, Sydney, NSW, Australia, },
                        series = {SA '23}
                        }

Acknowledgements

This work was supported in part by National Key Research and Development Program of China (2022YFF0904301).