Glass Segmentation using Intensity and Spectral Polarization Cues
- Title
- Glass Segmentation using Intensity and Spectral Polarization Cues
- Authors
- Mei, Haiyang; Dong, Bo; Dong, Wen; Yang, Jiaxi; BAEK, SEUNG HWAN; Heide, Felix; Peers, Pieter; Wei, Xiaopeng; Yang, Xin
- Date Issued
- 2022-06-24
- Publisher
- IEEE Computer Society
- Abstract
- Transparent and semi-transparent materials pose significant challenges for existing scene understanding and segmentation algorithms due to their lack of RGB texture which impedes the extraction of meaningful features. In this work, we exploit that the light-matter interactions on glass materials provide unique intensity-polarization cues for each observed wavelength of light. We present a novel learning-based glass segmentation network that leverages both trichromatic (RGB) intensities as well as trichromatic linear polarization cues from a single photograph captured without making any assumption on the polarization state of the illumination. Our novel network architecture dynamically fuses and weights both the trichromatic color and polarization cues using a novel global-guidance and multi-scale self-attention module, and leverages global cross-domain contextual information to achieve robust segmentation. We train and extensively validate our segmentation method on a new large-scale RGB-Polarization dataset (RGBP-Glass), and demonstrate that our method outperforms state-of-the-art segmentation approaches by a significant margin.
- URI
- https://oasis.postech.ac.kr/handle/2014.oak/114976
- Article Type
- Conference
- Citation
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, page. 12612 - 12621, 2022-06-24
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.