Conference paper
Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation
2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.6964-6973
IEEE/CVF International Conference on Computer Vision (ICCV) 2021 (Montreal, QC, Canada, 2021)
2021
Abstract
Semantic segmentation is a challenging task in the absence of densely labelled data. Only relying on class activation maps (CAM) with image-level labels provides deficient segmentation supervision. Prior works thus consider pre-trained models to produce coarse saliency maps to guide the generation of pseudo segmentation labels. However, the commonly used off-line heuristic generation process cannot fully exploit the benefits of these coarse saliency maps. Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixellevel affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks. The mutual boost between pseudo label updating and cross-task affinity learning enables iterative improvements on segmentation performance. Extensive experiments demonstrate the effectiveness of the proposed auxiliary learning network structure and the cross-task affinity learning method. The proposed approach achieves state-of-the-art weakly supervised segmentation performance on the challenging PASCAL VOC 2012 and MS COCO benchmarks.
Details
- Title
- Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation
- Authors/Creators
- L. Xu (Author/Creator) - The University of Western AustraliaW. Ouyang (Author/Creator) - The University of Western AustraliaM. Bennamoun (Author/Creator) - The University of Western AustraliaF. Boussaid (Author/Creator) - The University of SydneyF. Sohel (Author/Creator) - Murdoch UniversityD. Xu (Author/Creator) - Hong Kong University of Science and Technology
- Publication Details
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.6964-6973
- Conference
- IEEE/CVF International Conference on Computer Vision (ICCV) 2021 (Montreal, QC, Canada, 2021)
- Identifiers
- 991005543702207891
- Murdoch Affiliation
- School of Information Technology
- Language
- English
- Resource Type
- Conference paper
Metrics
100 Record Views