Know Your Attention Maps: Class-specific Token Masking for Weakly Supervised Semantic Segmentation
Type
conference paper
Date Issued
2025-10-19
Author(s)
Abstract
Weakly Supervised Semantic Segmentation (WSSS) is a challenging problem that has been extensively studied in recent years. Traditional approaches often rely on external modules like Class Activation Maps to highlight regions of interest and generate pseudo segmentation masks. In this work, we propose an end-to-end method that directly utilizes the attention maps learned by a Vision Transformer (ViT) for WSSS. We propose training a sparse ViT with multiple [CLS] tokens (one for each class), using a random masking strategy to promote [CLS] token - class assignment. At inference time, we aggregate the different self-attention maps of each [CLS] token corresponding to the predicted labels to generate pseudo segmentation masks. Our proposed approach enhances the interpretability of self-attention maps and ensures accurate class assignments. Extensive experiments on two standard benchmarks and three specialized datasets demonstrate that our method generates accurate pseudo-masks, outperforming related works. Those pseudo-masks can be used to train a segmentation model which achieves results comparable to fully-supervised models, significantly reducing the need for fine-grained labeled data.
Language
English
Book title
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)
Start page
23763
End page
23772
Pages
9
File(s)![Thumbnail Image]()
open.access
Name
Hanna_Know_Your_Attention_Maps_Class-specific_Token_Masking_for_Weakly_Supervised_ICCV_2025_paper.pdf
Size
7.95 MB
Format
Adobe PDF
Checksum (MD5)
ccb1cbc63b11b23cc9e4581f895eab67