FSPGD: Rethinking Black-box Attacks on Semantic Segmentation

To address the issue that traditional black-box attacks only affect the output logits while ignoring the relationship between spatial structure and category features, the study proposes the Feature Similarity Projection Gradient Descent (FSPGD) method. This method uses a dual-loss design: the external loss forces cleaning of differences between clean samples and adversarial features to weaken cross-model alignment, while the internal loss reduces the feature consistency of different spatial instances within the same category. Extensive experiments on the Pascal VOC 2012 and Cityscapes datasets using CNN and Transformer architectures show that FSPGD achieves advanced transfer rates, consistently outperforming traditional and baseline methods such as SegPGD, CosPGD, and RP-PGD. Additionally, adversarial training using FSPGD samples enhances the model’s robustness to unseen attacks. The related code has been open-sourced.