Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors

The researchers proposed a new technique called “Semantic Overlays” aimed at mitigating prompt injection attacks in large language models. This technique uses pre-trained adapters to apply non-textual channels in small model residual flows, adding identity labels to input segments so that the model cannot determine the origin of the segments based solely on text. Unlike traditional guided vectors, this method is trainable, adaptive, and selectively applicable, enabling the model to encode complex semantic reshaping perceptions of specific segments. Experimental results show that in five prompt injection benchmarks, the SEP separation rate increased from 24.3% to 99.0%, the success rate of TensorTrust attacks decreased to 6.2%, the success rate of AlpacaFarm attacks became zero, and labeled segments remained readable (character similarity exceeded 95%).