Locating and Steering Refusal Beyond Attention

The study suggests that in new architectures such as State Space Models (SSM), the secure representation of rejection instructions has the ability to be migrated across different architectures. Experiments show that, whether the underlying architecture is Transformer or SSM, the secure representation is determined by a single direction in the residual flow; this direction only changes the spatial orientation without reshaping the structure, and the representation spaces of different models can be aligned through rigid rotation. Harmful detection tools trained on SSM can accurately identify attack inputs; removing the aligned direction causes the model to abandon attacks it originally rejected, while random directions have weak effects. Control experiments confirm that the key factor lies in the estimated position of the interfering direction (write point) rather than the application position. By fixing the direction strength through a gated mechanism triggered by detectors, the success rate of jailbreak attempts can be reduced in four architectures: SSM, Transformer, cycle, and hybrid, and attacks optimized for defense can be prevented. The conclusion indicates that it is not necessary to rebuild security tools when migrated to new architectures; simply re-estimate the direction at the write points of the new architecture.