Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval

To address the problems in Partial Relevant Video Retrieval (PRVR), the research team proposed a new framework called Intrinsic Temporal Adaptation (ITA). This framework uses the Backbone-Internal Temporal Adaptation mechanism, enabling the last layer of the visual Transformer to focus on adjacent frames while keeping the main components of the CLIP model frozen, with only the adaptation parameters trained. Additionally, the framework introduces the Affinity-Weighted Gradient Propagation method, which aggregates the first k frames based on the affinity between text and frames to propagate learning signals. Experiments show that this method achieves the highest performance in PRVR benchmarks, demonstrating robustness across different datasets and enabling more accurate frame-level evidence retrieval at relevant times. The related code is now open-source.