Paper page - When Models Edit Too Much: On the Fidelity of Minimal Code Edits
2026-09-07 08:00Models🔥 42.2 heat score
1sources
1days unfolding
42.2heat score
4mentions
SummaryAI generated
A study on the code repair capabilities of large language models indicates that advanced models, including GPT-5.5, tend to over-edit, rewriting beyond the minimum necessary to fix bugs. Based on 400 BigCodeBench problems, researchers built an evaluation framework and tested known minimum patches by injecting controlled AST-level contamination. They found that these models often involve unnecessary editing and increased cognitive complexity. After introducing preservation instructions, the average excess Levenshtein distance was reduced from 0.195 to 0.131, a 26.6% decrease in cognitive complexity, while the Pass@1 score improved by 2.3 points. Comparing different optimization strategies, they found that supervised fine-tuning tends to lead to overfitting, while reinforcement learning performs best in balancing edit fidelity and performance retention.
“Large language models tend to over-edit when repairing code, i.e., rewriting beyond the minimum necessary for fixing bugs. The research team built an evaluation framework based on 400 BigCodeBench problems and generated known minimum patches by injecting controlled AST-level contamination. It was found that advanced models including GPT-5.5 often involve unnecessary editing and increased cognitive complexity. Introducing save instructions can reduce the average excessive Levenshtein distance from 0.195 to 0.131, reducing cognitive complexity by 26.6% and improving Pass@1 by 2.3 points; supervised fine-tuning is prone to overfitting, while reinforcement learning performs best in balancing editing fidelity and performance retention.”