How we make AI coding more cost efficient without sacrificing task quality
GitHub Copilot has optimized AI programming efficiency through four improvements: retaining useful context, removing useless formatting, simplifying instructions, and eliminating the need for additional retrieval of background information. These changes were evaluated based on offline benchmark tests and verified through controlled online experiments before being implemented. They apply to multiple products, including Copilot CLI, apps, and code review. Research indicates that simply shortening the output of a single tool call (e.g., using RTK) can reduce local token consumption, but it often leads to model rereading or retrying due to missing key information, thereby increasing overall task costs and time consumption. Therefore, efficiency evaluations should cover the entire process from user requests to final results. Based on this, the team developed a selective output compressor. Analysis shows that installation, build, testing, and lint outputs often contain redundant noise, while source code and arbitrary command results should be retained. In earlier versions, excessive compression caused the model to perform redundant tasks. After feedback from benchmark tests, the strategy was adjusted: commands like `cat` and `git diff` were left as they were, and search results were reorganized without discarding content.