How to evaluate LLMs before production
When evaluating the LLM system used to reduce false positives in GitHub secret scanning, the team found that benchmark tests alone cannot reflect the challenges of the production environment. The article points out that as the system approaches deployment, inputs often become ambiguous, labels inconsistent, and context missing, resulting in offline metrics improvements not necessarily translating into actual production performance. The author shares practical methods from prototype to production: first, make product decisions rather than adjusting model components; define core goals for secret scanning scenarios as reducing false positives and maintaining sufficient recall rate to ensure security. The team divided the evaluation criteria into three levels: main results (such as reduced false positives), security constraints (such as the lower limit of recall rate), and operational safeguards (such as latency and cost), to avoid treating all metrics as equally important and to select the optimal configuration while satisfying both security and feasibility.