From Preferences to Principles: Rubric-Based Alignment for Grounded Knowledge Answers

To address the challenge of难以 measure high-quality answers with a single scalar in open-domain questions, Apple ML Research’s team introduced a reward framework based on a rubric. This approach aims to solve the problem of capturing multiple quality dimensions through fine-grained supervision. The new method generates a specific query rubric based on retrieved evidence and broken down into multiple quality dimensions during the fine-tuning phase, thereby providing precise supervisory signals. Experimental results show that this method achieves average improvements across three key evaluation axes: composition, grounding, and instruction adherence, effectively enhancing the quality and relevance of the models’ generated answers.