Online Change-point Detection for Cooperative Multi-Agent Reinforcement Learning

To address the issue of unreliable experience in cooperative multi-agent reinforcement learning due to changes in the environment or task objectives, researchers have proposed a lightweight online change point detection algorithm called “Past Reward Pattern” (PPR). This algorithm smooths the agent rewards, highlights recent changes, and uses a statistical drift detector to identify significant deviations, with algorithm independence. The study was evaluated in a custom Speaker-Listener environment, covering two controlled non-stationary scenarios. Experiments showed that directly applying the detector to raw rewards often led to missed detections, while smoothing the rewards baseline could detect issues earlier but produced numerous false alarms. PPR effectively limited redundant detections and accurately identified controlled deviations, achieving a good balance between detection speed and alarm stability. It can serve as a reliable reward-based monitoring tool during the training of cooperative MARL systems.