CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution

CoSkill proposes a unified multi-agent reinforcement learning framework, aiming to achieve collaborative adaptation on a hierarchical skill library by jointly training reasoning agents and learnable meta-skills agents. This framework reconsiders static meta-skills workflows into dynamic agents, enabling the reasoning agents to take actions based on the retrieved task skills and sub-step skills, thereby guiding the meta-skills agents to optimize these skills. In the ALFWorld and WebShop benchmarks, CoSkill performed significantly better than existing baselines, achieving success rates of 98.4% and 90.6%, respectively, which is an improvement of approximately 3.5 to 6.2 percentage points compared to existing methods. Additionally, the framework demonstrated better performance in terms of sample efficiency, asymptotic performance, and wall clock efficiency. The relevant code has been made open-source.