ARIA - An Agentic Framework for Autonomous Testing of Infotainment Systems

On September 7, 2026, arXiv released the ARIA framework, which conducted end-to-end testing of an Android in-vehicle infotainment system using a multi-agent large language model. In 30 test scenarios, ARIA successfully completed 28 (with a coverage rate of 93.3%) and identified all known defects without any false positives. The framework employed a closed-loop pipeline consisting of four dedicated agents and a reporting phase, capable of automatically generating interactive scripts, reports, reproducable scripts, and visual evidence based on single-sentence scenario descriptions. Evaluation showed that the multi-agent design was significantly superior to the single-agent baseline; the first-round test false positive rate decreased from 72.0% to 52.6%, with 8 false positives due to navigation limitations, insufficient image resolution, or lack of gesture support. The study also analyzed token consumption, call counts, and cost data, indicating that system stability varied with complexity but fault detection remained consistent, demonstrating the potential of integrating visual testing into the continuous integration (CI) process.