[AAAI 2026] RL-Studio: A System for Multi-Phase Reinforcement Learning Experiments

Existing reinforcement learning (RL) has remained confined to the Online RL and Offline RL paradigms, along with Offline-to-Online RL, which combines the two. However, real-world industrial settings and robot control environments demand a more complex process in which the learning method, algorithm, and environment (Sim-to-Real) must flexibly change.

To overcome these limitations, LG AI Research's PI Lab developed the Multi-Phase RL framework and the platform RL-Studio, which supports it. Rather than simply moving through a fixed stage from 'Offline to Online,' it is designed to systematically manage various learning transitions.

The research team demonstrated performance through two scenarios.

• Fine-Tuning scenario: After pre-training with TD3-BC (Offline), fine-tuning with TD3 (Online) achieved high sample efficiency.
• Adaptation scenario: When applying a policy trained in a simulation environment to a real environment (e.g., changes in gravity), switching to a different algorithm (SAC) enabled rapid convergence.

RL-Studio serves as an experimental platform that helps researchers systematically approach key questions such as 'when to switch phases' and 'which combination is optimal.'