Samsung Introduces CG-TTRL for Context-Guided On-Device LLM Adaptation

Samsung R&D Institute United Kingdom researchers introduced CG-TTRL (Context-Guided Test-Time Reinforcement Learning), a framework designed to improve the reasoning capabilities of large language models directly on user devices without relying on cloud resources or labeled data. The method addresses the limitations of standard Test-Time Reinforcement Learning (TTRL), which often reinforces incorrect reasoning when majority voting fails, by integrating In-Context Learning (ICL) into the unsupervised fine-tuning pipeline.

How CG-TTRL Works

CG-TTRL augments user queries with retrieved examples consisting of queries, step-by-step solutions, and final responses. To ensure efficiency on devices, the system uses TF-IDF features and cosine similarity to select the most relevant examples from a context pool, avoiding the need for auxiliary models or heavy computation. This context guides both the exploitation phase (generating predictions for majority voting) and the exploration phase (updating model weights), ensuring that the pseudo-labels used for reinforcement are more accurate and that weight updates are anchored to correct reasoning patterns.

Performance and Efficiency

The framework was evaluated on 1.5B parameter models including Qwen2.5-Math-1.5B-Instruct, Qwen2.5-1.5B-Instruct, and DeepSeek-R1-Distill (1.5B) across benchmarks such as MATH-500, GSM-100, AMC, AIME 2024, and GPQA. CG-TTRL consistently outperformed standard TTRL, with DeepSeek-R1-Distill improving from 37.2% baseline accuracy to 59.4% with CG-TTRL, compared to 56.8% with TTRL.

Key advantages for on-device deployment include:

• Robustness under domain shift: Significant gains were observed on out-of-domain benchmarks like GPQA and AIME, where CG-TTRL improved Qwen2.5-Math-1.5B-Instruct accuracy by 40.7% and 133.3% relative to the base model, respectively.
• Fast adaptation: The method achieves strong performance after only a few update steps. After a single epoch (three update steps), CG-TTRL raised average accuracy to 53.0%, whereas TTRL reached only 49.5%.