Apple Announces 'LoopCD' Research Reducing Loop Transformer Compute by Up to 48%

Apple's research team proposed LoopCD, a training-free contrastive decoding method to enhance the inference performance of loop transformers. This technique is applied at inference time without additional training or auxiliary models, performing contrastive decoding by using the first iteration's prediction (h1) as a weak reference and the last iteration's prediction (hR) as a strong prediction.

Core Principles and Variants

LoopCD is provided in two variants.

• LoopCD-Logits: Combines in the logits space, requiring the output layer to be executed twice.
• LoopCD-Hidden: Combines in the hidden state space, executing the output layer only once, resulting in negligible increase in compute.

The guidance strength (ω) can be set to a fixed value or adjusted adaptively based on token-wise probability differences.

Key Performance Improvements

Experiments targeting small models (0.37B–4B scale) such as Ouro, Huginn, Parcae, and Looped-Qwen3 yielded the following results.

• Mathematical Reasoning: The AIME 2024 pass@1 score for Ouro-2.6B-Thinking increased from 61.88% to 73.33%.
• Code Generation: The HumanEval pass@1 score for the Huginn-0125 model improved from 22.56% to 31.71% when applying LoopCD-Hidden.
• Compute Efficiency: In settings where the number of iterations was halved, accuracy remained equal to or better than full-iteration standard decoding, reducing theoretical FLOPs by 22.5%–48.2%.

Limitations and Caveats

• Validation Scope: Validated only on small models under 4B; effectiveness on large models remains unconfirmed.
• Inconsistency: Performance drops were reported on some benchmarks, such as mathematical reasoning pass@1 and GSM8K for Looped-Qwen3.
• Actual Latency: The compute reduction is based on theoretical FLOPs; the reduction in wall-clock latency in actual serving environments has not been measured.
• Code Availability: Apple has not released the code, but a repository from a POSTECH study presenting similar ideas is currently available (though it does not include Apple's hidden state variant, etc.).