AI Briefing#in-context-learning
The latest AI and developer news about #in-context-learning, with the original source and a short summary.
The latest AI and developer news about #in-context-learning, with the original source and a short summary.
The study demonstrates that gating mechanisms in SSMs cause models to learn in-weights memorization solutions, delaying or preventing convergence to correct in-context learning even without architectural memory limits.
The framework improves reasoning accuracy in 1.5B parameter models by integrating In-Context Learning into Test-Time Reinforcement Learning.
A 300M-parameter transformer trained on synthetic recurrent sequences achieves 0.9–2.4 bits per byte on real languages after reading 1 million bytes.
Kumo Tabular achieves state-of-the-art accuracy on TabArena with ELO 1950 while being 17x faster than LimiX-2, trained entirely on synthetic data.
Anthropic researchers released a mathematical framework proving that Induction Heads are the core mechanism for In-context Learning in 2-layer Attention-only Transformers.
LG AI Research has released EXAONE Tabular, a lightweight tabular foundation model pre-trained without real data.
LG AI Research has released 'EXAONE Tabular', a foundation model for tabular data pre-trained exclusively on synthetic data without using any real-world data.
Introducing BDH-CQ, a system that demonstrates high efficiency on the ARC-AGI-1 benchmark through recurrent latent reasoning.
They presented a methodology that leverages the commonsense knowledge embedded in LLMs to plan complex tasks step by step, improving planning accuracy through mutual information-based ranking and environment feedback.
Google has released TabFM, a model that can make predictions for classification and regression tasks on tabular data without separate training.
During pretraining, LMs repeatedly oscillated between pattern imitation and generalization.
FST improves both sample efficiency and the forgetting problem together in LLM continual learning.
A benchmark of 4 tasks and 9 datasets verified the limits of LLMs' context understanding.