AI Briefing#transformers
The latest AI and developer news about #transformers, with the original source and a short summary.
The latest AI and developer news about #transformers, with the original source and a short summary.
A new recipe distills Pythia-1B into a Mamba model, achieving a perplexity of 14.01 compared to the teacher's 13.86.
The new interface supports choice, score, and noul primitives with a unified API across transformers and llama.cpp.
Hugging Face proposed a draft standard for the Transformers Model Integration Contract (MIC) to strengthen CI regression detection.
AI2 introduces StateBench, a four-level benchmark revealing that while recurrent architectures generalize better than transformers, top models like Gemini 3.8 Flash and Claude Opus 5.5 still fail on hard state-tracking tasks.
The new standard uses small CPU fixtures to verify model behavior on PRs, catching a UMT5 attention bug that persisted for two months.
Presented at EMNLP 2026, the method reuses key-value buffers from earlier layers to mitigate representation collapse in ~1B parameter models.
A drop-in component for transformers that reduces peak memory by 1.5-3.4x while maintaining accuracy.
Transformers, Diffusers, and Kernels now run natively on TPUs using the TorchTPU backend, eliminating the need for torch_xla or JAX.
The open-source release includes 5M to 209M parameter checkpoints supporting zero-shot forecasting with up to 32,768 historical time steps.
The method reduces pretraining costs by over 99% and enables controlled comparisons of hybrid architectures like Gated KalmaNet, Gated DeltaNet, and Mamba-2.
Experiments with models from 135M to 1B parameters show LIFT outperforms standard Transformers on language modeling and reasoning tasks under token-matched budgets.
Hugging Face Transformers has released a feature that allows direct inference of GGUF models in a compressed state without dequantization on Apple Silicon Macs, utilizing llama.cpp's ggml Metal kernels.
Hugging Face has added native support for the GGUF format to the Transformers library, enabling debugging and flexible use of locally quantized models.
Hugging Face Transformers now directly supports the GGUF format from llama.cpp, simplifying local inference on Apple Silicon.
Parcae uses a looped recurrent structure to make 770M aim for 1.3B Transformer-level performance.
Qwen3.6 is the latest Qwen model family with strengthened agentic coding and thinking-context preservation.
Released a Skill and verification harness that helps port transformers to mlx-lm.
SSMs equipped with tool-use generalize beyond their length limits to problems of arbitrary length.