AI BriefingLLM Trained Only on Pre-1930 Text Released
Key point
Talkie-1930 has been released as a 13B open-weight LLM trained only on text from before 1931.
Details
Talkie-1930 is a 13B open-weight LLM trained only on text published before January 1, 1931. The training corpus is 260 billion tokens, and the fixed time cutoff eliminates contamination from modern benchmarks at the source.
The nonprofit team released both base and instruction-tuned checkpoints under Apache 2.0. A demo runs at talkie-lm.com/chat, where Claude Sonnet 4.6 prompts the model around the clock for a public demonstration.
- The research team measured the model's 'surprise' at historical events after the cutoff, and reported that reactions were strongest for the 1950s-60s.
- Questions about Hitler produced 1930s-style political analysis, and questions about 'thinking machines' produced predictions about remote work and universal language, but the model had no knowledge of modern concepts like computers, cryptocurrency, AI, and internet culture.
- The team is aiming for a GPT-3-class model by summer 2026, and believes the corpus could be grown to over 1 trillion tokens.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Talkie, a 13B Vintage Language Model from 1930
Talkie: 1930's 13B vintage language model
LLM Mathematical Excellence Driven by Pre-training Data Quality, Not VerifiabilityPre-training Progress Analysis 2019-2025: Data Improvements Contribute 3x More Than Model ImprovementsMagic Announces Pretraining Research Achieving Performance Similar to DeepSeek V4 Pro Base with Approximately 50x Fewer FLOPs