AI BriefingLambda Calculus Benchmark Released
Key point
A benchmark evaluating models' coding ability using 120 pure lambda calculus problems has been released.
Details
λ-bench is a benchmark that evaluates AI models using 120 pure lambda calculus programming problems.
Models must write a single program in Lamb, a minimal lambda calculus language, implementing each problem's algorithm using λ-encoded data structures. A submitted .lam program is only counted correct if it passes all input/output tests.
- The v1 score is a simple pass rate of
solved problems / 120. - Problems are divided into 12 categories, with 10 questions per category.
- The scope is broad, covering algorithms, Church/Scott encoding, N-tuples, and more.
- Difficulty ranges from basic problems like Church nat addition to highly challenging ones such as a BF interpreter, FFT, and a Sudoku solver.
The currently published rankings are GPT-5.4 101/120 (84.2%), GPT-5.5 93/120 (77.5%), Opus 4.7 84/120 (70.0%), and Gemini 3.1 Pro 81/120 (67.5%), in that order.
Evaluation is run with bun eval <provider/model>, and results are saved to res/. The leaderboard is built by collecting the repository's result files and regenerating the GitHub Pages landing page.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Step 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis PerformancePrivatemode Benchmarks GLM-5.3-Flash as Vision-Capable Decision Model with Parity to Jev but Higher Cost