AI Briefing
KO

Workspace Experiment

·2026.04.19 03:32

Key point

CWT reduced compute by **about 45%** while maintaining a **PPL difference of 1.7%** versus the baseline.

Details

CWT is an architecture that completely removes the transformer's residual stream and replaces it with a structured workspace instead.

In the core comparison, CWT used 22.9M core compute (attn+FFN), while the compute-matched baseline used 41.7M. As a result, despite about a 45% reduction in core compute, quality remained close, with only a PPL difference of 1.7%.

It also explains that using a structured workspace allows more direct tracking of the model's internal behavior on a per-token basis, making 3D visualization and logging easier than with a regular transformer.

The paper, model weights, and model code have all been released as open source. The author added that this is their first formal research paper, and that there were budget and compute constraints.

open-sourcebenchmarktransformerarchitectureresearch-papervisualizationlarge-language-models
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditOpen-Weight CLM Released as Jev Alternative, Agent Decision-Making 4x FasterRedditMica v0.1 4B Released: Open-Source Decision Model for Agent Loops Trained for Under $30Hacker NewsQwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance MetricsPyTorchKRTrillion Labs Releases GravityOCR, an OCR Model Applying Self-Speculative DecodingHacker NewsJevper Released: A Jev Classification Wrapper for OpenAI-Compatible Models with Automatic Logprobs DetectionGeekNewsRootless Linux Sandbox 'Drop' Released for Isolating AI Agents and Third-Party Programs