Interaction Model - A Scalable Approach to Human-AI Collaboration

Thinking Machines Lab has released a preview of the Interaction Model, aiming to reduce the collaboration bottleneck created by turn-based dialogue. The key point is that the model accepts and outputs audio, video, and text as real-time streams, without waiting for the user to finish speaking.

The system is divided along two axes.

• Interaction Model: handles conversation, interruptions, overlapping speech, and visual reactions in microturns of 200ms units.
• Background Model: handles long-horizon reasoning, tool use, search, and generation tasks asynchronously, streaming results back.

The architecture is close to minimal preprocessing with joint training. Audio uses dMel, images use 40x40 patches + hMLP, and audio decoding uses a flow head. On the inference side, a streaming session appends 200ms chunks to a persistent sequence in GPU memory, and latency and determinism are managed using gather+gemv, batch-invariant kernels, and NVLS.

In evaluations, TML-Interaction-Small was released as 276B MoE / 12B active, recording an FD-bench V1.5 average of 77.8 points and turn-taking latency of 0.40 seconds. On Audio+Tools it scored 82.8% / Pass@1 68.0%, Audio MultiChallenge APR was 43.4%, and Harmbench text refusal rate was 99.0%. Meanwhile, QIVD Video+Audio was relatively weak at 54.0%.

New evaluation axes were also proposed. TimeSpeak and CueSpeak measure time awareness and overlapping speech, while RepCount-A, ProactiveVideoQA, and Charades measure visual proactivity. Existing commercial real-time models have not yet meaningfully solved these tasks, and long-session context management and handling unstable networks remain open challenges.

A limited research preview is planned to open within the coming months, with a larger model to be released later this year.