AI that Predicts Brain Activity

TRIBE v2 is a tri-modal foundation model that jointly handles video, audio, and language, predicting human brain activity across diverse naturalistic environments and experimental conditions.

• Trained on integrated data from 1,000+ hours of fMRI and 720 subjects, it showed high accuracy even for novel stimuli, tasks, and subjects.
• It delivered performance improvements of several-fold over existing linear encoding models.
• Using the model's internal representations, the team performed in silico experimentation, reproducing decades of accumulated classical neuroscience findings within software.
• Interpretable latent features also revealed fine-grained topography of multisensory integration.
• Code and weights have been released on GitHub and Hugging Face.

The key point is that by unifying stimulus-response relationships into a single model, it becomes possible to simulate neuroscience experiments across a much broader range.