Mark Zuckerberg Opens Developer Access to Muse Connectors... Voice Recognition Model Performance Released Separately

On September 18, Mark Zuckerberg announced that access would be opened for developers to build Muse Connectors. When developers provide an API, Muse provides the context of agents, browsers, and users' actual intent to automate service access and processing. New connectors are immediately available at muse.ai/platform.

Meanwhile, Muse's security architecture was introduced on September 8. Muse is a personal agent designed with privacy and security at its core, storing data and credentials in the 'Muse Secure VM', an isolated Linux computer equipped with a browser, CPU, memory, and storage. A separate 'Sentinel agent' running within the VM approves all actions and data leaving the network, enforced by the kernel, which requests user approval when necessary.

Additionally, 'Muse Voice Transcribe', the first real-time audio recognition model from MSL (Meta Superintelligence Labs), was released on September 1. This model achieved SOTA in streaming speech-to-text and supports speaker diarization and endpointing within a single model. Through 'Adaptive delay' technology, it waits longer for difficult words and quickly confirms easy ones, demonstrating performance close to the Pareto front in the speed-accuracy trade-off. It was trained on over 70 languages (with 25 verified at launch) and maintains performance in code-switching scenarios, sessions longer than one hour with more than 20 speakers, and noisy audio environments.