AI BriefingCohere Model Vault: Verifiable Confidential AI Inference
·2026.10.23 05:36
Key point
The feature uses Oblivious HTTP and remote attestation to protect data in transit and verify hardware isolation on NVIDIA H100 GPUs.
Details
- Cohere Model Vault's Confidential Computing feature is now in public beta on Microsoft Azure using NVIDIA H100 GPUs, with support for NVIDIA Blackwell infrastructure (including B200 GPUs) in progress toward general availability.
- The system protects inference by running the model serving stack in a Confidential Virtual Machine (CVM) with hardware-based isolation, extending the security boundary from the CPU to the GPU via NVIDIA Confidential Computing.
- Request payloads are encrypted using Oblivious HTTP (OHTTP), ensuring that only the attested CVM can decrypt sensitive data, while intermediate infrastructure handles routing without accessing plaintext content.
- Remote attestation acts as an active security control, allowing clients to cryptographically verify that the hardware, firmware, and software match approved policies before sending sensitive requests.
- The architecture is built on open-source technologies like Kata Containers and Confidential Containers, with Cohere planning to open-source the confidential inference stack to enable customer verification.
- The solution does not replace application security, data residency controls, or compliance requirements, and it does not protect against physical side-channel attacks, denial-of-service attacks, or application-level vulnerabilities like prompt injection.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Nvidia OpenShell: Secure Runtime for Autonomous AI Agent Fleets