Vercel AI Gateway adds Private Inference support for self-hosted and fine-tuned models

·2026.11.16 09:00

Key point

The feature allows teams to connect custom endpoints to AI Gateway, enabling unified authentication, observability, and fallback configurations for private models.

1 / 4

Details

  • Vercel AI Gateway now supports Private Inference, allowing users to connect custom model endpoints for fine-tuned, self-hosted, or prerelease models.
  • Private models use the same authentication, SDKs, APIs, and observability tools as catalog models, consolidating management in a single platform.
  • Users can switch between private and catalog models by changing only the model ID, as AI Gateway translates requests to the specific endpoint format.
  • Private models are scoped to the specific Vercel team, hidden from the public catalog, and support Zero Data Retention per provider.
  • Integration is available via the AI Gateway dashboard or Vercel CLI version 65.16.0 or later, using the private/ ID format.
  • Observability features include viewing private model requests alongside catalog traffic in logs and trace drains, with metrics for requests, tokens, and time to first token.
  • Optional fallback configurations allow requests to route to a catalog model if the private endpoint fails.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.