Instantly Spin Up a vLLM Server with Hugging Face Jobs

Using Hugging Face's Jobs infrastructure, you can instantly run a private LLM endpoint based on vLLM without server provisioning or Kubernetes setup.

Key Features and Usage:

• Easy Execution: You can spin up a model directly on a GPU instance by specifying a Docker image (vllm/vllm-openai) via the hf jobs run command.
• OpenAI-Compatible API: The launched server follows the OpenAI API specification, so it can be immediately queried using an existing Python openai client or curl.
• Security and Access Control: The created endpoint is not public, and it operates as a gateway that requires a Hugging Face token (Read permission) to be included as a Bearer token for access.
• Cost Efficiency: You are only billed for the time used (in seconds), making it useful for testing or batch generation tasks. After the job ends, you can explicitly stop it with hf jobs cancel to prevent wasted costs.