Hugging Face Launches Inference Optimization Service 'HUGS'

HUGS (Hugging Face Generative AI Services) is an inference microservice designed to deploy open models on your own infrastructure in an optimized state, instantly and with zero configuration.

Key Features and Benefits:

• Zero-Configuration Deployment: Run models with optimal performance on various accelerators such as NVIDIA and AMD GPUs instantly, without complex engineering processes.
• OpenAI-Compatible API: Supports a drop-in replacement that allows existing Generative AI applications to switch to a HUGS deployment environment with almost no code changes.
• Security and Infrastructure Control: Models and data can be operated safely within a company's in-house infrastructure without being exposed to the external internet.
• Enterprise-Grade Reliability: Built on Hugging Face's TGI (Text Generation Inference), providing SOC2 compliance and long-term support.

It is currently available through the AWS and GCP marketplaces, with Azure support to be added in the future.