AI Briefing#tgi
The latest AI and developer news about #tgi, with the original source and a short summary.
The latest AI and developer news about #tgi, with the original source and a short summary.
HuggingFace's LLM serving solution TGI now natively supports Intel Gaudi hardware.
Hugging Face's TGI is introducing a multi-backend architecture that supports various inference engines such as vLLM and TensorRT-LLM.
Hugging Face's TGI has launched a feature that dynamically serves multiple LoRA adapters from a single base model.
It introduces a benchmarking tool that analyzes TGI's performance metrics and how to use it to optimize LLM deployment.
Hugging Face has launched a Messages API compatible with the OpenAI API to support transitions to open LLMs.
Hugging Face's LLM serving solution TGI is now officially supported on AWS Inferentia2 and Amazon SageMaker.
Hugging Face has released benchmark results for over 60 configurations to optimally deploy Llama 2 on Amazon SageMaker.
Hugging Face has launched a TGI-based LLM inference container dedicated to Amazon SageMaker.