AI BriefingHuggingFace Blog news
The latest AI and developer news from HuggingFace Blog, with short summaries.
Visit website →Key Points of MoE (Mixture of Experts)
This explains in detail how the MoE architecture works, how it is trained, and the trade-offs during inference.
Mistral releases Mixtral 8x7b
Mistral AI's new MoE model Mixtral 8x7b has been released, and it outperforms GPT-3.5.
SetFitABSA: High-Performance Sentiment Analysis with Small Amounts of Data
Using SetFit, a new aspect-based sentiment analysis (ABSA) framework was released that outperforms LLMs even with small amounts of data.
HuggingFace Improves LoRA Inference Speed by 300%
HuggingFace kept the base model in memory and dynamically swapped only LoRA adapters, improving inference speed by 300%.
Hugging Face Releases Optimum-NVIDIA Inference Library
Hugging Face has launched the Optimum-NVIDIA library, which speeds up LLM inference by up to 28x on NVIDIA platforms.
Hugging Face expands AMD GPU acceleration support
Hugging Face announced optimized support for running LLMs on AMD Instinct GPUs without any separate code modifications.
Root Cause of DROP Benchmark Scoring Error Identified
The abnormally low model scores observed on the DROP benchmark in the Open LLM Leaderboard were found to be caused by a normalization error in the evaluation metric.
LCM LoRA Dramatically Speeds Up SDXL Generation
With LCM LoRA, SDXL models can generate images quickly using just 4-8 steps without any separate retraining.
Prodigy-HF Released: Integrated Support for HF Models
Prodigy-HF, a plugin that directly connects the data annotation tool Prodigy with Hugging Face, has been released.
AWS Inferentia2 Accelerates Llama 2 Inference
It provides a method for optimizing and deploying Llama 2 models on AWS Inferentia2 using Hugging Face's optimum-neuron.
Hugging Face Launches Storage Regions Feature
Hugging Face has introduced a Storage Regions feature in the Enterprise Hub for regulatory compliance and improved performance.
A Guide to Training Your Own Coding Assistant
Introduces how to build a personal coding assistant optimized for a specific codebase using Hugging Face's public repositories, along with HugCoder.
Renumics Spotlight, Hugging Face Dataset Visualization Tool Released
With Renumics Spotlight, you can interactively visualize and analyze Hugging Face datasets with a single line of code.
SDXL Inference Speed and Memory Optimization Guide
It introduces how to apply fp16 and use memory-efficient attention to speed up inference and reduce memory usage for the SDXL model.
PPO-Based RLHF Implementation Guide and Reproduction Results
By reproducing OpenAI's RLHF code, key technical details to watch out for when implementing PPO were compiled.
Hugging Face optimizes embedding model deployment
It guides how to deploy high-performance embedding models via Hugging Face Inference Endpoints.
Gradio-Lite Released for Browsers
Gradio-Lite has been released, allowing Gradio apps to run directly in the browser using Pyodide without any server infrastructure.
ONNX Runtime Accelerates 130,000 Models
ONNX Runtime supports over 130,000 models on Hugging Face, accelerating inference performance.
Hugging Face Prevents LLM Formatting Errors with Chat Template
Hugging Face has introduced a Chat Template feature that automatically applies the chat format used during model training.
Accelerating SDXL Inference with JAX and TPU v5e
Hugging Face Diffusers now supports SDXL inference on Cloud TPU v5e via JAX, improving performance and cost efficiency.