AI BriefingSkyPilot-Hugging Face Integration Achieves Zero Data Transfer Fees
Key point
Through collaboration between SkyPilot and Hugging Face, models and datasets can now be directly mounted and used on GPU clusters across clouds without any data transfer costs.
Details
Hugging Face and SkyPilot have collaborated to launch a feature that lets you load data at no cost from wherever your GPUs are, regardless of where the data is stored.
Previously, when the cloud hosting your models and datasets differed from the cloud hosting your GPUs, data transfer fees (Egress fee) would occur, but this update resolves that issue.
Key Features:
- hf:// scheme support: Using
hf://URLs, you can instantly mount Hugging Face Buckets, models, and dataset repositories into SkyPilot jobs. - Zero data transfer costs: Since Hugging Face Storage does not charge Egress fees when reading data, you can load data from any cloud (CoreWeave, Nebius, GCP, etc.) without additional costs.
- Efficient data processing: Through Xet-based deduplication technology, only changed portions are transferred, and with hf-mount, data can be mounted and used like a local path.
- Flexible infrastructure utilization: Through SkyPilot, you can find available GPUs across 20+ clouds, Kubernetes, and on-premises environments and run workloads instantly.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Finnish AI Cloud Startup Verda Raises $189 Million in Series B
Nvidia in Negotiations to Acquire Hugging Face for 19.5 Trillion Won