vLLM Hands-on Workshop Recap

Rebellions and SqueezeBits jointly held the vLLM Hands-on Workshop on October 29, emphasizing that it was the first developer-focused hands-on event led by a domestic NPU company.

In the participant survey, everyone responded with ‘satisfied’ or higher, and the biggest reaction was the experience of running vLLM directly on Rebellions NPU. It was also positively noted that developers familiar with PyTorch could follow along without any extra setup.

The workshop directly addressed representative questions about adopting NPUs.

• Whether existing code or MLOps pipelines need to be significantly changed
• How naturally it integrates with Hugging Face, PyTorch, and vLLM
• How MoE (Mixture-of-Experts) models can be efficiently distributed across NPUs

The hands-on environment was built on ATOM™ NPU servers and a Kubernetes-based resource pool, with a dedicated NPU node allocated to each participant. In the first session, participants experienced PyTorch Eager Mode, which operates with the same code structure as GPU, running tensor operations on NPU with just a single command, and also performed Phi-3 0.6B model inference using Hugging Face Transformers.

In the subsequent Graph Mode session, participants used RBLN Profiler and Perfetto to analyze NPU execution traces and pipeline structure at the tile level. They also compiled Prefill and Decode graphs separately and applied optimization kernels such as FlashAttention and KV Caching, confirming improved performance compared to Eager Mode.

The core of the event was the vLLM-Rebellions backend plugin. Based on torch.compile, this plugin connects vLLM to Rebellions hardware and directly implements Paged Attention and Continuous Batching on NPU, improving memory efficiency and throughput in real deployment environments.

It also provides an OpenAI-compatible API endpoint, allowing developers to leverage Rebellions' hardware acceleration without significantly changing their existing API workflows. In the final demo, distributed inference of the Qwen1.5-MoE model showed tensor parallelism and expert parallelism being used together, and Rebellions announced that it is preparing official MoE support for vLLM-RBLN going forward.

Beyond a simple demo, the workshop presented a production-grade Kubernetes-based hands-on environment where Hugging Face, PyTorch, and vLLM naturally connect within an NPU environment. Rebellions and SqueezeBits plan to continue regular workshops for the next 2 years, helping Korean developers master NPU-based vLLM deployment at a service level.