NVIDIA Unveils Medical Robotics Development Workflow

NVIDIA has released Isaac for Healthcare v0.4, an integrated pipeline for developing medical robots, unveiling an SO-ARM-based starter workflow that spans from simulation to real hardware deployment.

This workflow provides a 3-stage pipeline that enables medtech developers to rapidly build surgical assistance robots:

• Data collection: Collect teleoperation data from both simulation and real environments using SO-101 and LeRobot.
• Model training: Post-train the GR00T N1.5 model on the combined dataset, leveraging dual-camera vision.
• Policy deployment: Perform real-time inference on actual physical hardware via RTI DDS communication.

The core technology, the Sim-to-Real hybrid training approach, combines about 70 simulation episodes with 10-20 real-world data points, securing both the efficiency of simulation and the precision of real environments at the same time. In fact, more than 93% of the data used for training was generated through simulation.

Hardware requirements include an RT Core-based GPU (Ampere or higher, 30GB VRAM or more) and an SO-ARM101 manipulator, and the entire process can be run on a single DGX Spark workstation.