microduck_rl: Physics Simulation and Real-World Control Policy Learning for an 800g Bipedal Robot

Provides a reinforcement learning environment for Microduck, a compact bipedal robot weighing 800g and standing 25cm tall. Utilizing the mjlab framework based on MuJoCo Warp and the PPO algorithm, it trains various motion policies such as walking on flat ground, recovering from falls, kicking a ball, and roller skating. The trained policies are exported in ONNX format and immediately executed on the actual robot's runtime.

To reduce the sim-to-real gap, BAM actuators were applied, precisely modeling the voltage control laws and friction characteristics of the Dynamixel XL330 servos. Gear backlash of up to 2 degrees is simulated for each joint, and domain randomization includes battery voltage drops and command delays to enhance physical realism.

All policies are designed to share a 61-dimensional observation vector, allowing policies to be hot-swapped between walking, standing up, and skill motions during runtime. If you do not have a GPU, you can run training in the cloud via Hugging Face Jobs. Tools are also provided to directly control and verify policies using a keyboard in the CPU simulator.