VKORE-46M-NPU Claims Backpropagation Training on Inference-Only NPU: A Model Card Analysis

The VKORE-46M-NPU model card released on Hugging Face claims to have pre-trained a small language model from scratch without weights by performing backpropagation on an inference-only NPU. This challenges the prevailing notion that 'inference chips cannot be used for training.'

Verification Method and Results

The model adopts a decoder-only transformer architecture with approximately 46M parameters and was trained in an environment with 32 chips of the first-generation inference-only NPU, the Rebellions ATOM-Max series. The most notable aspect is the verification method. Training was conducted separately on NPU and GPU using the same initialization, data, and schedule, resulting in a loss trajectory correlation coefficient of 0.99 between the two. This is interpreted as strong evidence that backpropagation and optimizer updates on the NPU operate numerically almost identically to the GPU reference.

Technical Significance and Limitations

These results suggest the possibility of utilizing low-power accelerators deployed for inference as training resources during idle time, or performing on-device training while maintaining data sovereignty. However, this is a toy-scale experiment using a char-level tokenizer and the TinyStories dataset, and the core training techniques remain undisclosed due to patent applications. Scaling to large models and real-world data remains a future challenge.