Thinking Machines Trains Inkling 2 Small End-to-End in 4-Bit Precision

·2026.11.16 09:00

Key point

The team achieved an 8% pretraining and 20% RL speedup over MXFP8 with only a 2% loss in compute efficiency, using a mixed-precision recipe on NVIDIA GB300 hardware.

Details

  • Thinking Machines trained the Inkling 2 Small model end-to-end using native 4-bit forward passes, the largest scale for this precision, on NVIDIA Grace Blackwell Ultra (GB300) hardware.
  • The mixed-precision recipe uses NVFP4 for forward passes and MXFP8 for backward passes, delivering an 8% wall-clock speedup in pretraining and a 20% speedup in RL compared to full MXFP8 training.
  • Pretraining time for a model similar to Inkling 2 Small was reduced from several weeks to 68 hours on GB300 NVL72 racks, with a compute efficiency loss of approximately 2%.
  • Quantized communications resolve NVLink bottlenecks by dispatching tokens in NVFP4 and using a 32-dimensional random Hadamard transform (RHT) with S2F6 quantization for combine operations, maintaining model performance without measurable accuracy degradation.

Precision and Hardware

  • NVFP4 offers 15 PFLOPS throughput on GB300, compared to 5 PFLOPS for MXFP8 and 2.5 PFLOPS for BF16.
  • The recipe keeps the last 16.6% of layers in MXFP8 to maintain stability, while expert layers use NVFP4 with dedicated kernels.
  • In-house kernels codenamed BBBK (Big Beautiful Blackwell Kernel) saturate Tensor Cores, ALUs, and NVLink simultaneously using GB300-specific 2SM UMMA instructions.

Technical Implementation

  • Backward pass gradient bias is mitigated by re-quantizing forward tensors with stochastic rounding (SR) to ensure expected values match, supported by hardware acceleration on Blackwell GPUs.
  • The "Four Over Six" technique is applied to activation quantization to reduce error, while weight quantization uses per-expert outer scales.
  • S2F6 is an 8-bit fixed-point format (2 signed integer bits, 6 fractional bits) used for combine transmissions, reducing data volume from 16 bits to 8.25 bits per value.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.