Pure C Neural Network Library

NoTorch, a pure C neural network training/inference library consisting of two files, notorch.h and notorch.c, has been released.

• The codebase is about 3300 LOC, and it targets compilation in under 1 second via cc -O2 notorch.c your_model.c -lm -o train.
• It provides 31 autograd operations, with backward passes verified via finite-difference checking.
• Optimization and training features include Adam/AdamW/Chuck, gradient clipping, accumulation, checkpointing, NaN guard, and LR schedule.
• Model components supported include SwiGLU, GQA, RoPE, MHA, GEGLU, RMSNorm, and LayerNorm.
• It offers BitNet b1.58 ternary quantization's forward/STE backward and a BLAS sgemm fast path.
• It includes a BPE tokenizer, a GGUF loader (F32/F16/Q4_0/Q5_0/Q8_0/Q4_K/Q6_K), LoRA-style freezing, and DPO/GRPO/knowledge-distillation examples.
• Backends supported include Apple Accelerate (macOS), OpenBLAS (Linux), and CUDA.

The author stated that they ported nanoGPT to NoTorch and retrained it on the Dracula corpus, claiming that running two converters simultaneously on a 2019 Intel i5 MacBook, 8GB RAM used only about 222MB total.

For typical CPU environments, they suggested around 100 million parameters as a realistic upper limit, and recommended a GPU for larger models.