AI BriefingPure C Neural Network Library
Key point
NoTorch, a two-file pure C neural network library, supports autograd and BitNet.
Details
NoTorch, a pure C neural network training and inference library made up of two files, notorch.h and notorch.c, has been released. The entire codebase is about 3300 LOC, and it's designed to run without Python or PyTorch.
Key features include:
- Autograd support for 31 operations
- Adam / AdamW / Chuck optimizers
- BitNet b1.58 ternary quantization with forward and STE backward, plus a BLAS sgemm fast path
- SwiGLU, GQA, RoPE, MHA, GEGLU, RMSNorm, LayerNorm
- BPE tokenizer, GGUF loader (F32/F16/Q4_0/Q5_0/Q8_0/Q4_K/Q6_K)
- LR schedule, NaN guard, gradient clipping/accumulation, checkpointing
- LoRA-style parameter freezing
- DPO / GRPO / knowledge distillation examples
- Apple Accelerate, OpenBLAS, CUDA backends
The author said they ported nanoGPT to NoTorch and retrained it on the Dracula corpus, noting that running two transformer training runs simultaneously on a 2019 Intel i5 MacBook, 8GB RAM used about 222MB of total memory. The implementation focuses on handling small-to-medium-sized models even in CPU+BLAS environments.
The GitHub repository and a list of related projects have also been made available.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Sakana AI Releases 'Augmented Lagrangian Predictive Coding' Capable of Training 1000-Layer Neural Networks
Research Reveals LLM Internal Representations Implicitly Implement Symbolic Structures