Training on both Mac and T4

With Apple's provided 12GB checkpoint, regular LoRA hit memory limits.

• Standard LoRA required about 24GB to load the model and about 15GB GPU for training, and even a 24GB Mac ran into OOM.
• To address this, a custom QLoRA pipeline combining memory-mapped loading + 4-bit quantization was built.
• This approach reduced memory to about 1GB RAM and training GPU memory to about 5GB, making it runnable on a free Colab T4 and a 24GB Mac.
• On Mac, the not-yet-officially-released bitsandbytes Metal kernels (PR #1875) were installed via git, giving about 2x speedup over CPU fallback, but still about 4x slower than T4.
• A100 LoRA, T4 QLoRA, and Mac QLoRA all showed the same adapter quality, with accuracy rising from about 40% → 75% using just a small training set, and reaching about 86% when combined with retrieval.
• Additionally, a bug was found where each CLI tool call accumulated an adapter copy in the SIP-protected cache, causing about 160MB of accumulation, and it was confirmed that this led to 269GB of hidden disk usage over about 300 benchmark runs.