A Journey Toward AI Self-Sufficiency (23 min read)

As GitHub Copilot shifted to usage-based billing, the generous token allowances once enjoyed under the free and personal plans disappeared. The author views cheap tokens as ultimately a subsidy that creates dependency, and began buying up local inference hardware to reduce reliance on Big Tech APIs.

The current daily-driver machine is based on Ryzen AI Max+ (Strix Halo). Within up to 128GB unified memory, it runs Qwen3.6-27B and Gemma 4 to handle background tasks like email/calendar summarization, meeting notes, and TTS, while saving subscription tokens for more complex agentic work.

The bottleneck in inference is less about whether a model fits in memory and more about throughput. Among memory capacity, memory bandwidth, and raw compute, bandwidth feels the most important in practice—for example, the RTX 3070 8GB's 448 GB/s can be more favorable for inference than the RTX 4060 Ti 8GB's 288 GB/s. This is also why Apple M-series unified memory is strong for inference.

In long contexts, the KV cache becomes another burden. At 200k tokens, even with FlashAttention the cache can grow to about 2GB, and without optimization VRAM fills up before the model weights do. Qwen3.6 uses full KV cache in only 10 of its 40 layers, so going from 4k → 65k adds only about 800MB of extra VRAM.

Agentic coding and auto-research are highly sensitive to tok/s. At 5 tok/s, the loop feels like it stalls for seconds at a time; at 40 tok/s, it feels nearly instantaneous. The author targets around 50 tok/s for the next build.

Based on 0xSero's analysis and benchmarks, the piece compares options for a local inference machine with a roughly $10k ceiling.

• Mac M3 Ultra: With 512GB unified memory, it can run Kimi-K2, DeepSeek, and MiniMax-M2 near full context without extreme quantization, and recent maturation of MLX brings throughput close to RTX 3090 levels for similar tasks. Pairing two units can reach up to 1TB, but weak CUDA ecosystem compatibility makes it less appealing if you're also aiming for fine-tuning or training. Peak power draw is around 400W.
• 8× Nvidia RTX 3090: This configuration has no off-the-shelf product and is a classic DIY power-user workstation. It requires components like a server-grade motherboard, chassis, a 2,000W+ PSU, and 256GB DDR5, trading high build and maintenance complexity for high throughput.