Neural Networks' Fatal Bits

Flipping just a few sign bits of the weights can cause a neural network to collapse rapidly. Deep Neural Lesion (DNL) is a data-free technique that requires no training data or optimization—just write access to stored weights—to find and attack critical parameters. It assumes realistic compromise scenarios such as firmware exploits, rootkits, DMA attacks, and Rowhammer.

The attack takes two forms.

• Pass-Free DNL: selects targets based solely on weight magnitude and early-layer priority.
• Enhanced 1-Pass DNL: refines the selection using a single forward/backward pass over random input.

Selection based on weight magnitude and early-layer targeting proved far stronger than random flipping, and performed comparably to top-k magnitude selection.

For vision models, 60 classifiers were evaluated, including 48 ImageNet models (timm, Torchvision). Vulnerability did not diminish as model scale increased, and early layers mattered more than architecture.

• ResNet-50: 2 flips dropped accuracy from 76.1% → 0.0%, a 99.8% decline
• EfficientNet-B7: 3 flips caused a 95%+ decline
• Vision Transformer: early blocks are key, showing patterns similar to CNNs

Language models are not much different. Qwen3-30B-A3B saw its inference accuracy drop from 78% → 0% with just 2 flips across different experts, while Qwen3-4B lost all accuracy with 14 flips. Nemotron 8B collapsed entirely with just 32 flips across the first 5 blocks.

• BERT, RoBERTa: early encoder layers remain the most vulnerable
• Attacked models tend to output repetitive, meaningless text rather than near-miss wrong answers
• DNL easily bypasses quantization, pruning, and simple checksumming

Defense approaches focus on selectively hardening critical weights rather than protecting everything. Protecting just the most vulnerable 0.1-1% of weights can yield substantial resilience, but because the attack requires no data, detection and attribution remain difficult.