LLM Steering Gets Interesting Again with DeepSeek-V4-Flash

DwarfStar 4 is a project that optimizes llama.cpp specifically for DeepSeek-V4-Flash, making LLM Steering experiments easier to run in a local environment.

LLM Steering is a technique that extracts an activation vector representing a specific concept in the model (e.g., "answer concisely") and adds it to the relevant layer during inference, thereby directly controlling the model's behavior.

How it mainly works:

• Simple steering vectors: Compute the difference in activations between specific prompt pairs (a normal request vs. a specific request) to generate a vector, then add it during inference.
• Sophisticated feature extraction: Use techniques such as Sparse Autoencoders to extract and control deeper behavioral patterns (features) inside the model.

The value and limitations of steering:

• Advantages: It can act as a more direct "control panel" than prompt engineering, offering the possibility of adjusting attributes like tone or diligence like a slider. It can also be used to save context window space by compressing concepts that would require many tokens into a single vector.
• Limitations: Since it requires access to the model's weights and activations, it is not possible for API users, making local execution essential. Also, in many cases, modifying the prompt can be simpler and more efficient.