Research identifies LLM 'slop' style origins in SFT and DPO stages and proposes LoRA-based mitigation

Research into the origins of recognizable LLM assistant style ("slop") reveals that specific rhetorical structures are introduced during Supervised Fine-Tuning (SFT) and amplified by Direct Preference Optimization (DPO). Using Qwen3-4B-Instruct, a difference-in-means direction separates exaggerated writing from plain text, correlating up to 0.82 with a trope-based slop score. Logit-lens readouts identify a register axis where grand, abstract vocabulary opposes plain hedges, with cross-lingual directions such as "not X, but Y" promoting the Chinese term "而是".

Training Stage Impact

Measurements on OLMo 2 1B across released stages show that the base model possesses the slop direction (cosine similarity 0.90 to 0.97 at middle layers) but writes near the plain end. SFT introduces rhetorical structures, while DPO doubles the slop score through buzzwords and markdown formatting. Analysis of OLMo's preference data indicates the chosen response is the sloppier one in 57% of 3,000 pairs.

Mitigation via LoRA

Fixed-vector subtraction at layers 12 to 23 reduces structural tropes within a narrow KL budget (~0.4 nats/token). Distilling this steering into a rank-16 LoRA cuts the slop score by 83% with no system prompt, maintaining lower KL divergence than direct steering and showing no drop in QA performance. The study notes limitations, including reliance on a custom regex detector, testing on only two models, and the absence of human evaluation.