Study Finds Post-Training Increases LLM Humor but Reduces Response Diversity

A new study investigates whether post-training enhances the humor of Large Language Models (LLMs) by comparing base models with their post-trained counterparts, specifically Tulu 3 (based on Llama 3.1 70B), OLMo 3.1 32B, and Qwen2.5. The research tracked 11 stages of training using 100 joke prompts, evaluated by 64 human raters across 2,330 head-to-head judgments.

Key Findings

• Increased Humor: In 5 of 7 training steps, the post-trained models produced jokes that humans judged to be funnier than those from the base models.
• Reduced Diversity: In 6 of 7 steps, the variety of jokes generated for the same prompt decreased. For example, asking for eight jokes on a single premise often resulted in eight variations of the same joke. The most significant drop in diversity occurred when moving from Qwen2.5 base to instruct.
• Shorter Punchlines: Jokes became 10–20 words shorter after early post-training stages, allowing models to reach the punchline more quickly.

Prompting Strategies

The study also tested specific prompting techniques:

• Planning: Asking the model to plan a line or two before the joke reduced variety in all 4 models tested, with no reliable improvement in funniness.
• Persona: Adopting a comedian persona recovered some variety across all 4 models, but only increased funniness in 2 of them.

The evaluation methodology involved humans judging the base versus final models, while a calibrated model judge compared the intermediate training stages.