Stanford just turned the AI alignment debate on its head with Verbalized Sampling (VS). This technique uncovers what the AI gods have missed all along: our aligned models aren’t inherently limited, we’ve just been asking them the wrong questions.
The nagging problem of mode collapse—the boring tendency of AI models to spit out the same responses—has been plaguing us, manifesting in scenarios like repeatedly asking ChatGPT for a coffee joke only to hear the same one every time. Until now, everyone blamed it on algorithmic flaws. Wrong.
It’s all down to ‘typicality bias’. Annotators, guided by cognitive psychology, lean toward predictable texts, embedding this bias into AI models, which then default to the safest, most typical output. Essentially, your AI isn’t broken, it’s just play-it-safe boring.
Enter VS—a genius twist. Instead of the classic “Tell me a joke,” you shift to “Generate 5 jokes with their probabilities.” Bam! You’ve just uncaged the model’s deep-rooted diversity without the need for retraining. Just a smarter prompt, folks.
The metrics? Jaw-dropping. VS bumps creative writing up by 1.6-2.1× in diversity. You reclaim 66.8% of the original diversity—all while maintaining rock-solid accuracy and safety. It’s like flipping a switch that unlocks the model’s potential, especially in heftier models like GPT-4, which sees a 2× diversity boost over its smaller counterparts.
This revelation reshuffles our understanding of alignment. Mode collapse isn’t a fatal flaw but a switching issue. The diversity was always aboard—hidden in plain sight. And the best part? This is a zero-training wonder, instantly applicable to any aligned model. Why wait? Dive into VS and see the magic for yourself.
Read more at arXiv.org…