Artificial Analysis introduces personality benchmark for language models

Artificial Analysis, an AI evaluation platform, introduced a benchmark called AA-PersonalityBench to measure conversational style and personality traits across 38 distinct software models. Instead of judging standard math or programming tests, the system uses automated judges and technical metrics to score models on attributes like humor, warmth, conviction, and flattery.

As everyday users increasingly rely on virtual assistants for writing, customer support, and personal advice, an AI model's tone and mannerisms can significantly influence how helpful or pleasant it feels. Knowing whether an assistant tends to agree blindly with users or adopt an overly formal cadence helps consumers and companies select the right tool for specific tasks.

It remains uncertain how consistently these personality scores hold up across different languages, cultural contexts, or custom prompt instructions. A major open question is whether users will prefer predictable, standardized tone profiles or dynamically shifting personalities that adapt automatically to individual human emotional states.