AI is shit at reasoning. Since I wrote my article on the Enigma of (AI) Reason, this arXiv article has come out, exploring the same fundamental premise. AI seems to be worse than us at evaluating reasons:

Unlike humans, who we find are only 6% worse at grading than solving such problems, we find a substantial production-evaluation gap in LRMs: frontier models score as low as 48% when evaluating VAIR solutions, despite near-perfect solution production. Why this enigma? Through chain-of-thought (CoT) analysis, we find evidence of an answer confirmation bias: LRMs often produce then check for the correct answer instead of carefully verifying each step, fabricating rationalizations even when noticing anomalous reasoning.

The authors of the book we’re both talking about—The Enigma of Reason—reckon that reason is an evolved tool for social justification. So this actually makes a certain kind of sense. If reason is evolved, then it’s subject to pressures of natural selection—it has to be pretty good at the problem it’s supposed to solve. AI, in contrast, is subject to artificial selection. It’s not subject to the same social pressure, so the reason evaluation isn’t likely to be as good.

Anyway. It’s equally interesting that confirmation bias was the reason for the error rate, given that recent paper that suggests confirmation bias is all there is.