Marginalium

I loved this. Why our conversations about AI are so weird to me. I have complained about this before. But this author does a better job:

When a coffee shop makes a bagel, it’s a pretty good bet they can make a croissant as well. Not every shop that has one has the other, but they’re pretty strongly correlated. We call this correlation “baking” …

Now imagine a coffee shop that’s been tasked to “achieve superbaking”. They make one bagel on Monday, ten bagels on Tuesday, and eighty million bagels on Wednesday. They’ve never made a croissant. Have they achieved superbaking?

A flat no seems like a silly answer. They went from a pathetically subhuman number of bagels to an outrageously superhuman number of bagels really quickly. If someone says “True baking includes croissants” as a way to dismiss it out of hand, that’s a pretty lazy denial of the obvious truth that something wild is going on back there …

What I’m contending here is that the word “intelligence” is like the word “baking” and it’s long past time we actually sit down and sort the bagels from the croissants. I am strongly against arguments of the form “Oh, it’s just parroting the data set - it’s not really thinking.” AI does a lot of things that we call “thinking” when we do them slower and worse. The fact it can also do those things should make us humble and curious, not proud and dismissive. But I think it’s equally silly to lump all these capacities together into “intelligence” and say “Intelligence is going up, so soon it will do everything intelligence can do.” You need to see some croissants before you conclude it’s actually baking and not just bageling.

And then they go and outline a lot of these exact strange distinctions. Very nice.