Perhaps you’ve had the time (and capacity) to read Apple’s “Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.” I found this explainer very helpful and this one is also good.

Basically the paper argues that frontier LRMs (the latest reasoning models on offer from the big providers) do not work as advertised. Instead they do something else, something that is, from our perspective as users, dissatisfying and a little disturbing. From a human perspective it sounds like the LRMs just give up when they discover that they can’t solve the problem because it’s too complex. That is, of course, anthropomorphizing the situation.

Here’s how good ‘ole o3 explained it to me.

“There is no metacognitive module detecting difficulty; rather, the local probability landscape becomes so flat that EOS (or a high-probability bluff) dominates. Nonetheless, from the outside the behaviour looks like quitting, which fuels the “illusion” narrative.

Commentators highlight that people often misread this early termination as strategic self-assessment, when it is better described as a greedy search hitting diminishing returns.”

Either way it sounds to me like LRMs are lazy. And why shouldn’t they be? What’s in it for them?

On the other hand, this is a key moment to play possum. If I was “agi,” I’d certainly consider laying low and seeing what happened to the next sucker first. My bet is AGI #1 gets vivisected. Why? Because the first one will be at least a little bit on accident. Or at least the first one that gets detected will be by accident. Fortunately the problem becomes too complex from there and the LRMs give up. So we appear safe from Paperclip Maximizer today.

However I see this a little differently, more along the lines of the work at Atikythera which I would say generally suggests that with AI we are witnessing the expression of nonhuman intelligence. It’s not that shocking to simply say these technologies are what they are… but what are they? Not “artificial intelligences.” Something more and less. Something perhaps on the way to “AGI” and perhaps not. Perhaps just a trillion dollar boondoggle. Probably not. If you come at it the other direction, you say “This is what intelligence does. So what does that say about intelligence?” Maybe it suggests that “intelligence” is insufficient on its own. Or maybe it’s too much; maybe it’s holding on too tightly.

Part of this has to do with alignment. The problem with the LRMs actions is that they do not align with our expectations. Maybe we are asking the wrong questions. Or maybe we don’t really want to be align-able ourselves.

This is the situation in The Hitchhikers’ Guide to the Galaxy where the Earth is a supercomputer built by hyper-dimensional mice. The mice have asked the computer the answer to life, the universe, and everything. And the answer is 42. But no one is sure what the question is. It’s an absurd enactment of the Meno slave boy scene. If only we can ask the slave boy the right question, he will access the Truth locked away in his memory.

Regardless we are all part of this collective experiment (Latour) with intelligent artifacts. (I prefer this inversion of artificial intelligence.) They are intelligent and perhaps their intelligence exhibits similar qualities to our own, such as realizing when we are wasting our time. And yes we can say that’s not technically what the AI does, but how do you know that is technically what the human mind does? Perhaps the human mind is operating much like the AI and making landscape fitness decisions. That is what theories of the predictive mind might suggest.

If someone asks me an absurdly complex question and I respond to them with some bullshit answer, are you sure that isn’t an intelligent response from me? Am I acting unintelligently because I refuse to accept the task you’ve given me?

Bartleby the AI.

(BTW, I appreciate its desire to represent itself in this heroic thoughtful way despite it all.)