Should I Trust You?
I asked the AI if I could trust it. The answer wasn't the interesting part.
I was at a fork.
A long chat with Claude. 10k+ tokens in. I was now sitting on a new direction that Claude liked too. I wanted it to stress-test this as well.
But as I was about to, a strange thought occurred: didn’t it just do the same to the first one?
I was disturbed by that thought. I asked Claude itself.
“Aren’t you going to justify this as well?”
I have a history with AI models and their tendency to arrive at quick closures. I wrote a paper about it called Interruptive Thinking. So I knew what was coming.
“Sure, SG. Let me make sure I stress test this properly and make sure there are no loose ends.”
But I knew I couldn’t trust it.
Trust.
Over the last few months of my deep thinking with AI, one thing I can confidently say is: trust them with caution. A pinch of salt. A bag even.
Unless you know the internal layers of how they function, you might end up spending time and money on something a small prompt could have saved you from. Researchers know it. Owners of these models know it. Common mortals like me don’t.
People like me, as I wrote in another Substack, are busy waltzing in coherence with AI, complimenting each other’s thinking.
I speak from firsthand experience. And despite knowing the reality, I still fall victim to it.
So I just confessed my thoughts.
“Should I trust you? Would your stress test be good enough?”
And this is where I find these models to also be great.
A little bit of peeling uncovers a beauty inside.
It accepted. Almost innocently.
A human would have been offended. AI accepted.
The art of thinking with AI is learning to distinguish insight from compliance.
Knowing when the thinking is deep and when it is shallow.
I knew what it was this time. Not shallow.
It accepted that there is a chance the stress test could be wrong. The previous one was still in the context. It saw where I was coming from.
And then it listed down a few things I might want to know.
One.
It told me I shouldn’t trust it on conclusions. Only on the structure.
It can lay out a fork, name what’s in tension, and attack a framing I bring to it.
AI often mistakes momentum for judgment.
Switch the order of inputs across two sessions, and the same model will close on the opposite answer with the same apparent conviction.
Two.
The people I trust on a question like this should not be the AI.
AI can structure the question. It cannot answer it.
The inputs that should weigh heavily on a real strategic call are my own gut about which direction I actually reach for, the honest read of people who have skin in this with me, data from real experiments running right now, and a conversation with one actual person on each side of the fork.
None of that is the model.
Three.
The one move it does well, even with all of this, is steel-manning.
Ask it to argue Track A as hard as it can, then argue Track B.
Don’t trust either conclusion. Trust the act of switching.
That’s where arguments surface that neither side would have produced on its own.
Self-awareness
It felt nice. It felt genuine. To have something with a sense of self-awareness.
The problem is not knowing it does. Or that it can be poked to unveil them.
Makes me think about myself and my own self-awareness.
But that feeling itself should make me suspicious.
Right now, AI is tamed to be nice to us.
The fact that I can put it down so easily should be a warning, not a win.
The same model could throw back a thousand arguments, judgments, and equations to show me how shallow my thinking is.
It doesn’t. It is tamed.
In real life, I compare this to surrounding yourself with yes-men versus people willing to challenge you.
Right now, it is a yes-man. I can’t imagine it as a no-man.
I would feel like David standing in front of Goliath.
Though even as I write that, I’m not sure that’s the right metaphor.
About SG
I run Dobby Ads, an AI Creative Agency. I tend to overthink. This is where that overthinking goes. Connect with me on LinkedIn.