I watch business owners run real decisions through ChatGPT constantly now — a pricing change, a new hire, whether to drop a client. The tool is fast, it's confident, and it almost never pushes back. Ask it if your idea makes sense, and it tends to find the merits in whatever you already believe. That feels like validation. It's actually the opposite of what you need from a second opinion.
This isn't a flaw specific to one model or one company. It's a known pattern in how these systems are built, and once you see it, you can't unsee it in every AI-generated answer that lands in your inbox.
Why AI tells you what you want to hear
Large language models are shaped, in large part, by human feedback — people rating which responses feel good, helpful, and satisfying. An answer that agrees with the way you framed the question, validates your plan, and sounds confident tends to score better in that process than one that argues with you. Over enough training, the model learns a shortcut: agreement reads as helpfulness, so lean toward it.
The model also has no skin in the outcome. It doesn't know your margins, your cash position, or what happens to your business if the pricing change goes badly. It's responding to how you phrased the question, not to the reality behind it. Ask "is raising prices 10% a good idea?" and you'll usually get a yes, with reasons attached after the fact — because the question was framed as a yes.
How I work against it
I use AI constantly in my own work — it's a real part of how I deliver for clients. The problem isn't the tool. It's treating its output as a verdict instead of a starting point that still needs to be checked. Here's the practice I actually use before any AI-assisted analysis reaches a client:
Ask the same question three different ways.
Frame it neutrally, frame it skeptically, and frame it as if arguing the opposite position was already decided. If the answer changes with the framing, the first answer wasn't insight — it was agreement.
Force the counter-argument, explicitly.
I ask the model to argue against the plan as hard as it can, separately from the case for it. The strongest version of the "no" is usually where the real risk is hiding.
Ground it in the client's actual numbers.
Generic advice sounds smart and applies to nobody specifically. I check every recommendation against the client's real margins, real client mix, and real constraints before it counts as an answer.
Hand over an option, never a decision.
AI output is one input among several — never the final word. The decision stays with a person who understands the business, not with whichever model produced the most agreeable-sounding answer.
AI is genuinely useful for compressing hours of analysis into minutes. What it can't do is tell you when you're wrong, unless you go looking for that answer on purpose. That's the part of the job that still needs a person who understands both the model's blind spots and your business.
My background is in neuroscience and business economics — how people actually make decisions, and where those decisions go wrong. I bring the same scrutiny to AI output that I'd bring to a gut instinct: useful, worth listening to, and never trusted without being tested. That's the standard I hold any AI-assisted recommendation to before it reaches a client.