Somewhere, right now, a smart and capable professional is about to trust an AI system that is confidently, fluently wrong. We know this because it keeps happening—increasingly, in places where the stakes are highest.
This past spring, a federal appeals court sanctioned attorneys for filing briefs containing AI-generated fictitious cases and citations. This joins a rapidly growing list: a California attorney fined for AI-invented quotations, a KPMG report found riddled with fabricated citations, even fake cases slipping into an order a judge ultimately signed.
In every one of these cases, there was a human in the loop. And in every one, the human failed—not because the technology malfunctioned, but because the person stopped acting as a decision maker and started acting as a rubber stamp for whatever the machine produced.
That failure is not just a story about careless lawyers. It is a warning about how we are teaching an entire generation of professionals to think about their own role in the age of AI, and the popular phrase “human-in-the-loop” is making it worse.
Human-in-the-loop escapes the lab
Human judgment is not a safeguard to be added to AI to catch its mistakes; it’s the entire point. The moment we treat it as optional oversight, we begin training people to defer to machines instead of thinking for themselves.
Here’s the irony: when engineers coined “human-in-the-loop,” they meant something precise: a system design in which a person intervenes at some point in an otherwise automated process. From the perspective of the engineer building that workflow, it makes sense as the human is one component in a larger machine.
But the phrase has escaped the engineering lab. It is now used to describe how lawyers, physicians, professors, researchers, and managers are supposed to work alongside AI—and in that context, it has it exactly backwards.
No professional experiences themselves as a component in a machine’s workflow. They experience themselves as central to the work, because they are.
When they borrow an engineer’s term to describe human judgment, they quietly demote the human from decision maker to safety mechanism.
The language casts the human as overseer, validator, fallback when automation fails rather than as the decision maker, the interpreter, the ethical actor, the creator of meaning. It treats human judgment as an add-on to the process rather than the purpose of the process.
I understand why the phrase took hold. Automation bias is real and well documented. A landmark study by Linda J. Skitka, Kathleen Mosier, and colleagues found that participants often followed automated decision aids even when contradictory evidence was available.
The mere presence of automation reduced independent information gathering and critical evaluation. “Human-in-the-loop” was meant as a guardrail against exactly this drift.
But the guardrail concedes the wrong premise. If humans are drifting out of their own decisions, the answer is to strengthen judgment, not to write human passivity into our vocabulary as though it were the natural order.
Prioritizing responsibility
I am not immune to the temptation either. A few months ago, in a rush, I asked AI to compile a list of my recent articles and speaking engagements. It produced an impressive list that sounded exactly like me.
But then I looked closer, and realized that several entries were actually hallucinations of plausible work I had simply never written. They were perhaps good ideas for future projects, but they were not true, and only I could know that.
As AI becomes more capable of producing answers, our obligation to cultivate judgment becomes even more urgent, not less. AI can generate possibilities, but only people can determine what is true, what is meaningful, and what is right.
If we fail to preserve that distinction, we risk educating students to simply become supervisors of machines rather than independent thinkers.
So let’s retire “human-in-the-loop” where it doesn’t belong. Educators should stop teaching AI literacy as tool adoption and start teaching it as judgment cultivation.
Institutions should adopt language that centers human responsibility rather than human oversight. The human was never one part of the loop, dropped in to catch the machine’s mistakes.
The human is the loop.
