Nobody Decided That

Consider a sales team that has used an AI ranking for eighteen months.

Accounts below the line receive less attention. Managers can override the ranking, but almost nobody does. Everyone knows how the system works.

Nobody can say when the ranking became the priority.

Nobody decided that. At least, nobody remembers deciding it.

A decision that was never made

It would be easier to understand if someone had made a bad call. There would be a meeting, a memo, a name. We could ask what they were thinking.

What happened was smaller and more ordinary. A system was introduced to help people prioritize. It ranked the accounts, and the ranking was useful. A line was drawn somewhere, for a practical reason that made sense at the time. People began opening the ranking before they opened their own notes. The weekly meeting was organized around it, because that was convenient.

Each step was reasonable. None of them was announced as a decision. Over eighteen months, a tool that helped people prioritize became the way priority was set.

In Why Better Judgment Matters in the Age of AI, I wrote that delegation is a choice and surrender is a drift. Organizations drift too.

Where the judgments enter

Nobody has to be careless for this to happen. In an AI-influenced decision, human judgment enters at three moments.

Before the system produces anything, someone decides what deserves to matter: what it should aim for, what counts as success, what is left out.

When the output arrives, someone decides what the evidence actually supports: what the result means, and how much it establishes.

Then someone decides what deserves action: what happens because of the output, and who can disagree.

At each moment, the judgment can be made deliberately, or it can be made by default. From the outside the difference is invisible. From the inside it often is too.

Why this matters

Once defaults and assumptions have hardened into practice, an organization can no longer tell two things apart: what the system actually says, and what people have built around it.

People defend the second as if it were the first. “The system says so” becomes the answer, when the system is only part of the reason.

That makes the decision hard to question. A challenge to the practice feels like a challenge to the technology, and the technology is working.

A system can work exactly as designed while the decision around it moves somewhere nobody chose.

Why a technical review is not enough

Organizations ask good questions about their AI. Is it accurate? Is it secure? Is it compliant? These are questions about the system, and they deserve real answers.

A technical review can tell you important things about whether the system works. It cannot, by itself, tell you whether the decision around it is sound, because that question also depends on objectives, interpretation, authority and consequences outside the model itself.

Someone has to look

Looking is not an accusation. Often the finding is reassuring: the objective was chosen deliberately, the output is used for what it can support, and the people responsible have real room to disagree. That is worth knowing.

When it is not reassuring, it is better found by the people who own the decision than by a bad outcome.

You do not need an outside examiner to begin. Take one decision that matters and ask: when was this decided, and by whom? If the answer comes quickly and clearly, good. If it does not, you have found something worth examining.

That is the work I do in a Better Judgment Decision Audit: one real decision, examined with the people who own and use it.

The question is not only whether the system works. It is whether the judgments around it were ever made deliberately.

Explore the Better Judgment Decision Audit →