HR JUDGMENT CHALLENGE 001
THE COLLABORATION
SCORE
It's quarterly bonus time. A collaboration score, built from recorded meeting data, feeds straight into pay. One of your strongest performers just came in four points under threshold.
What would you do?
THE SITUATION
Top decile on the team. Four points under threshold.
It's the quarterly bonus cycle, and a new collaboration score feeds directly into it.
The model watches recorded meetings, generates a score, and the organization uses it to help decide bonus pay. Daniel's output is top decile on the team — every project on time, every one of them above benchmark.
His collaboration score is 61. The threshold is 65. The team average is 78.
Four points below threshold.
MAKE THE CALL
What do you do?
Choose before you continue.
WATCH THE CHALLENGE
Coming October 5.
The video for this challenge publishes October 5. Check back then — or work through the situation and questions below in the meantime.
THE JUDGMENT PROBLEM
What the model saw isn't everything the decision is about.
The score may be completely accurate about meetings. What happens outside them can still require judgment.
Teammates tag Daniel for help outside meetings. He responds fast across teams. Other people's work references his contributions. None of it happens on camera — none of it reaches the model.
Same person. Different picture, depending on what you let measure him.
A narrow lens can produce a confident score anyway. The blind spot stays invisible until it isn't.
THE MEASUREMENT BOUNDARY
The score isn't wrong about what it saw.
The question is whether that was enough to see.
A model can learn a real pattern from recorded meetings and still be measuring only a fraction of how someone actually collaborates. Accuracy inside the frame says nothing about what's outside it.
THE JUDGMENT DIFFERENCE
Accurate is not the same as enough.
Is the score accurate?
Is the evidence enough for this decision?
A model can be useful for coaching and still not be enough for pay. The consequence attached to a score changes how much evidence it needs behind it.
BETTER QUESTIONS
Before the score affects pay, ask:
What can this score actually see?
What does it structurally miss?
Does this proxy have enough evidence behind it for the consequence attached to it?
Daniel's case exposes the question. It doesn't answer it — is this one exception, or a wider pattern in how the score measures everyone?
SO, WHAT WOULD I DO?
Hold it until the score's boundary has been reviewed.
Not because Daniel is or isn't a good collaborator. This isn't about whether he is — it's about whether the score can actually tell you that.
A delay has a cost too. But I'd rather explain a short wait than let an untested proxy cut his pay.
THE HR JUDGMENT TAKEAWAY
Don't mistake what the model saw for what it claims to measure.
Match the evidence to the consequence attached to it.
A narrow lens can still produce a confident score.
Is the evidence behind this score strong enough for what you're using it to decide?
That question comes before the score touches pay — not after.
BRING THE CHALLENGE TO YOUR TEAM
Make judgment visible.
Judgment Challenges can be explored with your team through a private 90-minute Judgment Lab, turning the scenario into a practical conversation about proxies, measurement boundaries and pay decisions.