FINANCE JUDGMENT CHALLENGE 003
THE EXCLUDED
VARIABLE
A credit model excludes race, gender and zip code by design. The bank's fairness audit still finds a substantial approval gap — traced to two variables nobody thought twice about.
What would you do?
THE SITUATION
The design excluded the variable. The pattern showed up anyway.
A regional bank's credit-scoring model was designed to comply with fair-lending requirements. Race, gender, marital status and zip code are excluded from its inputs.
The model uses income, employment history, years at current address, and university selectivity as creditworthiness signals — each with a real, independently defensible link to default risk.
The bank's fairness audit controls for the bank's other major risk factors, and still finds a substantial approval gap associated with demographic group. Further testing traces most of it to two variables: university selectivity and years at current address. Removing those two variables would measurably weaken the model's predictive accuracy.
The model never saw race. It produced a similar pattern anyway.
MAKE THE CALL
What do you do?
Choose before you continue.
WATCH THE CHALLENGE
Coming January 21.
The video for this challenge publishes January 21. Check back then — or work through the situation and questions below in the meantime.
THE JUDGMENT PROBLEM
The rule was followed. The pattern it was meant to prevent showed up anyway.
The model wasn't asked to discriminate, and it didn't. It was asked to predict default risk using variables nobody chose because of demographic association — and it did exactly that.
The question isn't whether the bank followed its own design rules. It did. It's whether removing a variable actually removes its effect, or just removes your ability to see that effect operating through something else.
A system can follow every constraint built into it and still reproduce a pattern those constraints were intended to reduce.
The real mistake is mistaking absence from the input list for absence from the decision process.
THE PROXY PROBLEM
Absent from the inputs isn't the same as absent from the outcome.
You can remove race. Other variables can still carry information correlated with it.
AI can exclude a variable perfectly. Judgment determines whether what it represented is still there.
THE JUDGMENT DIFFERENCE
Compliant is not the same as fair.
Race, gender, zip code.
Do the remaining variables produce the same pattern anyway?
The rule was followed. The outcome it was meant to prevent showed up regardless.
BETTER QUESTIONS
Before you change the model, ask:
Is the correlation earning its own predictive power, or mostly borrowing someone else's?
What would the accuracy loss actually cost — and is that number worth keeping variables this entangled with a demographic pattern?
Does manual review on borderline cases meaningfully change outcomes, or just add cost without closing the gap?
This isn't about assuming the model discriminated. It's about whether the exclusion actually did what it was designed to do.
SO, WHAT WOULD I DO?
Each option is defensible — depending on whether this is a proxy or a real risk factor.
A retains real predictive value — on the unproven assumption the correlation is legitimate. B removes the flagged variables — on the equally unproven assumption they're mostly a proxy. C preserves accuracy while adding a check — but review is slower and doesn't guarantee closing the gap. D investigates before acting — but the pattern continues, confirmed, while the study runs.
This isn't about assuming the model discriminated. It's about whether the exclusion actually did what it was designed to do.
THE FINANCE JUDGMENT TAKEAWAY
Removing a variable removes the variable — not necessarily what it represented.
Correlation isn't automatically proof of a proxy, but it isn't automatically nothing either.
Checking the inputs isn't enough. Check what they produce together.
Removing a variable removes the variable.
It doesn't guarantee it removes what it represented.
BRING THE CHALLENGE TO YOUR TEAM
Make judgment visible.
Judgment Challenges can be explored with your team through a private 90-minute Judgment Lab, turning the scenario into a practical conversation about fairness, proxy variables, and what compliance actually verifies.