Credit Decisioning: Designing for Data You Do Not Control

The scoring model gets the attention. It’s the visible part, it has a number attached, and everyone in the room has an opinion about it.

It’s rarely what’s broken. The difficulty sits upstream, in the data feeding it: several external providers, each with its own format and its own latency, and each with a particular way of being unavailable at the worst possible moment.

That, and the rules keep changing. What counts as a relevant signal one quarter doesn’t always hold the next.

The problem is usually framed wrong

The request normally arrives as “we need to improve the model.”

Sometimes that’s right. More often the model is fine and the system around it is fragile: a provider times out and the application silently fails, or a rule change takes three weeks because the logic is tangled into four integrations.

So before touching any scoring, here’s what I’d want to understand.

Which of these data points actually change the decision? Teams accumulate providers. Somebody integrated a source two years ago for a rule that no longer exists, and it’s still being called on every application, still costing money per lookup. Ask what each source is for. Some of them won’t have an answer.

What should happen when a provider is slow, down, or returns something strange? Not what does happen. What should. It needs a real answer per source: decline, approve on what you have, hold for manual review, or retry. Nobody enjoys deciding this in the abstract, which is exactly why it ends up being decided at 2am during an incident by whoever is on call.

And the one that usually finds the real problem: how often do the rules change, who changes them, and how long does a change take to ship today? If the answer is “a developer, three weeks”, that’s what you’re actually fixing, whatever the brief said.

Separate the rules from the sources

The single design decision that matters most here: the thing that decides and the things that supply data should not be the same code.

Put each provider behind a boundary that does two jobs. It normalises whatever that provider returns into terms your decision logic understands, and it defines what happens when the provider fails.

The decisioning logic then works on normalised inputs and never learns that one bureau returns dates differently from another.

Two things follow, and they’re the two that hurt most when they’re missing:

• A rule change touches the rules, not the integrations.
• Swapping or adding a provider touches one boundary, not the decision.

Without that separation every scorecard adjustment reopens integration code, and every provider change puts the decision itself at risk. That’s how a two-line rule change turns into a three-week release, and why the team’s estimate sounds absurd to everyone outside it.

Decide what “we don’t know” means

Most credit systems are designed for the happy path, where every provider answers and the data is complete.

Real applications aren’t like that. A bureau is degraded. An open banking connection returns ninety days instead of twelve months. A document is missing.

You can’t prevent any of that. What you can decide is what the system should conclude when it happens. An application with partial data is not the same as an application that failed the rules, and systems that treat them identically quietly decline people they wanted.

So there’s usually a third outcome to design, sitting between approve and decline: held, with the specific missing thing named, and a route back once it arrives.

Write down why, not just what

Whatever the regulatory position, someone will eventually ask why a particular application was decided the way it was. If the honest answer is “that’s what the rules said at the time”, you need to know what the rules were at the time.

That means storing the inputs the decision actually saw, the version of the rules applied, and the outcome. Reconstructing a decision from today’s rules and today’s data tells you very little about a decision made eight months ago.

Nobody puts this in the first release. It is, reliably, among the first things someone asks for afterwards, usually with more urgency than the original request had.

Where I’d start

Map every external source against the rules that use it, then mark the ones nothing depends on any more.

That list is almost always shorter than the number of integrations you’re paying for, and the gap between the two is the cheapest win available. It also tells you which providers are worth designing carefully around and which are just history.

I’ve spent a fair amount of time on decisioning systems where data from several providers had to come together in real time behind a lending decision, and where the integrations had to bend every time the rules moved without breaking underneath them. The longer version is in this credit decisioning case study.

The per-provider mechanics - contracts, failure handling, monitoring - are the same in any external integration, and I’ve written those up as an API integration checklist.