The Approval That Wasn't a Decision
Why vendor risk programmes fail under scrutiny — and what a defensible decision actually requires
1. The Debrief
The call comes eighteen months after you approved the vendor.
It is not a routine check-in. The vendor has had an incident. Data was exposed. The scope is still being determined. Legal is on the line. Your regulator has sent a preliminary information request. The first question on the request is straightforward:
Provide documentation of your third-party risk assessment for this vendor, including the basis on which approval was granted and the evidence reviewed at the time of the decision.
You go to the system. You find the assessment. What you find is this:
— A completed questionnaire — eighty-three questions, mostly Boolean, submitted by the vendor eleven days before approval.
— A risk register entry showing GREEN, with a score of 74 out of 100, and the label 'Approved — Tier 2.'
— An email thread in which a colleague wrote 'looks fine to me' and the CISO replied 'agreed, proceed.'
— A SharePoint folder containing a SOC 2 Type II report, a penetration test summary, and a data processing agreement.
You send these to legal. Legal sends them to the regulator. The regulator replies with a follow-up:
Please confirm: (a) which specific controls were evaluated and on what evidentiary basis; (b) the methodology by which the score of 74 was derived; (c) the identity of the legal entity assessed; (d) who was accountable for the decision and under what authority.
This is where most organisations discover the gap. Not that the work wasn't done — it probably was, in some form. The gap is that no artifact exists which answers these questions. What exists is activity. What the regulator is asking for is a decision.
These are not the same thing.
2. Why Questionnaires Feel Like Risk Management — But Aren't
The questionnaire became the default instrument of vendor risk management for understandable reasons. It is scalable. It creates a record. It asks questions. It generates a score. And in a world where regulators ask 'did you conduct due diligence,' a completed questionnaire is defensible evidence that something was done.
The problem is not the questionnaire. The problem is what questionnaire tools were designed to produce.
Questionnaire tools are optimised for collection. They are built to get answers back from vendors efficiently, to track completion, to store responses. They have no model for what an answer is worth. A vendor ticking 'Yes, we have a patch management policy' produces exactly the same output whether that claim is backed by an audited ISO 27001 certification, a policy document reviewed internally last quarter, or a Google Doc someone wrote the week before the assessment.
The tool records the response. It does not evaluate the evidence. It cannot distinguish between a claim and a proof.
Response completeness is not evidence sufficiency. This is not a gap in execution. It is a category error.
This distinction matters because regulators under FCA SS2/21, DORA Article 28, and equivalent frameworks do not ask whether vendors were questioned. They ask whether risks were assessed, whether controls were verified, and whether decisions were documented. Verification is a different activity from collection. Documentation is a different artifact from a completed form.
Most vendor risk programmes have built robust collection infrastructure and called it a verification programme. The two look similar from the outside. They produce very different results when examined from the inside — particularly under the forensic conditions of an incident debrief or a Section 166 review.
The organisations that discover this gap tend to discover it at the worst possible moment.
3. Three Failure Modes
Vendor risk programmes fail under scrutiny in characteristic ways. These are not random failures. They are structural — produced reliably by the same design choices repeated across organisations, sectors, and tools. Understanding the structure of the failure is a prerequisite for building a programme that survives examination.
Failure Mode 1: Entity Identity Mismatch
You assessed a vendor. The question is: which one?
Most vendor risk programmes operate against brand names. You assessed 'Acme Cloud.' You have an entry in your risk register for 'Acme Cloud.' You have a folder of documents about Acme Cloud. But 'Acme Cloud' is not a legal entity. The legal entity providing your contracted service is Acme Cloud Services, a UK subsidiary of Acme Holdings GmbH, a German parent. The SOC 2 Type II you hold in your evidence folder was issued to Acme Holdings GmbH. The data processing agreement is with Acme Cloud Services. These are different legal persons.
In most incidents, this discrepancy is invisible until it becomes material. The moment a regulator asks which entity was approved, or a breach occurs and legal needs to establish who had contractual liability for the controls that failed, the mismatch surfaces. The evidence you hold does not cover the entity you contracted. Your assessment is attached to a brand, not a legal entity. It may not be defensible.
This is not an edge case. Vendor corporate structures are routinely complex, involve intermediate holding companies, and change through acquisition. Assessing the brand rather than the registered entity is the default behaviour of almost every vendor risk tool on the market, because almost no tool performs entity resolution — the act of establishing precisely which registered legal entity is the counterparty before any risk work begins.
You did not approve a brand. You approved a legal entity — or you should have. If you cannot say which one, the approval is incomplete.
Failure Mode 2: The Coverage Illusion
Evidence was received. The question is: what does it cover?
The SOC 2 Type II report is the most commonly misapplied piece of evidence in vendor risk management. Organisations treat its presence as a comprehensive signal of control maturity. In many cases, it is not.
A SOC 2 report covers a specific system boundary, a specific set of trust service criteria, and a specific time period. The audit opinion is valid for the scope described in the report — and only for that scope. An organisation using a vendor's SaaS application layer may receive a SOC 2 report that covers the vendor's data centre infrastructure, physical security, and network operations. If the application layer is explicitly excluded from the report's scope — a common feature of tiered service architectures — the evidence is real and the coverage is zero for the risk you are actually managing.
Most questionnaire tools have no model for coverage. Evidence is either present or absent. A document uploaded to the evidence folder is treated as covering the controls it was uploaded against. Whether the document actually speaks to those controls — whether its scope aligns with the service being contracted, whether exceptions noted in the report are material to your use case — is not evaluated. It is assumed.
The result is a confidence that is borrowed from a document that does not speak to the actual risk. The assessment looks complete. The protection is not there.
Evidence received is not evidence applied. Coverage is not implied by presence. A document in the folder is not the same as a control that is verified.
Failure Mode 3: Timestamp Decay
The decision was made. The question is: when, and to what state of the vendor's controls?
Vendor risk assessments have a shelf life. The evidence underlying an approval was accurate at a point in time. Controls change. Certifications expire. Audit findings from a SOC 2 report issued eighteen months ago describe the vendor's posture at the time of that audit — not now. The penetration test that satisfied your technical security requirements when the vendor was onboarded may cover a version of the application that no longer exists.
Most risk registers carry the approval forward indefinitely, or until a reassessment is manually triggered. The GREEN status that was valid at the time of approval continues to display against the vendor record regardless of what has changed in the intervening period. When an incident occurs, the register shows GREEN — because nobody updated it, because nothing triggered a reassessment, because the platform has no model for evidence decay.
But the deeper problem is not the stale status. It is that even if a reassessment was triggered and completed, there may be no frozen record of what was known at the moment the original approval was given. If legal asks 'what did you know about this vendor's patch management controls at the time of the incident,' and the answer is 'we cannot reconstruct that because the assessment was overwritten by the renewal,' the original approval is effectively undocumentable.
An approval without a frozen, timestamped record of the state of knowledge that supported it is not a decision. It is a historical event with no surviving artifact.
4. What a Decision Actually Requires
The three failure modes above are not independent. They share a common root: vendor risk programmes have been designed around the process of assessment rather than the production of a decision. The output of most programmes is an approval. That is not the same as a decision — at least not a defensible one.
A defensible vendor decision requires four things that questionnaire collection cannot provide.
Legal entity certainty
The decision must be attached to a specific registered legal entity — not a brand name, not a trading name, not a product. The counterparty to the contract, the entity whose controls are relevant, the organisation that a regulator can identify and investigate, must be established before any risk work begins. This is not administrative hygiene. It is the foundation of everything that follows. Evidence that does not attach to the correct entity does not cover the risk you approved.
Evidence with evaluated coverage
Evidence must be evaluated on two axes: how strong the evidence type is, and how much of the relevant control it actually covers. An audited certification is stronger than a vendor attestation. An attestation that covers the full control is more valuable than a certification that covers half of it. These are not binary distinctions. They require a model — a methodology for translating evidence into a confidence level that is derivable, explainable, and consistent. 'We reviewed the documents and were satisfied' is not a methodology. It is a judgment. Judgment is not reproducible. Reproducibility is what makes a decision defensible.
A frozen, timestamped record
The state of knowledge at the moment of decision must be permanently captured. Not as a status in a live system that reflects the current view of the vendor, but as an immutable artifact that records what was assessed, what evidence was evaluated, what confidence was derived, and what decision was reached — at a specific point in time, against a specific version of the vendor's controls. This record must survive platform migrations, vendor reassessments, and changes in personnel. It must be retrievable years after the decision was made. It must not be editable after the fact.
A derivable output
The decision must follow from the inputs by a methodology that can be explained and replicated. Given the same evidence, evaluated against the same controls, the same conclusion must be produced — not because the system forces it, but because the methodology is consistent enough that another reasonable assessor, given the same information, would reach the same result. This is what 'defensible' means in the context of a regulatory inquiry. Not that the decision was correct, but that it was reasoned — that it was the product of a process that a regulator can examine, follow, and evaluate.
These four requirements describe a decision engine — not a questionnaire tool. The category of tool most organisations are using is simply the wrong tool for the job they believe they are doing.
5. Determinism as a Legal Property
The word 'deterministic' sounds like a technical property. It is not. Or rather, it is not only that.
In engineering, determinism means that a system given the same inputs always produces the same outputs. In the context of vendor risk, this property has a legal significance that is often missed: it is the foundation of reproducibility, and reproducibility is the foundation of accountability.
Consider what a regulator or auditor is actually asking when they review a vendor decision. They are not, primarily, asking whether the decision was correct. They are asking whether the decision was made — whether it was the product of a defined, consistent methodology applied to a specific body of evidence, rather than the accumulated judgment of people who have since left the organisation, applying criteria that were never written down.
Judgment-based programmes fail this test structurally. Not because the people exercising judgment are incompetent or negligent, but because judgment is inherently personal, contextual, and non-reproducible. One assessor sees a SOC 2 with two minor exceptions and rates the vendor Moderate. Another assessor, at a different organisation, or in a different week, sees the same report and rates it High. Both are exercising reasonable professional judgment. Neither is producing a defensible decision, because neither is applying a methodology that could be examined and validated by a third party.
A deterministic methodology does not eliminate judgment. It structures it. It defines where judgment operates — in the design of the methodology — rather than in the application of it, assessment by assessment, vendor by vendor, assessor by assessor.
This matters for a specific regulatory reason. FCA SS2/21 requires firms to demonstrate that their outsourcing risk frameworks produce consistent, documented outputs. DORA Article 28 requires that ICT third-party risk assessments are conducted according to a defined methodology. Neither regulation prescribes what the methodology must be. Both require that one exists, that it is applied consistently, and that its outputs can be produced on request.
A deterministic risk engine satisfies this requirement structurally. The methodology is encoded, not described. The outputs are produced by the methodology, not by the assessors applying it. The same vendor, assessed under the same conditions, will always produce the same result. This is not a constraint on human judgment — it is the infrastructure that makes human oversight accountable, rather than merely present.
The override exists — there will be cases where a senior leader needs to approve an exception to a hard gate, where business requirements require accepting a risk that the methodology would block. But the override is structured, time-limited, and auditable. It is an explicit act of governance, not an invisible exercise of discretion. The exception is documented. The rule is preserved. And the documentation of the exception is itself part of the decision record.
6. The Artifact That Survives the Platform
Dashboards go stale. Risk registers reflect the current state, not the historical one. Scores in live systems are updated when new information arrives, overwriting the basis on which the original decision was made. Personnel change. Platforms are migrated. Evidence folders are reorganised. Emails are archived. And the institutional memory of why a vendor was approved, on what basis, with what caveats, under whose authority — dissipates.
What survives is a document.
Not a dashboard screenshot. Not a PDF export of a live report. A document that was generated at the moment of decision, frozen at that moment, containing a complete record of the state of knowledge that supported the decision: the legal entity assessed, the controls evaluated, the evidence reviewed and its coverage, the confidence derived, the decision reached, the person accountable, the timestamp. And a cryptographic hash — a fingerprint that allows any third party, at any point in the future, to verify that the document has not been altered since it was generated.
This document is what the industry has spent years generating activity to support — and consistently failing to produce.
The activity is real. The questionnaires are completed. The evidence is collected. The scores are calculated. The approvals are given. But the decision — the artifact that survives — is not there. What exists is a trail of inputs. What is missing is the output that a regulator can examine, a court can admit, an auditor can verify, and a board can understand.
The purpose of a vendor risk programme is not to conduct assessments. It is to produce defensible decisions. These are related activities. They are not the same activity.
Organisations that have built their programmes around collection — gathering responses, filing evidence, maintaining registers — have built infrastructure for the first activity. They have not built infrastructure for the second. The gap is not visible during normal operations. Risk registers look healthy. Scores look reasonable. Approvals are on file. The gap becomes visible at the moment of examination — an incident, an inquiry, a regulatory review — when the question shifts from 'did you assess this vendor' to 'show me the decision.'
That shift is not a new regulatory development. It is not the product of increased enforcement activity or changing guidance. It is the natural consequence of any event that requires accountability to be traced back to a specific act, by a specific person, on a specific basis, at a specific time.
The vendor risk programmes that survive these moments are the ones that produce decisions, not approvals. Decisions that are reasoned, reproducible, frozen at the moment they are made, and attached to the correct legal entity. Decisions that exist as artifacts independent of the platform that produced them.
Everything else is activity. Activity is necessary. It is not sufficient.
About this paper
This paper was written for risk, compliance, and audit professionals in FCA-regulated financial services and institutions subject to DORA. It does not advocate a specific product or vendor. It examines a structural problem that is, in our view, underdiagnosed — and consequential in proportion to how long it goes unaddressed.
KnightGrid builds deterministic vendor risk infrastructure for regulated financial institutions. If the arguments in this paper resonate with your experience, we would welcome a conversation.
