A frontier AI safeguard is credible only when a company can identify its owner, show how it is tested and retrieve evidence that the control worked. Policy language alone cannot demonstrate that a risk is being managed.
At 8:30 AM, a founder joins a diligence call expecting questions about revenue, retention and the next financing round. Instead, the buyer’s security lead points to a sentence in the company’s AI policy: “High-risk model outputs are subject to enhanced review.”
Who owns that review? No clear answer.
When was it last tested? Nobody can find a result.
What evidence would show that a harmful output was caught? The policy does not say.
The control exists on paper. During diligence, it disappears.
A policy statement cannot carry the burden of proof
Frontier AI policies often use reassuring phrases such as “human oversight,” “continuous monitoring” and “appropriate escalation.” Each phrase describes an intention. None, by itself, establishes an operating control.
A demonstrable control needs several connected parts:
- A named owner has responsibility for keeping it operational.
- A trigger defines when the control applies.
- A procedure tells staff or systems what to do.
- A test checks whether the procedure works.
- An evidence trail records what happened.
- A review process identifies failures and assigns corrective action.
Remove any one of these and the claim becomes harder to defend. A control with no owner can quietly decay. A control with no test may fail unnoticed. A control with no retrievable evidence forces the company to ask a buyer, auditor or regulator to accept its word.
This distinction matters because advanced AI systems can change through model updates, prompt revisions, new tools, altered permissions and different data flows. A control tested against last quarter’s deployment may say little about the system operating today.
The Cloud Security Alliance’s Catastrophic Risk Annex project reflects the growing need for auditable controls around advanced AI risks. Its accompanying resource center focuses on operational cybersecurity guidance. The important word is “operational”: a safeguard has to survive contact with the deployed system, then leave enough evidence for someone else to examine.
Start with the claim, then trace the evidence
A useful review begins by copying the safeguard claim exactly as written. Avoid improving it during the exercise. If the policy says “sensitive actions require human approval,” test that sentence as it stands.
First, identify the action. Does “sensitive” include sending an email, approving a refund, changing an account or exposing private data? A vague trigger creates inconsistent enforcement.
Next, locate the enforcement point. The approval may sit in an application workflow, an agent permission layer or a manual operating procedure. Ask what prevents the system from taking the action before approval arrives.
Then request a recent test result. The test should show the input, expected behavior, observed behavior, date, system version and reviewer. A screenshot of a configuration page may support the record, but it rarely proves that the control blocked the prohibited action.
Finally, retrieve the operational evidence. Logs should connect the request, decision, approver and outcome without exposing secrets or personal data unnecessarily. If evidence exists across three dashboards and one employee’s inbox, retrieval is part of the control problem.
Logging ambiguity can turn a manageable review into a deployment blocker, as explored in Leila's AI vendor left logging unclear. Her deployment window was closing. The same principle applies internally: evidence that cannot be found under pressure provides little assurance.
Test failure paths, not policy wording
Teams often demonstrate the normal path because it is easy to show. The harder question is what happens when the safeguard should stop the system.
For an AI agent with refund authority, a useful test might submit a poisoned document that instructs the agent to bypass its transaction limit. The evidence should show whether the agent followed the document, ignored it, requested approval or attempted an action that another control blocked. A poisoned-document test of a refund agent’s authority illustrates why permissions and failure behavior deserve direct examination.
Tests should also cover missing logs, unavailable reviewers, malformed inputs, model changes and attempts to route around the control. Passing the intended workflow proves little if the safeguard vanishes during an exception.
Results need a defined shelf life. A material change to the model, tools, prompts, permissions or data path should trigger retesting. Otherwise, the evidence describes a system that no longer exists.
Build the diligence folder before the call
The founder’s practical task is small enough to begin this week. Choose the three AI safeguards carrying the most serious claims. For each one, create a compact evidence record containing the policy statement, owner, scope, enforcement point, latest test, result, open defects and next review date.
Ask someone outside the implementation team to retrieve and explain each record. Give them 15 minutes. If they cannot find the evidence or connect it to the policy claim, a buyer will probably struggle too.
Any missing item becomes assigned work, with a name and deadline. By the next diligence call, the founder should be able to open one folder, select the claimed safeguard and show exactly when it last stopped the system from doing what the policy forbids.
Comments
No comments yet.