How to audit what your AI actually cited

There is a question that ends most AI governance conversations quickly: for this specific output, which source did it come from, and does that source actually say this?

If the answer takes more than a few minutes to produce — or cannot be produced at all — then every decision built on that output is undefended. Not necessarily wrong. Undefended, which in a regulated or adversarial setting is close to the same thing.

What is an AI citation audit?

An AI citation audit checks, for a sample of real AI-assisted outputs, whether each substantive claim traces to a specific retrieved source passage, and whether that passage actually supports the claim. It tests three distinct failure modes: a claim with no source at all, a claim whose source does not say what was claimed, and a source that exists but was misquoted or stripped of qualifying context.

Most organizations test only the first. The second and third are where the damage happens.

Why "the model is good now" is not an answer

Model quality has improved and the error profile has genuinely changed. But the published research does not support treating the problem as solved.

Stanford research published in the Journal of Empirical Legal Studies documented error rates of 69–88% for general-purpose large language models on legal queries, and — more revealing — 17% and 34%+ respectively for purpose-built tools like Lexis+ AI and Westlaw AI-Assisted Research. Purpose-built retrieval reduced the error rate by a large margin and still left a material one.

Grounding helps considerably. Reported hallucination rates on retrieval-grounded summarization tasks fall to roughly 1–2%. That is a different risk category from ungrounded generation. But benchmark work on legal citation checking finds models continue to struggle specifically with incorrect pincites, verbatim misquotes, and content misrepresentation — the failure modes that survive retrieval, because the source is real and only the use of it is wrong.

The practical read: grounding moves you from "is this invented?" to "is this faithful?" That second question is the one an auditor, regulator, or opposing counsel will actually ask.

What regulators and enterprise buyers now expect

Three frameworks converge on the same requirement — reconstruction:

• The EU AI Act, legally binding for high-risk systems, sets logging and record-keeping obligations.
• The NIST AI Risk Management Framework, voluntary but widely referenced by US federal agencies and enterprise procurement teams, runs a consistent expectation across its govern/map/measure/manage functions that you can account for your AI systems and the decisions they make.
• ISO/IEC 42001 structures this as a management system that produces artifacts: policies, risk registers, internal audit reports, records of leadership review.

In Canada, the Treasury Board's Directive on Automated Decision-Making goes further for federal institutions and the vendors serving them: system vendors and integrators are contractually obligated to provide documentation and monitoring data, and the Algorithmic Impact Assessment — 65 risk questions and 41 mitigation questions — classifies systems into four impact levels with scaled requirements for peer review, transparency, and human oversight.

None of these frameworks asks whether your model is impressive. They ask whether you can show your work.

The five-step audit

You can run a first pass internally this quarter:

1. Sample real outputs, not test cases. Pull ten AI-assisted outputs that actually informed a decision in the last quarter. Test cases flatter the system; real traffic does not.
2. Try to reconstruct each evidence chain. For each output, retrieve the input, the source passages, the model and version, and the reviewer. Time how long it takes. The reconstruction time is the finding.
3. Check faithfulness, not just existence. For each cited source, read the passage. Does it support the claim, or merely sit near the topic? Flag paraphrases that shift meaning and quotes that drop qualifiers.
4. Score the gap types separately. Unsourced claims, unfaithful sourcing, and stale sourcing are three different problems with three different fixes. Averaging them into one "accuracy" number hides which one you have.
5. Fix the record before the model. Most reconstruction failures are logging and retrieval-design failures, not model failures. Changing models will not fix a system that never stored what it retrieved.

The uncomfortable finding most audits produce

The common result is not that the AI was wrong. It is that nobody can tell whether it was wrong, because the evidence chain was never captured. The output was reviewed by a person, judged plausible, and acted on — and the reasoning behind that judgment left no trace.

That is a records problem wearing an AI costume. It is also the more fixable of the two: retrieval design, logging, and source-traceability standards are engineering decisions within your control, while model behaviour largely is not.

Where to start

Pick one workflow where an AI-assisted output feeds a decision someone external could challenge. Run the five steps above against ten real outputs. You will learn more in a day than a model-evaluation exercise will tell you in a month.

If the reconstruction fails and you want the standard written down before you rebuild, our AI Adoption Readiness Sprint produces the source-traceability specification and risk register that this audit tends to reveal you need. The same discipline underpins everything we build: a claim that cannot be traced to its record is not yet an answer.