ReviewDesk
← Back

ReviewDesk guide · By the Ingenious AI team · Published 14 November 2026 · Reviewed 14 November 2026

What should an annual ADMT programme review actually examine?

Generate a free adverse-outcome letter — no sign-up Build a free pre-use notice See pricing

The short answer. Re-read every notice against the live process. Re-run every scope determination against fresh measurements. Sample the year's letters and review notes for quality drift. Read coverage, breaches and correction patterns as one story. Test the vault export cold. Refresh the roles and the vendor answers. An annual review that does those six things — minuted, with owners and dates — is the programme's guarantee that last year's machinery still describes this year's operation.

What is the annual programme review?

An annual ADMT programme review is the organisation's structured self-examination of its automated-decision operation, distinct from (and feeding) California's statutory risk assessment: a yearly pass over scope, notices, correspondence quality, inbox performance, records integrity and roles, producing dated findings with owners. In ReviewDesk terms it consumes the workspace's own artefacts — workflows, decision histories, vault exports and packs.

How does the programme review differ from the risk assessment?

Audience and altitude. The California risk assessment is a defined pack about processing risks, due 1 April 2028 in ReviewDesk's model, described in the assessment guide. The programme review is the wider internal examination that makes the assessment's answers true: it checks the operation itself — whether notices match tools, whether letters kept their standard, whether the bench's authority is real. The assessment reports the programme; the review repairs it. Organisations that conflate them produce assessments describing a programme nobody inspected, which is the annual version of a letter describing a decision nobody made.

Sequence them: programme review in the autumn, findings actioned through the winter, assessment assembled in January from a programme whose claims were verified a season earlier. The rhythm converts the assessment from an act of authorship into an act of reporting — the same conversion the whole evidence philosophy keeps performing at every scale.

Examination one: scope, re-measured

Pull fresh numbers for every decision type — acceptance rates, review times, bypass usage — and re-run the determinations of the scope-test guide against them. The year's changes are the examiners: vendor model updates, threshold tunings, new decision types absorbed into existing workflows, volume growth that turned review into triage. Each gets a fresh worksheet entry, appended, dated. The review's hardest question belongs here: what did we start deciding this year — or start letting the tool decide — without noticing? Answer it from the tool inventory and the change log, not from memory, and expect at least one uncomfortable answer; a year with none usually means the inventory is incomplete.

New uses discovered in the review get the full treatment immediately — scope filed, notice built, workflow stood up — with the gap period recorded honestly. The recording matters: a scope gap found and closed by your own review is a governance success story; the same gap found by someone else's is a finding. Self-discovery is the entire economic argument for the annual review's existence.

Examination two: notices and letters, re-read cold

Read every live notice aloud, against the current process, hunting the drifts catalogued in the notice guide: factors changed, routes stale, timeframes no longer met, a vendor name retired. Then sample the year's letters — a spread across decision types and outcomes — against the five-part anatomy of the examples guide: reasons still specific, values still anchored, jurisdiction notes still right, review offers still at full size. Correspondence drifts by millimetres — a paragraph softened here, a route shortened there — and only the annual cold read sees the accumulated centimetres.

Include the confirmation and status texts in the sample; they are the most-read documents in the programme and the least reviewed. And read two complete files as their subjects would: the notice you were shown, the letter you received, the status page you watched, the outcome you got. Coherence across that journey is the programme's user experience, and no dashboard reports it.

Examination three: the inbox's year, as one story

Coverage first — received, answered, median days, oldest cases, breaches with causes — then the patterns: overturn rate and its movement, correction share by source, request mix by decision type, escalation frequency. Read together, these numbers narrate the year: a rising correction share on one factor is a data story; a falling median with thinning notes is a quality story; a breach cluster in one month is a capacity story. The review's job is to name each story and attach its action — the discipline of the coverage guide applied at annual scale.

Interview the bench as part of the examination. Reviewers know which cases the system mishandles, which evidence never arrives, which criteria they privately consider unworkable — knowledge that exists nowhere in the records. An annual hour of structured listening, minuted into findings, is the cheapest primary research in the programme. Skip it, and the review examines the programme's paper while its people hold the truth.

Examination four: records, tested not trusted

Run the cold tests: one random closed case exported and reconstructed by a stranger to it, per the vault guide; trace counts reconciled against decision logs for a sample month; the retention schedule's deletions verified as executed; the access list read name by name. Records work is believed exactly as far as it is tested, and the annual review is where the testing is scheduled rather than hoped for. Every failure found is a routing or discipline fix with an owner — the same conversion the quarterly tests perform, at audit depth.

Examine the 'not recorded' census across the packs as its own finding. The gaps' count and composition — shrinking, static, growing; concentrated in one workflow or spread — is the programme's instrumentation health in one view. A review that celebrates closed gaps and names the owners of the open ones keeps the honesty engine of the evidence guide running for another year.

Examination five: roles, vendors and the year ahead

Refresh the role sheet from the roles guide: names, deputies, authorities still accurate after the year's joiners and leavers. Re-run the vendor diligence questions whose answers decay — versions shipped versus versions notified, correction service levels actually met, export tested once more. Then look forward: the coming year's known dates from the calendar, the planned decision types and their scope work, the capacity the request trend implies. The review closes with a dated findings list — owner and deadline per item — and the date of next year's review booked before the room disperses.

Keep the whole artefact short enough to be read next year. Ten honest pages with a findings table outlive, in usefulness, the sixty-page audit liturgy nobody reopens. The review's reader is next year's reviewer, reconstructing what this year knew; write for them, file it with the programme's records, and let the chain of annual reviews become the programme's long memory — the institutional version of the decision history's discipline, one level up.

Your five actions for the annual programme review

  1. Re-measure every scope determination against fresh acceptance and review-time numbers; hunt specifically for unnoticed new uses.
  2. Re-read all notices against the live process and sample the year's letters, confirmations and outcome notes for drift.
  3. Read the inbox's year as one story — coverage, overturns, corrections by source, breaches by cause — and interview the bench.
  4. Test the records cold: random-case export, trace reconciliation, retention deletions verified, access list read aloud.
  5. Refresh roles and vendor answers, book next year's dates, and minute every finding with an owner and a deadline.

Worked example: the review that found the fourth decision

A fictional operator — Cascade Homes, from the assessment guide — runs its first annual programme review. Scope re-measurement runs cleanly for its three known workflows. The tool-inventory hunt does not: a property-management platform update, shipped in spring, quietly enabled a 'tenancy health score' that two property managers had begun consulting on renewal decisions. No notice, no workflow, forty-one renewals touched. The review records the finding in its own words, stands the workflow up within the month — notice, letters, inbox routing — and carries the gap period honestly in the year's assessment narrative.

Leadership's verdict on the review is the one this guide predicts: the finding was the value. A programme whose own examination catches its drift, documents it and closes it inside a quarter has demonstrated the exact competence the regimes exist to produce. The forty-one renewals were handled with letters and review offers issued retrospectively — imperfect, recorded, and infinitely better than the alternative timeline in which a tenant's adviser found the fourth decision first.

Frequently asked questions

How is a programme review different from the California risk assessment?

The assessment is the defined statutory pack about processing risks. The programme review is the wider internal examination — scope, notices, letter quality, inbox performance, records tests, roles — that makes the assessment's claims true. Run the review in the autumn; assemble the assessment from it in January.

What is the single most valuable examination?

Scope, re-measured — because drift hides there. Vendor updates and absorbed decision types create unnoticed new uses; the review that hunts them from the tool inventory and change log finds the programme's real risk before anyone else does.

How should findings be recorded?

As a dated list with an owner and a deadline per item, filed with the programme's records, short enough to be re-read next year. Next year's reviewer is the document's primary reader.

Should reviewers be interviewed?

Yes — an hour of structured listening. The bench holds knowledge about mishandled cases, missing evidence and unworkable criteria that exists nowhere in the records, and the annual review is where it becomes findings.

Is this guide legal advice?

No. ReviewDesk is templates and workflow tooling, not legal advice. The review is an operating practice from ReviewDesk's model; consult counsel for your obligations.

Keep reading: How do you train hiring managers to work inside an ADMT notice and review process? · How do you choose ADMT compliance software? (A buyer's checklist) · How should you handle a formal complaint about an automated decision? · How do you prepare for future automated-decision rules without knowing what they will say? · What must a Colorado pre-use notice for an automated decision tool say? · All ReviewDesk guides · ReviewDesk home · Try the free desk · Free letter generator · Free notice builder · Pricing

ReviewDesk is built by Ingenious AI. Templates and workflow tools — not legal advice.