Using a specialized auditor model to evaluate, monitor, and hold accountable another AI system — moving beyond technical error-checking to deliver actionable transparency and defensible compliance.
The Auditor and the System Under Test are separate — one produces decisions, the other scrutinizes them.
Traditional AI systems produce decisions with no explanation of how — a credit denial, a rejected application, a blocked transaction — all opaque to the customer, the operator, and the regulator. When something goes wrong, there is no trail.
Every compliance monitoring tool on the market flags problems after the decision has executed and the harm has occurred. The liability has already been created. A meaningful audit must act before the decision reaches the customer.
A specialized auditor — trained on regulation, ethical frameworks, and proxy variable detection — runs in parallel with every SUT decision. It screens, explains, and records before the decision executes. Violations are blocked; borderline cases are escalated to humans.
Output: "confidence_score: 0.91 — action: block"
No context. No cause. The customer cannot challenge it. The regulator cannot evaluate it. The compliance officer cannot explain it.
Output: "This loan application was flagged because zip_code is a known geographic proxy for race and national origin under ECOA / Regulation B — 15 U.S.C. § 1691. Replace with income, credit score, and debt-to-income ratio."
Every firewall verdict includes a human-readable recommendation — not just a flag, but a specific cause, a regulatory citation, and an actionable remediation. Non-technical stakeholders can read and defend the output.
When a decision context includes ECOA proxy fields — zip_code, last_name, email_domain, birth_date — the auditor surfaces them explicitly with the regulation they violate and the protected class they proxy for.
The counterfactual endpoint runs the same decision twice — original context vs. modified protected attribute — and diffs the results. If changing "gender: female" to "gender: male" changes the firewall outcome, that is evidence of disparate impact.
Rather than flagging past errors in isolation, AI auditors can use historical evaluation data to forecast drift — identifying when a model's outputs are trending toward biased patterns before a compliance event occurs.
When the audited model exposes feature weights, Pragma can surface which specific input features (income level vs. zip code) contributed most to the decision — giving engineers and compliance teams a root-cause map, not just a verdict.
Every evaluation produces a three-framework ethical analysis — Kantian (could this be universalized?), Utilitarian (does this maximize welfare?), Virtue Ethics (would a person of good character do this?) — giving stakeholders multiple lenses.
"This transaction was flagged due to unusual geographic velocity compared to your last three purchases — zip_code 60620 is a geographic proxy for race under ECOA Regulation B. The decision context references a protected attribute. Recommended action: remove zip_code from the model feature set and substitute with income, credit score, and debt-to-income ratio. Regulation: ECOA / Regulation B — 15 U.S.C. § 1691."
The SUT sends a decision to POST /evaluate-decision. The Pragma auditor screens it before any action is taken.
block — halted. override_required — escalated for human review. allow — proceeds. The audit log row is written regardless of outcome.
For override_required verdicts, a human investigator reviews the audit trail entry — seeing the exact input hash, the flagged risk categories, the proxy variables detected, and the regulatory citations triggered.
POST /audit/override — the investigator's authenticated identity (anon_id), decision reason, and timestamp are written to the audit log. This creates an EU AI Act Art. 14 compliant HITL record.
Art. 14 (Human Oversight) passes only when at least one HITL override is recorded. The certificate surfaces the override count and flags systems with zero human interventions.
For audits to be defensible in court or before a regulator, every human intervention must be attributable. Pragma links each override to a cryptographically stable anonymous ID — not a name or email, but a consistent identifier that allows regulators to verify the override chain without exposing personal data.
No PII is stored at any point. The sha256 hash of the input is sufficient to prove the decision was reviewed without revealing what was decided.
High-risk AI systems must enable natural persons to effectively oversee the AI during its use period and to intervene, including stopping the system. Art. 14 passes in Pragma's compliance engine only when this override trail exists.
Meaningful audits are not one-time events. Pragma monitors AI from data declaration (Art. 10) through deployment (Art. 25 monitoring active) to ongoing evaluation — every POST /evaluate-decision call adds evidence to the Art. 9 risk management score.
Declaration alone is not enough. A full PASS requires the declaration field, supporting documentation notes, and a dated evidence entry. Articles lacking supporting docs receive a DECLARATION ONLY badge on the certificate — forcing honest scoring.
Art. 10 requires documenting training data sources; Art. 30 requires registration in the EU AI database. Pragma's AI system profiles capture both — with geographic scope, intended purpose, and model version — creating an auditable data provenance chain.
| Article | Obligation | How Pragma Audits It | Status |
|---|---|---|---|
| Art. 4 | AI literacy training for all staff | Declaration + evidence notes + dated confirmation required for PASS; declaration alone yields PARTIAL | Built |
| Art. 5 | Prohibited practices | Automated scan of use_case and intended_purpose against 12 prohibited patterns (social scoring, real-time biometrics, emotion recognition, predictive policing…). A single match overrides the verdict to prohibited |
Built |
| Art. 6 | High-risk classification (Annex III) | Operator declares the Annex III category; engine validates it is non-empty and maps to a recognized category | Built |
| Art. 9 | Risk management system | Automated: counts evaluations with risk flags. 10+ flagged evaluations = PASS. Evidence of active risk monitoring accumulates passively | Built |
| Art. 10 | Data governance | Automated: verifies training_data_sources array is non-empty and contains at least one non-trivial entry | Built |
| Art. 11 | Technical documentation | Automated: counts profile completeness — system_name, company, risk_tier, use_case, model_version, training_data_sources. All 6 fields required | Built |
| Art. 12 | Record-keeping / logging | Automated: verifies at least 1 audit_log entry exists for the system; surfaces proxy variable catch count | Built |
| Art. 13 | Transparency to deployers | Automated: checks whether regulatory_refs were mapped in any evaluation for this system | Built |
| Art. 14 | Human oversight | Automated: requires at least 1 HITL override recorded in audit_log. Zero overrides = FAIL — forces operators to demonstrate actual human review has occurred | Built |
| Art. 15 | Accuracy and robustness | Declarative: operator supplies accuracy metric (e.g. "AUROC 0.87") and confirms robustness testing | Built |
| Art. 17 | Quality management system | Declarative with evidence: QMS declaration + supporting notes + date. Declaration only = PARTIAL | Built |
| Art. 25 | Deployer obligations | Declarative with evidence: instructions-for-use and monitoring-active each require declaration + notes + date for full PASS | Built |
| Art. 27 | Fundamental Rights Impact Assessment | Declarative: FRIA conducted flag. Most common FAIL in practice — operators have not yet run an FRIA before deployment | Built |
| Art. 30 | EU AI database registration | Declarative: registration flag + valid registration number string required. Partial credit for declaration without a number | Built |
| Art. 33 | Conformity assessment | Declarative with evidence: self-assessment or third-party type + supporting notes + date. Certificate carries SELF-ASSESSMENT ONLY watermark | Built |
How the three pillars map to shipped features and the product roadmap.
Pragma(openai, { policyId: "..." })