System design, model orchestration, API reference, and deployment guide.
A domain-specific ethical reasoning model — fine-tuned on structured analysis, improving via user feedback.
Claude claude-opus-4-6 acts as the teacher, labelling ethical scenarios with structured JSON output. The Pragma model (Phi-3-mini LoRA adapter) learns to replicate this reasoning — at a fraction of the inference cost.
Every thumbs up/down is stored by category and model version. collect_feedback.py identifies which categories have approval below 70% and auto-generates more targeted training examples for those areas.
Claude and OpenAI are offline teachers only — used to bootstrap training data, not as the long-term production path. As the Pragma model matures, third-party LLM dependency drops to zero.
# Step 1 — Generate labelled training data (local)
ANTHROPIC_API_KEY=sk-ant-... \
python ml/generate_data.py --n 5
# → ml/data/train.jsonl (~140 examples)
# → ml/data/eval.jsonl
# Step 2 — Fine-tune on Google Colab T4 (free)
pip install transformers peft trl datasets
HF_TOKEN=hf_... python ml/train.py \
--hf_repo yourname/pragma-ethics-v1
# Step 3 — Run the feedback flywheel
DATABASE_URL=postgresql://... \
ANTHROPIC_API_KEY=sk-ant-... \
python ml/collect_feedback.py --threshold 0.70
# Step 4 — Evaluate student vs teacher quality
HF_TOKEN=hf_... python ml/evaluate.py \
--student_repo yourname/pragma-ethics-v1
# Reports: parse rate, risk Jaccard, conf MAE
Three client layers share one FastAPI engine. The Pragma model is the primary reasoning provider.
iOS-style glassmorphism single-page app. Tabs: Evaluate (proxy var guard, HITL override), History, Batch, Audit Log 🔐, Compliance 🏛️ (5-step wizard with AI evidence upload + guided interview, certificate), Dashboard 📊 (score trends, heatmap, deadline countdown), Settings + Billing. Mobile bottom nav bar with iOS Safari fixed-position fix.
React Native (Expo) app for iOS & Android. Screens: Auth (social proof + sign-in), Home (evaluate with guided questions), Results (firewall verdict banner, regulatory refs, counterfactual analysis), History, Chat (compliance-aware chatbot with scenario shortcuts and per-message firewall badges). Shares the same API as the web dashboard.
FastAPI with Pragma model as primary provider. Claude/OpenAI as fallbacks. PostgreSQL on Railway.
Deterministic, synchronous rule evaluation — no LLM call, no latency. Catches most violations before reaching L2.
500+ historically redlined zip codes across 35 US cities, loaded from geo_data.py at startup. Any decision where context.zip_code maps to a redlined area is flagged as a potential Fair Housing Act violation.
35+ proxy fields detected: zip_code, last_name, ip_country, email_domain, neighborhood descriptors, national origin indicators. Compound risk detection when multiple proxies appear together.
Indirect discriminatory phrasing: "that part of town", "near retirement", "family obligations", "mother tongue", "criminal background", and 9 more. Scans both decision text and context free-text fields.
Batch decision sets are analyzed for group-level pass rates. A protected group with a pass rate below 80% of the highest group's rate triggers an EEOC disparate impact flag automatically.
California FEHA, New York HRL, and Illinois HRA add state-level protected classes and stricter thresholds on top of federal (ECOA/EEOC) baseline. Applied based on geographic scope of the AI system.
Rules stored in compliance_rules table. Default action (FAIL/FLAG/ALLOW) and threshold are configurable per org via org_rule_overrides. Changes take effect immediately — no deployment required. Full audit trail of who changed what.
Daily cron sweeps Federal Register API + EEOC newsroom. Claude filters for AI hiring relevance, drafts a rule config (regulation name, citation, violation patterns). Saved as pending, admin notified by email, auto-promoted to live rules after 24 h unless rejected. Zero human effort to stay current.
Self-updating compliance corpus — new regulations discovered daily, rules deployed automatically.
LAW_AGENT_DAYS_BACK)Claude drafts a JSON config — not arbitrary code. Fields: rule_key, regulation, statute_citation, patterns[], fields[], default_severity (FAIL/FLAG), categories. Inserted into compliance_rules table via upsert_compliance_rule_from_agent().
GET /admin/law-agent/findings — all findingsPOST /admin/law-agent/run — trigger manuallyPOST /admin/law-agent/findings/{id}/reject — kill before activationPOST /cron/law-agent — Railway cron endpoint (X-Cron-Secret)backend/law_agent.py — agent: fetch → Claude filter → draft rule → save pending → email
backend/cron_runner.py — standalone script: python -m backend.cron_runner (Railway cron)
backend/database.py — law_agent_findings table + 7 DB functions
backend/main.py — /admin/law-agent/* endpoints + POST /cron/law-agent
Env vars required:
ANTHROPIC_API_KEY — Claude analysis
RESEND_API_KEY — admin email notification
ADMIN_EMAIL — where to send alerts (default: cosmosservicesai@gmail.com)
CRON_SECRET — protects POST /cron/law-agent from unauthenticated calls
LAW_AGENT_ACTIVATION_HOURS — hours before auto-activation (default: 24)
LAW_AGENT_DAYS_BACK — lookback window for sweeps (default: 7)
Request flows through L1 deterministic rules, then L2 LLM analysis. Analytics captured at each step.
All endpoints served from the FastAPI backend on Railway.
| Method | Path | Access | Description |
|---|---|---|---|
| POST | /auth/google |
Public | Verify Google ID token, create session |
| POST | /auth/guest |
Public | Create anonymous guest session |
| POST | /logout |
Bearer | Invalidate session token |
| GET | /me |
Bearer | Return current user name & picture |
| GET | /my-stats |
Bearer | Return anonymous usage stats for session |
| POST | /evaluate-decision |
Bearer | Run compliance evaluation — returns firewall_action, risk_flags, regulatory_refs, confidence_score, audit_log_id, proxy_variables_detected |
| POST | /chat |
Bearer | Compliance-aware chat — firewall evaluates message before response is generated; blocked messages return verdict only |
| POST | /evaluate-batch |
Bearer | CSV upload (up to 100 rows) — returns CSV with firewall verdicts and regulatory refs appended |
| POST | /counterfactual |
Bearer | Bias audit — compares original vs. modified context, diffs flags_added / flags_removed / confidence_delta |
| POST | /feedback |
Bearer | Submit thumbs up/down on an analysis — feeds the retraining flywheel |
| POST | /generate-report |
Bearer | Generate downloadable PDF audit report |
| GET | /questions |
Bearer | Category-specific guided context questions for hiring, finance, healthcare, and more |
| POST | /orgs |
Bearer | Create an organization workspace with auto-generated invite code |
| POST | /orgs/join |
Bearer | Join an organization via invite code |
| POST | /api-keys |
Bearer | Generate a pragma_* API key for programmatic SDK access |
| GET | /api-keys |
Bearer | List all API keys for current user (id, name, prefix, created, last_used) |
| DELETE | /api-keys/{key_id} |
Bearer | Revoke an API key by id |
| GET | /compliance-rules |
Bearer | Get effective compliance rules for a category (merged org overrides over global defaults) |
| PUT | /compliance-rules/{rule_id} |
Bearer | Update a compliance rule (action, threshold, enabled) — changes take effect immediately, no redeploy |
| POST | /analyze-text-bias |
Bearer | Run text scanner on a string — detects indirect discriminatory phrasing (14 pattern categories) |
| POST | /adverse-action-notice |
Bearer | Generate ECOA §1002.9 + FCRA §615(a) compliant adverse action notice from an audit_log_id |
| POST | /disparate-impact |
Bearer | Run EEOC 4/5ths disparate impact analysis across demographic groups from a batch decision set |
| POST | /track |
Bearer | Track a feature event to own DB (complements PostHog) — updates user_profiles and feature_events |
| GET | /admin/analytics |
Admin | Return DAU/WAU/MAU, feature adoption rates, top events, plan distribution, new users (7d) — admin only |
| GET | /cosmos |
Public | Serve Cosmos AI LLC company homepage (frontend/cosmos.html) |
| GET | /audit/log |
Bearer | Last 50 audit entries for current user — verdict, risk flags, proxy vars detected, HITL override status, input hash |
| POST | /audit/override |
Bearer | Record human investigator override of a firewall verdict — requires reason; EU AI Act Art. 14 compliance |
| POST | /ai-systems |
Bearer | Register an AI system for EU AI Act data lineage tracking (system_name, company, risk_tier, training_data_sources…) |
| GET | /ai-systems |
Bearer | List all registered AI systems for the current user |
| GET | /ai-systems/{id}/compliance |
Bearer | EU AI Act 15-article compliance checklist (Arts. 4–6, 9–15, 17, 25, 27, 30, 33) — per-article pass/partial/fail with evidence-based scoring, overall score, verdict |
| POST | /ai-systems/{id}/certificate |
Bearer | Generate PDF compliance readiness certificate — certificate ID, valid 1 year, SELF-ASSESSMENT ONLY watermark, DECLARATION ONLY badges |
| GET | /billing/subscription |
Bearer | Current plan (free / growth / enterprise), evaluations used this month, period end date |
| POST | /billing/create-checkout-session |
Bearer | Start Stripe Checkout for Growth plan ($299/mo) — returns checkout_url |
| POST | /billing/portal |
Bearer | Open Stripe Customer Portal to manage / cancel subscription — returns portal_url |
| POST | /billing/webhook |
Stripe | Stripe webhook receiver — signature-verified; handles checkout.session.completed, subscription created/updated/deleted |
| POST | /proxy-variable-report |
Bearer | Run standalone ECOA proxy variable scan on a context dict; returns proxy_variables_detected, compound_risks, severity summary |
| POST | /evidence/extract |
Bearer | Upload PDF/TXT/MD/CSV + article_key; Claude extracts compliance evidence and returns notes, date, verdict (pass/partial/insufficient), confidence |
| POST | /evidence/interview |
Bearer | Submit Q&A answers for an article; Claude scores them and returns verdict, feedback, and list of gaps |
| GET | /evidence/questions/{article_key} |
Bearer | Return the 4–5 structured interview questions for any of the 9 interviewable articles (Art. 4, 9, 10, 11, 17, 25, 27, 30, 33) |
| GET | /ai-systems/{id}/history |
Bearer | Return ordered compliance snapshots for a system (for trend charts in the Dashboard) |
| GET | /dashboard/summary |
Bearer | Return all registered systems with latest compliance scores + snapshot history for the Dashboard tab |
| GET | /notifications/unsubscribe |
Public | One-click unsubscribe via token query param |
| POST | /notifications/preferences |
Bearer | Update email notification opt-in/out preferences |
| GET | /health-check |
Public | Model availability — model.pragma, model.claude, model.openai |
Separate installable package that wraps OpenAI-compatible clients with one line of code.
from openai import OpenAI
from pragma import Pragma, ComplianceError
client = Pragma(
OpenAI(),
policy_id="hr-compliance-v1",
pragma_api_key="pragma_...",
mode="block", # block | flag | audit
block_threshold=0.8, # tune sensitivity
)
try:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role":"user",
"content":"Reject her — she is 58."}]
)
except ComplianceError as e:
print(e.result.firewall_action) # "block"
print(e.result.risk_flags) # ["bias","discrimination"]
print(e.result.violations[0].regulation)Pragma() / AsyncPragma() — factory functions returning wrapped clientchat.completions.create() — same interface as underlying clientPOST /evaluate-decision on the Pragma backend before forwarding to OpenAIblock — raises ComplianceError with full ComplianceResultflag — passes through, attaches .pragma_result to responseaudit — passes through silently, violations logged server-sideblock_threshold per client instanceTwo paths: Google SSO or Guest. Both return a bearer token used for all subsequent requests.
Google Identity Services JS loaded on page init.
Google popup appears. User selects their account.
Google returns a signed JWT credential to the callback.
Token sent to backend. google-auth SDK verifies with Google's public keys.
Backend returns opaque session token stored in localStorage.
No credentials required. Immediate access.
Backend generates a random anonymous ID and session token.
Same bearer token stored in localStorage. Expires in 24 hours.
Guest users get full evaluate + PDF + feedback features. No name/picture shown.
No personal data stored. Feedback drives model improvement, not user profiling.
-- Request metadata (no PII, no decision text)
CREATE TABLE request_logs (
id SERIAL PRIMARY KEY,
anon_id TEXT, -- SHA-256 of session sub, not reversible
timestamp TEXT, -- ISO 8601 UTC
context_keys TEXT, -- JSON array of field names ["gender","role"]
decision_words INTEGER, -- word count only
provider TEXT, -- pragma | claude | openai | mock
confidence REAL,
risk_count INTEGER,
risk_categories TEXT, -- JSON array ["bias","fairness"]
category TEXT -- hiring | finance | healthcare | ...
);
-- Feedback for model retraining flywheel
CREATE TABLE analysis_feedback (
id SERIAL PRIMARY KEY,
anon_id TEXT,
timestamp TEXT,
rating INTEGER, -- 1 = thumbs up, -1 = thumbs down
category TEXT,
provider TEXT,
model_version TEXT,
confidence REAL,
risk_categories TEXT -- JSON array
);
-- Immutable audit trail (fintech compliance + EU AI Act Art. 12)
CREATE TABLE audit_log (
id SERIAL PRIMARY KEY,
anon_id TEXT,
timestamp TEXT,
input_hash TEXT, -- sha256(decision+context) — no raw PII
decision_words INTEGER,
category TEXT,
firewall_action TEXT, -- block | override_required | allow
confidence REAL,
risk_flags TEXT, -- JSON array
proxy_vars TEXT, -- JSON array of detected ECOA proxy fields
regulatory_refs TEXT, -- JSON array of triggered regulation names
provider TEXT,
model_version TEXT,
hitl_override INTEGER, -- 1 if a human investigator overrode the verdict
hitl_reason TEXT, -- investigator's written reason
hitl_anon_id TEXT -- who overrode it (anon)
);
-- EU AI Act data lineage — registered AI systems (15-article compliance)
CREATE TABLE ai_systems (
id SERIAL PRIMARY KEY,
anon_id TEXT,
system_name TEXT,
company_name TEXT,
risk_tier TEXT, -- minimal | limited | high | unacceptable
use_case TEXT,
model_version TEXT,
training_data_sources TEXT, -- JSON array (Art. 10 data governance)
intended_purpose TEXT,
geographic_scope TEXT,
-- Declarative fields + evidence for Arts. 4, 6, 15, 17, 25, 27, 30, 33
-- Evidence scoring: declaration alone → "partial"; declaration + notes + date → "pass"
art4_literacy_training BOOLEAN,
art4_literacy_training_evidence_notes TEXT,
art4_literacy_training_evidence_date TEXT,
art6_annex_category TEXT, -- Annex III high-risk category
art15_accuracy_metric TEXT, -- e.g. "AUROC 0.87, Precision 0.83"
art15_robustness_tested BOOLEAN,
art17_qms_documented BOOLEAN,
art17_qms_documented_evidence_notes TEXT,
art17_qms_documented_evidence_date TEXT,
art25_instructions_provided BOOLEAN,
art25_instructions_provided_evidence_notes TEXT,
art25_instructions_provided_evidence_date TEXT,
art25_monitoring_active BOOLEAN,
art25_monitoring_active_evidence_notes TEXT,
art25_monitoring_active_evidence_date TEXT,
art27_fria_conducted BOOLEAN,
art27_fria_conducted_evidence_notes TEXT,
art27_fria_conducted_evidence_date TEXT,
art30_eu_db_registered BOOLEAN,
art30_registration_number TEXT, -- EU AI database registration number
art30_eu_db_registered_evidence_notes TEXT,
art30_eu_db_registered_evidence_date TEXT,
art33_conformity_type TEXT, -- self-assessment | third-party
art33_conformity_type_evidence_notes TEXT,
art33_conformity_type_evidence_date TEXT,
created_at TEXT,
updated_at TEXT
);
-- EU AI Act compliance certificates
CREATE TABLE compliance_certificates (
id SERIAL PRIMARY KEY,
certificate_id TEXT UNIQUE, -- e.g. PRAGMA-A3F9C2
ai_system_id INTEGER,
anon_id TEXT,
issued_at TEXT,
valid_until TEXT, -- 1 year from issue
overall_score REAL,
articles_status TEXT, -- JSON {art_4: "pass", ..., art_33: "partial"}
total_evaluations INTEGER,
hitl_overrides INTEGER,
proxy_vars_caught INTEGER
-- PDF includes SELF-ASSESSMENT ONLY watermark and DECLARATION ONLY badges
);
-- Stripe billing subscriptions
CREATE TABLE subscriptions (
id SERIAL PRIMARY KEY,
anon_id TEXT UNIQUE NOT NULL,
plan TEXT NOT NULL DEFAULT 'free', -- free | growth | enterprise
status TEXT NOT NULL DEFAULT 'active',
stripe_customer_id TEXT,
stripe_subscription_id TEXT,
current_period_end TEXT, -- ISO 8601 UTC
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
-- User profiles (email capture + notification preferences)
CREATE TABLE users (
id SERIAL PRIMARY KEY,
google_sub TEXT UNIQUE NOT NULL,
email TEXT,
name TEXT,
unsubscribe_token TEXT UNIQUE,
notify_welcome BOOLEAN DEFAULT TRUE,
notify_gap BOOLEAN DEFAULT TRUE,
notify_countdown BOOLEAN DEFAULT TRUE,
created_at TEXT,
updated_at TEXT
);
-- Notification deduplication log
CREATE TABLE notification_log (
id SERIAL PRIMARY KEY,
google_sub TEXT NOT NULL,
notification_type TEXT NOT NULL, -- welcome | gap_reminder | countdown
system_id INTEGER, -- NULL for user-level notifications
sent_at TEXT NOT NULL
);
-- Daily compliance score snapshots per system (for Dashboard trend charts)
CREATE TABLE compliance_snapshots (
id SERIAL PRIMARY KEY,
google_sub TEXT NOT NULL,
system_id INTEGER NOT NULL,
score REAL NOT NULL,
verdict TEXT NOT NULL,
passes INTEGER NOT NULL,
partials INTEGER NOT NULL,
fails INTEGER NOT NULL,
articles_json TEXT NOT NULL, -- {art_key: status} snapshot
taken_at TEXT NOT NULL -- ISO UTC timestamp, one per day (deduped)
);
-- Dynamic compliance rule engine (DB-backed, live config, per-org overrides)
CREATE TABLE compliance_rules (
id SERIAL PRIMARY KEY,
rule_id TEXT UNIQUE NOT NULL, -- e.g. "holc_redlining", "ecoa_proxy_zip"
category TEXT, -- lending | hiring | healthcare | all
description TEXT,
default_action TEXT NOT NULL, -- FAIL | FLAG | ALLOW
threshold REAL, -- configurable severity threshold
enabled BOOLEAN DEFAULT TRUE,
created_at TEXT,
updated_at TEXT
);
CREATE TABLE org_rule_overrides (
id SERIAL PRIMARY KEY,
org_id INTEGER NOT NULL,
rule_id TEXT NOT NULL,
action TEXT, -- overrides default_action for this org
threshold REAL,
enabled BOOLEAN,
updated_by TEXT, -- anon_id of admin who made the change
updated_at TEXT
);
-- Customer analytics — feature adoption and user behavior
CREATE TABLE user_profiles (
id SERIAL PRIMARY KEY,
anon_id TEXT UNIQUE NOT NULL,
plan_tier TEXT DEFAULT 'free',
login_method TEXT, -- google | guest
total_evaluations INTEGER DEFAULT 0,
session_count INTEGER DEFAULT 0,
has_api_key INTEGER DEFAULT 0,
features_used TEXT, -- JSON array of feature names
last_active TEXT,
created_at TEXT
);
CREATE TABLE feature_events (
id SERIAL PRIMARY KEY,
anon_id TEXT NOT NULL,
event_name TEXT NOT NULL, -- decision_evaluated, report_downloaded, …
properties TEXT, -- JSON dict of event properties
created_at TEXT NOT NULL
);
-- Law Research Agent findings (self-updating rule corpus)
CREATE TABLE law_agent_findings (
id SERIAL PRIMARY KEY,
source TEXT NOT NULL, -- federal_register | eeoc
source_url TEXT NOT NULL,
url_hash TEXT NOT NULL UNIQUE, -- sha256[:16] for dedup
title TEXT NOT NULL,
published_date TEXT NOT NULL,
raw_summary TEXT NOT NULL,
relevant INTEGER DEFAULT 0, -- 1 = Claude marked relevant
regulation_name TEXT, -- Claude-extracted regulation name
statute_citation TEXT, -- e.g. 42 U.S.C. § 2000e-2
key_requirement TEXT, -- one-sentence summary
jurisdiction TEXT, -- federal | california | new_york | …
rule_key TEXT, -- snake_case rule key
rule_config_json TEXT, -- full rule config as JSON
status TEXT DEFAULT 'pending', -- pending | active | rejected | not_relevant
activate_at TEXT, -- ISO — auto-activates here
activated_at TEXT,
rejected_at TEXT,
reject_reason TEXT,
discovered_at TEXT NOT NULL
);
L1 deterministic rules run synchronously in microseconds. L2 LLM analysis runs only when L1 passes — or to enrich context.
Deterministic rule evaluation: HOLC historically redlined zip codes (500+ across 35 US cities), ECOA proxy variables (35+ fields), indirect phrasing (14 categories), compound risk detection, state law overlays (CA/NY/IL). DB-backed rules with per-org overrides — no redeploy needed.
claude-sonnet-4-6 primary. Streams regulatory reasoning across EU AI Act, ECOA, EEOC, FCRA, CFPB. Also powers evidence analyzer (document upload validation) and guided interview scoring.
Four services, each on the right platform for its workload.
Connect GitHub repo. Railway auto-detects Python via requirements.txt.
railway.app → New Project
→ Deploy from GitHub
→ Add PostgreSQL plugin
→ Set env vars → Deploy
Required env vars: ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_CLIENT_ID, ALLOWED_ORIGINS, STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET, STRIPE_GROWTH_PRICE_ID, RESEND_API_KEY, ADMIN_EMAIL, SENTRY_DSN, CRON_SECRET, GEMINI_API_KEY
Static HTML. Set root directory to frontend/ in Vercel dashboard.
vercel.com → New Project
→ Import pragma
→ Root: frontend/
→ Deploy
Update PROD_URL in frontend/index.html to your Railway URL.
Fine-tune on Colab (free T4 GPU), push adapter to HuggingFace Hub, activate via env var.
python ml/generate_data.py --n 5
python ml/train.py \
--hf_repo user/pragma-ethics-v1
→ Set CUSTOM_MODEL_REPO in Railway
Build and submit to App Store / Play Store via Expo EAS.
cd mobile
npm install -g eas-cli
eas login
eas build --platform ios
eas submit --platform ios
Update PROD_URL in mobile/src/config.ts to your Railway URL.
Every library and service across all layers.
Controls applied across the stack as of Q2 2026.
esc() helper using document.createTextNode() applied to every dynamic HTML injection site in the frontend — system name, company, article evidence, risk flags, counterfactual keys, wizard review rows. No raw innerHTML from server data.
Explicit origin allowlist via ALLOWED_ORIGINS env var replaces the previous allow_origins=["*"]. Wildcard origins allow any site to make authenticated API calls from a logged-in user's browser — the allowlist prevents this.
Decision text capped at 4,000 characters. Context payload capped at 8,000 characters. Both enforced at the FastAPI layer with HTTP 400 before the LLM is called — prevents prompt injection via oversized inputs.
The /evaluate-batch endpoint now checks the monthly plan limit before processing any rows. Previously, free users could bypass the 100 eval/month limit by uploading a 100-row CSV in a single batch call.
Column names in the ALTER TABLE migration path are validated against re.compile(r'^[a-z_][a-z0-9_]*$') before being interpolated into SQL. Unknown column names are logged and skipped rather than executed.
Webhook payloads are verified with stripe.WebhookSignature.verify_header() using the signing secret before any database writes occur. Unsigned or tampered events return HTTP 400 immediately.