AI Governance Platform — Enterprise Control Layer for Safe LLM Adoption
A personally designed and fully implemented AI governance platform that addresses the central tension of enterprise AI adoption: how do you let teams use LLMs freely while enforcing data protection, regulatory compliance, cost control, and a complete audit trail — without the governance becoming the thing people route around?
This is a personal reference architecture — designed, implemented, and iteratively hardened by me as a solo architect and developer. Unlike the anonymised case studies above, this project is fully open: the source code, architecture decisions, and their rationale are all available. The project addresses a problem that every organisation adopting LLMs faces but few solve architecturally: governance. Not just routing requests to providers, but enforcing who can use what, preventing sensitive data from leaving the organisation, controlling costs, maintaining an audit trail suitable as compliance evidence, and giving administrators self-service tools to manage all of this — without requiring a deployment for every policy change. The challenge is not technical integration — it is building a governance layer that is strict enough for regulators, flexible enough for developers, and manageable enough for non-technical administrators.
Enterprise AI adoption without a governance layer creates compounding risks across security, compliance, and cost. Provider credentials are distributed across applications, making rotation and revocation nearly impossible. Sensitive data — personal identifiers, financial records, internal secrets — can flow to external providers without detection. There is no unified view of cost, usage, or access patterns. EU compliance (GDPR accountability under Art. 5(2), security of processing under Art. 32, EU AI Act record-keeping) requires demonstrable data protection controls that no direct integration can provide. Security policies are hardcoded, requiring developer intervention for every change. And there is no approval workflow — any user with credentials can access any model without oversight.
- EU/CZ regulatory environment — GDPR accountability (Art. 5(2)) and security of processing (Art. 32), plus EU AI Act deployer obligations, require demonstrable data governance and control over AI interactions
- Four fundamentally different provider APIs (OpenAI, Anthropic, Azure OpenAI, Ollama) that must appear uniform to consumers
- Two distinct access patterns: API keys for machine-to-machine integration, SSO for human operators — both requiring the same governance controls
- Data sovereignty is tiered: sending a Czech birth number to a cloud provider outside the EU creates third-country transfer risk requiring adequate safeguards; processing on a local model inside the perimeter eliminates that transfer risk. Classification and enforcement are therefore separate concerns
- Governance policies must be manageable by non-technical administrators without code deployments
- Solo architect — every feature must justify its complexity; what NOT to build is as important as what to build
Sole architect and implementer. End-to-end responsibility: from problem analysis and system design through full-stack implementation (~9,000 LOC backend, ~6,500 LOC frontend, 400 tests) to iterative hardening across multiple review cycles run through a multi-agent audit pipeline I built.
The platform enforces a single governed boundary — one chokepoint through which all LLM traffic flows. The boundary is enforceable, not just architectural: provider credentials are held exclusively by the platform (key custody), so applications cannot call providers directly. Complete enforcement also requires restricting provider console access so individuals cannot generate their own keys. In hardened deployments, network egress rules provide an additional layer. Every concern (identity, authorisation, data protection, cost control, audit) is enforced consistently regardless of the consumer or provider. But governance is not just about enforcement — it is about manageability. The platform provides a self-service admin control panel where administrators can configure DLP rules, rate limits, security policies, and compliance settings through a UI — without code changes or deployments. An RBAC model with role-based access (viewer, editor, admin, super_admin) controls who can change what. Sensitive DLP policy changes — disabling system rules or weakening blocking policies — go through a mandatory approval workflow with self-approval prevention. The request pipeline enforces an invariant sequence — authentication, authorisation, rate limiting, prompt security scanning, DLP, provider routing (with automatic failover), output risk evaluation, and audit logging — in that order, for every request. The key design principle is default-safe: every security control defaults to the restrictive option and requires explicit operator action to relax.
This platform is not an alternative to Microsoft Copilot — it is its complement. Copilot governs AI within the M365 ecosystem; this platform governs what Copilot structurally cannot cover: processing in-region or outside cloud providers, self-hosted and local models, custom fine-tunes, the integration API/app path, and AI outside the M365 ecosystem — all under a single governance boundary with data sovereignty.
Architecture of the governed LLM gateway. The entire platform runs inside the customer's own infrastructure (self-hosted, single-tenant) — the dashed boundary marks that perimeter. Internal applications reach LLMs only through the gateway: provider credentials are held exclusively by the platform (key custody), so applications cannot call providers directly. In hardened deployments, network egress rules can further restrict outbound access to the gateway only. Inside the boundary the gateway enforces four concern areas — consumer access (authentication, access policy, rate limiting & cost control), security controls (DLP scanning, prompt-risk evaluation, output validation), provider control (abstraction, model routing with failover), and audit & governance (audit trail, policy enforcement). Local models stay inside the perimeter; external providers sit outside it and are reached only through the gateway. Operators administer policy through the admin control panel under RBAC, with a mandatory approval workflow for sensitive DLP changes.
- Governance as a product, not infrastructure — the platform is not a proxy with config files. It has a full admin UI for policy management, RBAC for access control, and approval workflows for sensitive DLP changes. This makes governance accessible to compliance officers and security teams, not just developers
- Key custody as the enforcement mechanism — provider API keys are held exclusively by the platform; consuming applications never see them. This enforces the governance boundary for all application traffic. For complete enforcement, provider console access must also be restricted so that individuals cannot generate their own keys — this is an organisational control, not a platform one. In hardened deployments, network egress rules provide an additional layer. This is a question any governance buyer will ask: "what prevents a developer from calling the API directly?" Key custody enforces it for applications; provider console access control enforces it for people
- Self-hosted, single-tenant deployment — the platform runs entirely within the customer's infrastructure: no shared SaaS control plane, no governance data leaving the perimeter, no vendor lock-in on the governance layer. Only the LLM request itself leaves the perimeter — routed exclusively through the gateway. This is a structural differentiator against SaaS-only governance competitors and a hard requirement for regulated industries and critical infrastructure
- Tiered data sovereignty — the same DLP engine runs on every request, but enforcement differs by provider type. Cloud providers get strict blocking (PII, financial data, national IDs); local models get relaxed policy (warn, not block). The architectural insight: classification and enforcement are separate concerns. DLP rules are configurable through the admin UI with regex validation and real-time preview
- RBAC with approval workflow — four permission tiers with graduated access. Weakening a security control — DLP blocking policy, output validator mode, or rate limit — requires admin approval; self-approval is prevented. Every change is audit-logged with before/after state. This prevents both accidental misconfiguration and unauthorised policy relaxation
- Self-service admin control panel — DLP patterns, rate limits, security scanner policies, and compliance settings are all manageable through a UI. Administrators do not need to edit environment variables, config files, or redeploy. This is the difference between a governance tool and a governance platform
- Default-safe configuration — output validation blocks high-risk responses on both streaming and non-streaming paths. On streaming responses, DLP scans accumulated content periodically and terminates the stream mid-flight if sensitive data is detected. Rate limiting fails closed on Redis outage (blocks requests rather than allowing unlimited spend). JWT signature and issuer verification is active when JWKS is configured. Each of these is a deliberate availability trade-off: brief disruption is better than silent policy bypass
- Metadata-only audit trail — every request is logged with full context (who, what model, when, latency, tokens, cost) but zero prompt or response content in the audit trail. Playground conversation content is stored separately with defined retention and full DSAR/erasure coverage. This supports governance requirements while clearly scoping data liability. Policy changes are separately audit-logged with full diff tracking
- Scope & limits — this is a reference architecture and deployer tooling, not a certified compliance product. Compliance artifacts (ROPA, DPIA, FRIA templates, breach notifications) are scaffolding a DPO must validate. Per-output human review for high-risk use cases, legal-basis determination, DPA/SCC handling, and the organisation's ISMS remain the deployer's responsibility. Provider-side processing (e.g. Copilot within M365) is out of scope by design
- Complete governance boundary for 4 LLM providers — every request authenticated, authorised, scanned, rate-limited, and audit-logged. Enforcement backed by key custody (applications never hold provider credentials)
- Self-service admin UI for DLP rules, rate limits, security policies, and compliance configuration — zero-deployment policy management
- EU/CZ-specific integrated DLP with 19 patterns (Czech birth numbers, IČO, IBAN, bank accounts) with NFKC Unicode normalisation to prevent evasion
- Enterprise RBAC with 4-tier role model and mandatory approval workflow for weakening a sensitive security control (DLP policies, output validator, rate limits)
- Self-hosted single-tenant deployment — all data stays within the customer's infrastructure, no external dependencies on the governance layer
- Iteratively hardened through multiple rounds of adversarial review using a custom multi-agent audit pipeline — each round identifying issues, each followed by targeted remediation with verification
Technologies & Standards