Design decisions
Architecture decision records.
| ADR | Decision | Rationale / consequences |
|---|---|---|
| 001 | Separate .NET data/control planes | Gateway has no admin routes; Admin API never forwards inference bodies. Separate listeners/roles limit blast radius, but both use shared DB/Redis. |
| 002 | Vue + OIDC BFF, not browser bearer tokens | Secure cookie, code+PKCE, CSRF; IdP owns MFA. SameSite=Lax supports OIDC redirects. SaveTokens=false limits leakage; some IdPs ask logout confirmation. |
| 003 | Metadata-only accounting and masked audit | No prompt/response persistence, even for troubleshooting. Downstream legal case records stay in the approved case system; identifiers can still be personal/public-record data. |
| 004 | Virtual-key HMAC and certificate-protected DP credentials | Plain key shown once, pepper outside DB. DP key ring shared in DB and encrypted in production. Pepper/certificate loss requires recovery or key replacement. |
| 005 | Redis atomic reserve/reconcile | Shared rate/budget/circuit state and invalidation across services; restrict namespace/commands. In-memory mode is only for local/test environments. |
| 006 | JSON passthrough, data-driven provider/model profiles | New model families do not require a release. Rewrites/translation apply only supported fields; provider contract tests are still needed. |
| 007 | Rotation retains budget lineage | Immediate revoke default, optional 24h grace; both predecessors/replacements share spend so rotation cannot evade budgets. |
| 008 | File-based Aspire, Compose generated + mandatory override | Aspire is source of resource graph; approved override supplies deployment-specific TLS/ACL/roles/network/container controls. Never deploy base alone or containerize AppHost. |
| 009 | Production external IdP/provider; local test services only in run mode | Dev realm/FakeLlm/Vite are not production workloads. Production images contain static Vue assets served by BFF. |
| 010 | Accessibility design goal with explicit evidence boundary | Originally WCAG 2.2 AAA with four themes; since 2026-10-09 the target is WCAG 2.2 AA (24 px targets) with three themes (light, dark, Lumen). Contrast/component/browser automation; manual screen-reader/usability/IdP review needed, no automated certification claim. |
| 011 | Chiseled-extra and strict transport | Nonroot, no shell, ICU/time-zone data for Swedish budgets. HTTPS probes use a dedicated process mode; municipal PKI/secret delivery required. |
| 012 | Testcontainers integration fallback | File-based AppHosts are not supported by DistributedApplicationTestingBuilder. Real Postgres/Redis/Kestrel tests plus real-stack Playwright cover runtime behavior. |
| 013 | Routing rules use an in-house CEL-subset condition language and never widen access | Modelled on Bifrost. Conditions are written in a CEL subset implemented in Domain instead of a community CEL package: no new dependency or licence cost, no loops or side effects, linear-time regex (non-backtracking, literal patterns only), bounded length/depth, and positioned validation errors plus explain traces. Admin-authored expressions run on the request path, so the cost is bounded by expression size and budget/token lookups happen only when a rule uses them. Rules run after the PII policy and before candidate selection; provider allow-lists, residency, the PII on-prem restriction, endpoint kind and circuit state apply to whatever a rule chose, and rules cannot see credentials or prompt content. There is no silent return to the requested model: a matched rule whose models cannot serve the request fails with no_eligible_provider. |
| 014 | Rule ownership follows the organisation, but the user decides what happens when it changes | Rules are scoped to a key, team, department or globally (checked in that order, then by priority). Rules often belong to a service rather than the org chart, so deleting a team or department that has rules asks whether to delete them or deactivate them (default) until a new owner is assigned; deactivated rules are orphaned, cannot be enabled, and are reassigned explicitly. There is deliberately no foreign key (an orphaned rule never matches). A key-scoped rule follows the key through rotation, like budgets. Routes and models used by a rule cannot be deleted or renamed. Only global rules are exported/imported because ids differ between environments. |
| 015 | Data for other teams goes through interfaces, never the database; security events are pushed over OpenTelemetry | The gateway is open source and adopters run different databases, warehouses and SIEMs, and isolate services behind an integration layer. Security events are log records in the Ume.LlmGateway.Security category with stable event and attribute names; an adopter's OTel Collector routes them to any SIEM, so the gateway has no SIEM-specific code. Refused keys are counted per (reason, endpoint, key, client network) and flushed in bounded batches, because unauthenticated callers must not be able to grow the database; the presented key is never kept. BI and management get a separate read-only Data API (planned) with OAuth scopes per consumer class and incremental cursors. Retention, minimum group size and source-address detail are adopter settings with privacy-preserving defaults. See data-access.md. |
| 016 | Old data is removed by one database function, and usage is summarised by month first | Retention is set in the admin UI (usage data and audit log separately, 90–3 650 days, default 365; audit may be kept for ever) and applied nightly by the admin API, which calls ume_retention_step. The function runs with the table owner's rights, so the admin role never gets DELETE on usage, refused-key, alert or audit rows: the audit log stays append-only for everything younger than the retention, and the 90-day minimum holds even against a direct database call. Usage goes in whole Europe/Stockholm months, after its monthly totals are written in the same transaction that moves the boundary (UsageRetainedFrom); readers take totals before the boundary and records after it, so budgets (whose long periods start on a month) and monthly reports stay exact. |
| 017 | Updates without downtime are opt-in, by instance replacement rather than in-place reload | Most installations run one instance, so compose.prod.yaml is unchanged; compose.rolling.yaml adds second gateway and admin API slots on their own host ports for the adopter's existing proxy, because adopters already run nginx, Nginx Proxy Manager or F5 and a bundled load balancer would duplicate it. Every service fails readiness on the stop signal and, for Shutdown__DrainSeconds, answers with Connection: close before closing its listener: a proxy retries a refused connection for any method but not a POST sent on a pooled connection the server closed, which measured as lost model calls without the drain. Connections the kernel accepted but the stopping instance never read are still reset, so the nginx example retries POSTs that failed before any response (non_idempotent), limited to 10 s so a long request is never repeated. Kubernetes gets Kustomize manifests rather than a Helm chart: plain YAML that adopters patch in their own overlays or GitOps repositories, no template language to maintain, and kubectl already includes it. Only backward-compatible migrations allow a rolling update, so every release states Rolling update: yes/no. |
Accepted for POC on 2026-10-07 (013-014 on 2026-10-09, 015 on 2026-10-10, 016 on 2026-10-12, 017 on 2026-10-16). Reassess durable accounting, HA, egress isolation and per-use-case legal controls before production approval.