34 KiB
34 KiB
Software Architecture Specification — VKIST MSK Platform
Scope: Sprint 1-2 (FR-25 Synovitis Grading + Multi-Modal NLP Integration)
Parent: SOLUTION_ARCHITECTURE_SPEC.md, SPRINT_1_2_ARCHITECTURE_SPEC.md
Workflow: Understand → Model (C4) → Specify → Decompose → Plan
1. Problem Statement
Build a reproducible, air-gapped-first musculoskeletal ultrasound analysis platform that performs automated synovitis grading (FR-25) with Vietnamese-language NLP explanations, auditable RAG citations, and HITL finalization — deployable on a single-hospital K3s cluster under ≤10 Mbps LAN constraints, with ≤150 MB idle app bundle and ≤1.5 s inference latency.
2. Requirements
Functional (Sprint 1-2)
- FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3
- Grad-CAM overlay on primary viewport (zero extra clicks)
- Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing) across 3-tier LLM system (Edge Gemma → Gemini → MedGemma)
- BERT drift monitor against baseline MOH corpus
- RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks (mandatory for cloud tiers, lightweight for edge)
- Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware)
- ladybugDB ontology traversal for anatomical entity disambiguation
- 3-model LLM consult system: Edge Gemma (browser WebLLM local), Gemini (GCP Vertex AI, orchestration/translation), MedGemma (Modal, clinical deep-reasoning) with MOH guideline citations and cost guarding (<20% MedGemma usage)
- Circular 46/2018 PDF report generation
- Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19)
Non-Functional (Critical)
- NFR-4: 150 MB idle app bundle
- NFR-5: ≤1.5 s inference (on-prem Triton); cloud tiers targeted at <30s median
- NFR-7: ≤200 ms TTFT token streaming; progressive streaming with 25s timeout for cloud responses
- NFR-8: Fault-tolerant across Wi-Fi drops (state preserved)
- NFR-10: Automated Generative Safety Guardrails: <90% verification prohibited, 100% of LLM-generated patient text explanations must pass verification. 3-tier guardrail: prompt rules + BERT edge detection (Tier 1) → Vertex AI safety filters (Tier 2) → RAG-Referee mandatory validation (Tier 3)
- NFR-11: Onboarding ≤ 45 minutes
- NFR-12: Zero-Friction Explainability Integration
- NFR-13: Spatial Layer-Activation Mapping
- NFR-14: No client-side GPU/neural accelerator required
- NFR-15: Circular 46 EMR compliance
- NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit for cloud LLM tiers (Gemini + MedGemma)
- NFR-16a: Emergency Cloud Fallback with Redaction (PoC-SCOPE). Cloud LLM API (GCP Vertex AI Gemini + Modal MedGemma) invoked ONLY when browser-side WebLLM (Edge Gemma) is unavailable or orchestrates cloud task. FastAPI Redaction Middleware MUST strip all Decree 13 PII fields before egress. GCP CSEK + 30-day lifecycle + 1-year auto-delete. Audit-log commit + consent before egress. MedGemma usage <20% target with alerting. NFR-16a retired upon PoC sign-off.
- NFR-17: Immutable audit log
- NFR-18: 100% LLM text cites MOH protocol via RAG. Mandatory RAG pre-processing for all 3 tiers (not optional tool-calling). RAG-Referee validates MedGemma output.
- NFR-19: HITL digital signature before FINALIZED/ARCHIVED
3. Constraints & System Context
- On-prem: K3s on Dell PowerEdge (single hospital, ≤10 Mbps LAN)
- Model: GemmaE2B-Q4 (~1.3 GB) distributed via intranet CDN → GCP CDN fallback → WASM local (WebLLM) or cloud MedGemma on Vertex AI (NFR-16a) → MOH templates
- Data: Postgres + pgvector, Redis (5 constrained data types), MinIO, IndexedDB client prefs
- Auth: Keycloak + RBAC inside K3s; GitLab/Jira on cloud VM (NFR-16a exception)
- CI/CD: Jenkins inside K3s → cloud GitLab via SSH
- Failover: NGINX + Keepalived VIP (≤2s switch)
4. C4 Models
4.1 Context Diagram (Tier 1)
See master: SOLUTION_ARCHITECTURE_SPEC.md §8.1
Actors: Radiologist (UP5), Senior Expert (UP1), Support (UP4), Admin
External: PACS, EMR/HIS, Triton, ladybugDB, pgvector, GemmaE2B/MedGemma, EmbeddingGemma
4.2 Container Diagram (Tier 2)
@startuml "VKIST_MSK_Software_Architecture_Containers"
!include <C4/C4_Container>
title C4 Container Diagram - VKIST MSK Platform Software Architecture
Person(radiologist, "Diagnostic Radiologist (UP5)", "Primary user: loads DICOM, reviews grading, finalizes reports.")
Person(admin, "System Administrator", "K3s ops, model updates, observability, failover.")
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Clinical protocol validation, threshold approval.")
Person_Ext(support_staff, "Support Staff (UP4)", "Patient registration, case queue.")
System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
Container(pwa, "React PWA Frontend", "React 18, TypeScript, Zustand, LiteRT, MediaPipe, Dexie.js", "View-angle validation via WASM; DICOM preview; Grad-CAM overlay; Decree 13 scrubber; IndexedDB for encrypted sessions and user prefs.")
Container(nginx, "Active-Passive Gateway", "NGINX + Keepalived", "SSL termination, VIP load balancing, instant failover (≤2s).")
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
System_Boundary(edge_servers, "Application Server Cluster") {
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.")
Container(cloud_llm_gateway, "Cloud LLM Gateway", "FastAPI (Python 3.12, Uvicorn)", "Routes orchestration/translation to Gemini (GCP) and clinical deep-reasoning to MedGemma (Modal); NFR-16a redaction, consent, audit enforcement; consult_mode state machine extension; cost guarding for MedGemma usage.")
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, RAG-Referee validation for MedGemma output, mandatory RAG pre-processing for all tiers.")
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.")
Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.")
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.")
Container(obs_stack, "Observability Stack", "Prometheus + Grafana", "Metrics scrape, dashboards, alerting.")
}
}
System_Boundary(db_vm, "Database VM") {
ContainerDb(postgres, "PostgreSQL + pgvector", "PostgreSQL 16 + pgvector", "EMR records, spatial markers, MOH guideline HNSW index, session embeddings, audit ledger.")
ContainerDb(redis, "Redis Cache", "Redis 7 (AOF + RDB)", "JWT sessions, MOH guideline chunks, DICOM metadata, consult_mode state, rate-limit counters.")
ContainerDb(minio, "MinIO Object Store", "MinIO S3-compatible", "DICOM payloads, segmentation overlays, Grad-CAM heatmaps, report PDFs, model weight staging.")
}
}
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + EmbeddingGemma RAG embeddings.")
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.")
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.")
System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.")
System_Ext(vertex_ai, "GCP Vertex AI (Gemini)", "Gemini via REST", "PoC-only NFR-16a inference tier: orchestration, translation, UI planning; redacted payloads only.")
System_Ext(modal_medgemma, "Modal MedGemma", "FastAPI + Transformers (T4 GPU)", "PoC-only NFR-16a clinical deep-reasoning endpoint; redacted payloads with RAG-Referee validation; keep-warm instances for latency budget.")
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443")
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH")
Rel(senior_expert, pwa, "Validates protocols, approves thresholds", "HTTPS 443")
Rel(support_staff, pwa, "Registration, case queue", "HTTPS 443")
Rel(pwa, nginx, "API requests + pre-validated DICOM", "HTTPS 443")
Rel(nginx, api_gw, "Routes upstream", "HTTP 8000")
Rel(api_gw, edge_inference, "Forwards /api/*", "HTTP 8000")
Rel(api_gw, rag_svc, "Forwards /rag/*", "HTTP 8001")
Rel(api_gw, auth_svc, "Validates OIDC tokens", "HTTP 8080")
Rel(edge_inference, triton, "3-step inference + embeddings", "gRPC 8001")
Rel(edge_inference, postgres, "Queries guidelines, writes cases/embeddings/audit", "SQL 5432")
Rel(edge_inference, redis, "Session, rate-limit, consult_mode state", "TCP 6379")
Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
Rel(edge_inference, auth_svc, "Token introspection", "OIDC")
Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090")
Rel(edge_inference, cloud_llm_gateway, "Routes cloud consult", "HTTP 8000")
Rel(cloud_llm_gateway, vertex_ai, "Gemini proxy (NFR-16a redacted)", "HTTPS 443 / REST")
Rel(cloud_llm_gateway, modal_medgemma, "MedGemma proxy (NFR-16a redacted)", "HTTPS 443 / REST")
Rel(cloud_llm_gateway, redis, "Consult mode, consent, rate-limit", "TCP 6379")
Rel(cloud_llm_gateway, postgres, "Audit log append", "SQL 5432")
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API")
Rel(rag_svc, edge_inference, "RAG + RAG-Referee results", "HTTP")
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
Rel(audit_svc, redis, "Outbox lock, EMR retry counters", "TCP 6379")
Rel(edge_inference, emr, "Report push (via audit-svc wrapper)", "HL7/FHIR")
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)")
Rel(postgres, minio, "Backup checkpoint", "S3 API")
@enduml
Container communication summary:
| Source | Target | Protocol | Purpose |
|---|---|---|---|
| PWA | NGINX | HTTPS 443 | All API requests |
| NGINX | Envoy | HTTP 8000 | Route upstream |
| Envoy | Edge Inference | HTTP 8000 | /api/* |
| Envoy | Cloud LLM Gateway | HTTP 8000 | /api/cloud-orchestrate, /api/cloud-consult |
| Envoy | RAG Service | HTTP 8001 | /rag/* |
| Envoy | Keycloak | HTTP 8080 | OIDC validation |
| Edge Inference | Cloud LLM Gateway | HTTP 8000 | Route cloud consult |
| Cloud LLM Gateway | Gemini (Vertex AI) | HTTPS 443 / REST | Orchestration, translation, UI planning (NFR-16a) |
| Cloud LLM Gateway | MedGemma (Modal) | HTTPS 443 / REST | Clinical deep-reasoning, report finalization (NFR-16a) |
| Cloud LLM Gateway | Redis | TCP 6379 | Consult mode, consent, rate-limit |
| Cloud LLM Gateway | Postgres | TCP 5432 | Audit log append |
| Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings |
| Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW |
| Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode |
| Edge Inference | MinIO | S3 API | DICOM, overlays, reports |
| Edge Inference | Keycloak | OIDC | Token validation |
| RAG Service | Postgres | TCP 5432 | pgvector HNSW |
| RAG Service | Redis | TCP 6379 | Guideline cache, pub/sub |
| Audit Service | Postgres | TCP 5432 | Immutable audit ledger |
| Audit Service | EMR | HL7/FHIR | Finalized report push |
| K3s | GCP CDN | HTTPS 443 | Signed-URL model weight emergency fetch |
4.3 Component Diagrams (Tier 3)
Edge Inference Service (edge-inference-svc)
@startuml
!include <C4/C4_Component>
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/cloud-orchestrate redirect")
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
Component(router, "Inference Router", "consult_mode state", "Tier selection: Edge Gemma (tier_1) → Gemini (tier_2, orchestration) → MedGemma (tier_3, clinical) → Templates (tier_4)")
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR + Cloud LLM calls; fail-open to templates")
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; mandatory pre-generation for all tiers; enforce NFR-18 citation")
Component(referee, "RAG-Referee", "BERT classifier", "Reject MedGemma text if citation confidence < threshold; mandatory for tier_3")
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override; cloud_llm_escalation events")
}
Rel(api, stream, "delegates", "SSE")
Rel(api, preproc, "validates image", "sync")
Rel(api, router, "selects tier", "sync")
Rel(router, pipeline, "invokes", "gRPC")
Rel(router, api, "redirects cloud consult", "HTTP")
Rel(router, stream, "fallback text", "SSE")
Rel(api, gradcam, "requests overlay", "sync")
Rel(api, report, "generates PDF", "sync")
Rel(api, rag, "queries MOH", "SQL")
Rel(rag, referee, "validates citations", "sync")
Rel(api, nlp, "scrubs output", "sync")
Rel(api, audit, "writes event", "sync")
@enduml
Cloud LLM Gateway
@startuml
!include <C4/C4_Component>
Container_Boundary(cloud_gw, "cloud-llm-gateway (FastAPI)") {
Component(api, "Cloud API Controller", "REST + SSE", "/api/cloud-orchestrate, /api/cloud-consult, /api/cloud-consult/stream")
Component(gateway, "Cloud LLM Router", "task_type matcher", "Routes orchestration/translation → Gemini (tier_2); clinical deep-reasoning → MedGemma (tier_3)")
Component(consent, "Consent Enforcer", "Redis + NFR-16a checklist", "Validates consent token + redaction manifest before egress")
Component(audit, "Cloud Audit Emitter", "PostgreSQL append-only", "egress_consent, egress_response, cloud_llm_escalation events")
Component(cost, "Cost Guard", "Redis counter + Prometheus", "Tracks MedGemma usage ratio; alerts if >20% over 24h window")
Component(referee, "RAG-Referee Trigger", "BERT classifier", "Mandatory validation for MedGemma output before SSE to PWA")
Component(rag, "RAG Pre-processing", "pgvector SQL", "Mandatory top-k injection before any cloud LLM generation")
}
Rel(api, consent, "verifies", "sync")
Rel(api, gateway, "routes", "sync")
Rel(gateway, rag, "injects chunks", "sync")
Rel(gateway, referee, "validates output", "sync")
Rel(gateway, audit, "logs egress", "sync")
Rel(gateway, cost, "increments counter", "sync")
@enduml
4.4 Deployment Diagram (Tier 3)
@startuml
!include <C4/C4_Deployment>
LAYOUT_LEFT_RIGHT()
Deployment_Node(hw, "Dell PowerEdge (Hospital)") {
Deployment_Node(k3s, "K3s Cluster") {
Deployment_Node(pod_edge, "Pod: edge-inference-svc") {
Container(ci, "FastAPI", "Python 3.12, Uvicorn")
}
Deployment_Node(pod_rag, "Pod: rag-svc") {
Container(cr, "FastAPI", "Python 3.12, asyncpg")
}
Deployment_Node(pod_audit, "Pod: audit-svc") {
Container(ca, "Node.js", "WAL writer")
}
Deployment_Node(pod_gw, "Pod: api-gateway") {
Container(cg, "Envoy", "TLS termination, rate limit")
}
Deployment_Node(pod_auth, "Pod: auth-svc") {
Container(ck, "Keycloak", "OIDC, RBAC")
}
Deployment_Node(pod_obs, "Pod: observability") {
Container(cp, "Prometheus")
Container(cf, "Grafana")
}
}
Deployment_Node(db_vm, "DB VM") {
ContainerDb(pg, "PostgreSQL + pgvector HNSW")
ContainerDb(rd, "Redis")
ContainerDb(mn, "MinIO")
}
}
Deployment_Node(triton_node, "GPU Node (Triton)", "NVIDIA A10/T4, 24 GB VRAM") {
Container(tc, "Triton Inference Server", "ONNX/TensorRT, gRPC 8001")
}
Deployment_Node(ext, "External (NFR-16a governed)") {
ContainerDb(s3v, "S3 Vectors / GCP CDN / Vertex AI")
}
Rel(ci, pg, "SQL", "TCP 5432")
Rel(ci, rd, "Cache", "TCP 6379")
Rel(ci, mn, "Blobs", "S3 API")
Rel(ci, tc, "Inference", "gRPC 8001")
Rel(k3s, db_vm, "SQL", "TCP")
Rel(ci, ck, "Auth", "OIDC")
Rel(ci, ca, "Audit", "HTTP")
@enduml
5. Component Specifications
5.1 React PWA
| Concern | Decision |
|---|---|
| Framework | React 18 + TypeScript + Zustand |
| Styling | Tailwind CSS (mobile-first) |
| DICOM viewer | Cornerstone.js + custom canvas layer |
| Grad-CAM | <canvas> overlay; base64 PNG from FastAPI |
| Local model | LiteRT (MediaPipe Tasks Vision) — angle pre-classifier |
| Client storage | Dexie.js over IndexedDB (encrypted sessions, prefs) |
| Offline | Service Worker: cache-first for shell; network-first for API |
| Bundle | Tree-shake Cornerstone extras; split vendor chunk |
| PHI safety | Decree 13 regex scrubber before any network write |
| Edge guardrail | guardrail.worker.ts (WebWorker): Transformers.js BERT for hallucination/mal-intention detection; prompt injection scoring; scope-breach detection. OpenRedaction + pii-filter + js-data-anonymizer run in main thread or dedicated worker for redaction pipeline. Separate from cv.worker.ts (LiteRT) and llm.worker.ts (WebLLM) with no shared WASM memory. |
5.2 FastAPI Application (edge-inference-svc)
app/
├── main.py # Entry, middleware, lifespan
├── config.py # Pydantic settings (env-driven)
├── middleware/
│ ├── auth.py # Keycloak OIDC validation
│ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII
│ └── audit.py # Append-only event emitter
├── routers/ # FastAPI route handlers (existing + new cloud LLM)
│ ├── analyze.py # POST /api/analyze (sync 3-step pipeline)
│ ├── pacs.py # C-MOVE proxy + DICOM upload
│ ├── emr.py # HL7/FHIR push with outbox
│ ├── admin.py # Model update, drift review, cache invalidation
│ ├── cloud_orchestrate.py # POST /api/cloud-orchestrate (Gemini proxy for orchestration/translation)
│ └── cloud_consult.py # POST /api/cloud-consult + GET /api/cloud-consult/stream (MedGemma proxy)
├── services/ # Business logic modules (existing + new cloud gateway)
│ ├── inference_router.py # consult_mode state machine
│ ├── triton_client.py # gRPC with retry decorator
│ ├── rag.py # pgvector top-k + citation formatter
│ ├── referee.py # BERT drift + RAG confidence gate
│ ├── guardrail.py # Edge BERT violation scoring, session termination, cloud mitigation trigger
│ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable
│ ├── report.py # Circular 46 PDF + HITL signature
│ ├── ontology.py # ladybugDB C++ bindings wrapper
│ ├── audit_writer.py # WAL append
│ └── cloud_llm_gateway.py # Routes between Gemini (tier_2) and MedGemma (tier_3) based on task_type + consult_mode; NFR-16a consent/redaction/audit enforcement
├── models/
│ ├── dto.py # Request/response schemas (Pydantic v2)
│ ├── domain.py # Case, Session, Grade, Embedding entities
│ └── enums.py # ConsultMode (tier_1..tier_4), Grade, Tier
└── infra/
├── cache.py # Redis client (5 data types only)
├── db.py # SQLAlchemy async engine + pgvector
└── storage.py # MinIO S3 client
5.3 Knowledge Service (rag-svc)
Separated from inference to allow independent scaling of RAG queries:
- Endpoints:
POST /rag/query,POST /rag/referee-check,GET /rag/guideline/{version} - Reads pgvector only; no Triton dependency
- Publishes guideline update events to Redis Pub/Sub for cache invalidation
5.4 Data Models (Postgres)
Key tables (migration via Alembic):
guidelines (id, version, title, source_url, embedding vector(768), active_from, retired_at)
sessions (session_hash, radiologist_id, case_id, consult_mode, created_at, closed_at)
cases (case_id, patient_hash, joint_site, dicom_checksum, final_grade, finalized_at, signer_id)
embeddings (id, session_hash, chunk_text, embedding vector(768), source, created_at)
audit_events (id, event_hash, session_hash, actor, action, tier, consent_token, redaction_manifest, payload_hash, ts)
emr_outbox (id, case_id, payload, status, attempts, next_retry_at)
user_prefs (user_id, jsonb, updated_at) # synced from IndexedDB
Unique constraint: cases.case_id final grade requires signer_id != NULL (NFR-19).
5.5 Redis Keys (Exact Schema)
| Key pattern | Type | TTL | Purpose |
|---|---|---|---|
session:{hash} |
String + Hash | 3600s | JWT session validation |
guideline:{ver}:{chunk_id} |
String | 604800s (7d) | MOH guideline chunk |
dicom:{session_hash} |
String | 43200s (12h) | Per-session DICOM headers |
consult_mode:{session_hash} |
String | 7200s | Tier state (tier_1..tier_3b) |
rate:{actor_id}:{window} |
String | 30s sliding | Rate-limit counter |
Key-space invalidation: guideline:{ver}:* deleted on version bump via Postgres NOTIFY listener.
5.6 Triton gRPC Contract
service InferencePipeline {
rpc Run3Step (DICOMBytes) returns (PipelineResult);
rpc ExtractEmbedding (TextChunk) returns (EmbeddingVector);
}
message DICOMBytes {
bytes raw_dicom = 1;
string session_hash = 2;
uint32 max_vram_mb = 3;
}
message PipelineResult {
AngleClass angle = 1;
InflammationFlag inflammation = 2;
SegmentationMask mask = 3;
SynoviumMeasurement measurement = 4;
Grade grade = 5;
bytes gradcam_png = 6;
}
Idempotency invariant: identical raw_dicom + session_hash → identical output bytes. No partial state between steps.
5.7 Frontend Guardrail Runtime
WebWorker Topology
| Worker | Runtime | Role | Memory Cap |
|---|---|---|---|
cv.worker.ts |
LiteRT (WASM) | CV inference: angle pre-classifier, image preprocessing | Isolated WASM instance |
llm.worker.ts |
WebLLM (WASM) | GemmaE2B local generation, Gemma Functions tool-calling | Isolated WASM instance (NFR-4 1.5GB heap) |
guardrail.worker.ts |
Transformers.js (WASM/WebGPU) | BERT classification: hallucination, prompt-injection, scope-breach scoring | Shared Web Worker thread (no WASM memory pool) |
Isolation rules:
- No
SharedArrayBufferorAtomicsbetween workers — no raw WASM memory sharing. - Communication via
postMessagewith structured clone (no transferable object reuse for audit integrity). - Unload priority on memory pressure (browser
memorywarningevent):llm.worker.tsfirst, thenguardrail.worker.ts, thencv.worker.ts. - IndexedDB is the only persistence layer shared across workers; written by main thread or dedicated serializer, never read inside LLM context window.
Edge Guardrail Decision Contract
User Input → OpenRedaction + pii-filter → js-data-anonymizer → BERT Guardrail
↓
PASS: forward to RAG → LLM
FAIL: terminate LLM session → cloud mitigate
Server-side FastAPI contracts:
POST /api/guardrail-check(internal): accepts BERT score + query hash; returnsPASS|MITIGATE.POST /api/redaction-ground-check: accepts client manifest hash + sanitized payload; returnsPASS|CLEANED|ERROR.CLEANED= server refined residual PII and continues.ERROR= server unable to clean → client receives structured error.POST /api/consult/stream: now includesX-Guardrail-VersionandX-Redaction-Manifest-Hashheaders for audit.
IndexedDB Schema Additions
| Table | Key | Columns | Invalidation |
|---|---|---|---|
guardrail_models |
[model_name, version] |
artifact_hash, size_bytes, load_timestamp | Version mismatch |
policy_config |
[policy_name] |
version, rules_json, bert_thresholds | Admin push |
audit_tokens |
[session_hash, entity_type] |
token_value, created_at | Session expiry |
6. Interface Contracts (Selected)
6.1 REST Endpoints
| Method | Path | Auth | Request | Response | Notes |
|---|---|---|---|---|---|
| POST | /api/analyze |
JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, ≤1.5 s target |
| POST | /api/cloud-orchestrate |
JWT + consent | JSON (task_type, prompt) | JSON text + tier | Gemini proxy; orchestration/translation/UI tasks |
| POST | /api/cloud-consult |
JWT + consent | JSON (task_type, prompt) | JSON text + tier | MedGemma proxy; clinical deep-reasoning |
| GET | /api/cloud-consult/stream |
JWT + consent | query params | SSE text stream | MedGemma streaming; 25s timeout, 30s budget |
| POST | /api/emr/push |
JWT | case_id | 202 Accepted | Outbox if offline |
| POST | /api/admin/models |
Admin | modelZip | 200 OK | K3s rolling restart |
| GET | /api/rag/citations?q= |
JWT | query string | JSON top-5 chunks | NFR-18 |
| POST | /api/safety/guardrail-check |
JWT | payload | GuardrailResult | NFR-10 edge guardrail |
| GET | /api/health |
— | — | HealthStatus | Liveness |
| GET | /api/model-registry |
JWT | — | ModelCatalog | Model registry |
| POST | /api/models/register |
Admin | modelZip | RegistrationResult | Model upload |
6.2 SSE Consult Stream Contract
event: token
data: {"text":"The","confidence":0.92,"tier":"tier_2"}
event: citation
data: {"source":"MOH guideline 2024 §3.2","page":12,"tier":"tier_2"}
event: tier_change
data: {"from":"tier_1","to":"tier_2","reason":"edge_unavailable"}
event: done
data: {"tier":"tier_2","latency_ms":340,"model":"gemini-2.5-pro"}
7. Build vs Buy Matrix (Sprint 1-2 Specific)
| Component | Build | Buy | Decision | Rationale |
|---|---|---|---|---|
| Frontend PWA | Build | — | Build | Custom DICOM viewer + Grad-CAM layer; thin client requirement |
| FastAPI backend | Build | — | Build | Tight Decree 13 + NFR-16a middleware; in-house domain logic |
| Triton inference | — | Deploy self-hosted | Deploy | Open-source stack; no SaaS |
| Ontology (ladybugDB) | Build | — | Build | Embedded C++; SNOMED-CT mapping custom to MSK |
| pgvector | — | Postgres extension | Use | Zero infra overhead |
| Redis | — | OSS | Use | Scoped to 5 types; self-hosted |
| MinIO | — | OSS | Use | S3-compatible; self-hosted |
| Auth | Build (Keycloak) | Auth0/Okta SaaS | Build | NFR-16: Keycloak on-prem, no SaaS identity |
| Cloud LLM Gateway | Build | — | Build | FastAPI service enforcing NFR-16a redaction/consent/audit; routes Gemini/MedGemma based on task_type |
| Gemini (Vertex AI) | — | GCP PoC | Use (NFR-16a) | Orchestration, translation, UI planning; PoC only with redaction |
| MedGemma (Modal) | — | Modal deploy | Deploy (NFR-16a) | Clinical deep-reasoning; PoC only with redaction + RAG-Referee |
| EMR integration | Build | Mirth Connect | Build | Thin HL7 wrapper FastAPI → EMR; keeps surface area small |
| CI/CD | Jenkins in K3s | SaaS GitHub Actions | Build | Jenkins inside LAN; cloud GitLab via SSH |
| Issue tracking | Self-hosted Jira | Atlassian Cloud | Build (NFR-16a) | Cloud VM, compensating controls |
8. Task Decomposition
Phase 1: Foundation (Parallel)
-
T1-A Infra: K3s bootstrap + network policy (S)
Deploy K3s on Dell PowerEdge; Calico network policies; NGINX + Keepalived VIP; TLS secret automation. -
T1-B Database VM: Postgres + pgvector + MinIO + Redis (S)
Install Postgres 16 + pgvector extension; MinIO (4 disk RAID); Redis AOF+RDB; backup cron to MinIO. -
T1-C CI/CD: Jenkins in K3s + GitLab on cloud VM (M)
Jenkins agents as K3s jobs; pre-push PII hook; GitLab RDB backup job to MinIO; IP-whitelist IAM. -
T1-D Auth: Keycloak realm + RBAC (S)
Realmvkist-msk; roles: radiologist, senior_expert, admin; clientpwa,edge-svc,rag-svc. -
T1-E PWA shell: React + Zustand + PWA manifest (S)
Offline-capable shell; Dexie.js setup; Service Worker with cache-first for shell only.
Phase 2: Core ML Pipeline
-
T2-A Triton server + 3-step model ensemble (L)
Angle → Inflammation → Segmentation pipeline; gRPC server on port 8001; TensorRT engines. -
T2-B FastAPI edge-inference-svc (L)
/api/analyzeendpoint; DICOM ingest; pipeline orchestration; Grad-CAM overlay generation. -
T2-C Circuit Breaker + consult_mode state machine (M)
pybreaker around Triton + EMR; consult_mode Redis keys; SSE status push to PWA. -
T2-D Retry decorators (Triton + EMR) (S)
Exponential backoff; idempotency enforcement; outbox queue for EMR failures.
Phase 3: NLP & Knowledge
-
T3-A pgvector guideline ingestion pipeline (M)
Ingest MOH PDFs → chunk → embed (BERT/Writing-Alignment) → HNSW index; Postgres NOTIFY pub/sub. -
T3-B ladybugDB ontology setup (M)
SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI. -
T3-C RAG service + RAG-Referee (M)
/rag/query; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail. Mandatory for MedGemma (Tier 3) output; lightweight for Gemini (Tier 2) and Edge Gemma (Tier 1). -
T3-D 3-Model LLM System: Edge Gemma + Gemini + MedGemma (L)
Tier 1 (Edge Gemma): Browser WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in dedicated WebWorker.
Tier 2 (Gemini): GCP Vertex AI (GCP Vertex AI Gemini) for orchestration, translation, UI planning; FastAPI wrapper with streaming + Decree 13 redaction middleware.
Tier 3 (MedGemma): Modal-deployed MedGemma large-24b for clinical deep-reasoning and report finalization; FastAPI gateway with keep-warm instances, 30s timeout, RAG-Referee validation.
Tier 4: MOH guideline templates (rule-based fallback).
Consult mode state machine extended:tier_1→tier_2→tier_3→tier_4. -
T3-E Decree 13 scrubber (client + server) (M)
Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens. Applied to all cloud tiers (Tier 2 + Tier 3). -
T3-F Cloud LLM Gateway (M)
FastAPI service (/api/cloud-orchestrate,/api/cloud-consult,/api/cloud-consult/stream); routes based ontask_type; enforces consent + redaction + audit before egress; tracks MedGemma usage for <20% cost guard target; extends consult_mode state machine.
Phase 4: Compliance & HITL
-
T4-A Immutable audit log (M)
Append-only WAL writer (audit-svc); schema per NFR-17; every tier transition + consent + finalize event. -
T4-B HITL signature gate (S)
cases.finalized_at+signer_idnon-null constraint; digital signature capture (VKI token + timestamp). -
T4-C Circular 46 PDF report generator (M)
WeasyPrint template; includes grade, Grad-CAM image, MOH citations, signer block, audit hash.
Phase 5: Observability & Hardening
-
T5-A Prometheus + Grafana dashboards (M)
Triton VRAM/utilization; inference latency p50/p99; Redis hit rate; circuit-breaker state; K3s node health. -
T5-A BERT drift monitor (M)
Weekly batch job comparing current session embeddings vs. baseline; alert admin via Grafana if KL divergence > threshold. -
T5-C Fallback chain integration test (S)
Simulate Triton down → verify Tier 3a consent flow + redaction; simulate CDN down → verify GCP CDN signed URL fallback.
9. Execution Plan
Week 1-2: Phase 1
Deliverables: K3s cluster up, DB VM ready, CI/CD pipeline green, PWA shell live.
Week 3-4: Phase 2
Deliverables: /api/analyze end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure.
Week 5-6: Phase 3
Deliverables: RAG queries return MOH citations; Edge Gemma (WebLLM) primary, Gemini (Vertex AI) for orchestration/translation, MedGemma (Modal) for clinical deep-reasoning; RAG-Referee validates MedGemma output; consult_mode state machine extended to tier_1→tier_2→tier_3→tier_4.
Week 7: Phase 4
Deliverables: Audit log immutable with new event types (egress_consent_gemini, egress_consent_medgemma, cloud_llm_escalation); HITL signature enforces finalization; Circular 46 PDF exports; cost guarding (<20% MedGemma usage) enforced.
Week 8: Phase 5 + Sprint Review
Deliverables: Dashboards (tier transitions, MedGemma usage count, latency p50/p99, consent events), drift monitor, integration tests, demo-ready PWA with 3-model routing status.
10. References
- Solution Architecture Spec — full pattern citations, trade-offs, NFR-16a design
- Sprint 1-2 Architecture Spec — sprint-scoped container/component/deployment diagrams
- Agent Skills — coding convention, secrets/PHI safety, contract hygiene
- Codebase Structure — backend/frontend/infra/knowledge/ml layout