39 Commits

Author SHA1 Message Date
DatTT127
7d5c583475 update the modal deployment logic
All checks were successful
Triton Modal Trigger / deploy-to-modal (push) Successful in 11s
2026-07-17 13:51:58 +07:00
DatTT127
196d243e03 update the modal triton workflow
Some checks failed
Triton Modal Trigger / deploy-to-modal (push) Failing after 7s
2026-07-17 13:34:34 +07:00
DatTT127
77c32e93cd update the modal triton workflow
All checks were successful
Triton Modal Trigger / deploy-to-modal (push) Successful in 4s
2026-07-17 13:33:40 +07:00
DatTT127
f011435ab0 update the modal triton workflow
All checks were successful
Triton Modal Trigger / deploy-to-modal (push) Successful in 5s
2026-07-17 13:33:10 +07:00
DatTT127
f55d628e8d update the modal triton workflow
All checks were successful
Triton Modal Trigger / deploy-to-modal (push) Successful in 6s
2026-07-17 13:32:12 +07:00
DatTT127
f35329d6c0 update the modal triton workflow
All checks were successful
Triton Modal Trigger / deploy-to-modal (push) Successful in 8s
2026-07-17 13:30:58 +07:00
DatTT127
53980e2afc update the workflow
Some checks failed
Triton Modal Trigger / deploy-to-modal (push) Failing after 8s
2026-07-17 13:28:28 +07:00
DatTT127
86b135fdf0 update the modal-triton workflows
Some checks failed
Triton Modal Trigger / deploy-to-modal (push) Failing after 7s
2026-07-17 13:27:02 +07:00
DatTT127
9dce7ff426 update workflow
Some checks failed
Triton Modal Trigger / deploy-to-modal (push) Failing after 8s
2026-07-17 13:22:17 +07:00
DatTT127
900e2bb68b update the triton modal trigger
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Successful in 22s
Triton Modal Trigger / deploy-to-modal (push) Failing after 8s
2026-07-17 13:17:02 +07:00
DatTT127
06197447f7 update_again
All checks were successful
Test Gitea Secrets / deploy-to-modal (push) Successful in 19s
2026-07-17 00:29:00 +07:00
DatTT127
d38ca6bf23 update_again
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 16s
2026-07-17 00:26:13 +07:00
DatTT127
0f7f0dddd8 update_again
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 14s
2026-07-17 00:24:06 +07:00
DatTT127
bbdc205be2 update_again
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 18s
2026-07-17 00:22:57 +07:00
DatTT127
4906005cb9 update_again
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 15s
2026-07-17 00:22:00 +07:00
DatTT127
ee1ae91a7a update
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 13s
2026-07-17 00:17:12 +07:00
DatTT127
7913119351 update the test_secret.yaml
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 16s
2026-07-17 00:02:14 +07:00
DatTT127
2635ddfe8f update the test
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 32s
2026-07-16 23:59:43 +07:00
DatTT127
7b9db01f2a update the workflow test
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 16s
2026-07-16 23:41:57 +07:00
DatTT127
1e6953d5a1 update the workflows test
Some checks failed
Test Gitea Secrets / deploy-to-modal (push) Failing after 25s
2026-07-16 23:28:23 +07:00
DatTT127
f67fb7a135 update test_secret.yaml
Some checks failed
Test Gitea Secrets / print-secret (push) Failing after 3s
2026-07-16 22:54:32 +07:00
DatTT127
f896e5c932 update test_secret.yaml
Some checks failed
Test Gitea Secrets / print-secret (push) Failing after 10s
2026-07-16 22:50:40 +07:00
DatTT127
94f029eb19 update test_secret.yaml
Some checks failed
Test Gitea Secrets / print-secret (push) Failing after 12s
2026-07-16 22:48:09 +07:00
DatTT127
45972daf29 update test_secret.yaml
Some checks failed
Test Gitea Secrets / print-secret (push) Failing after 11s
2026-07-16 22:46:28 +07:00
DatTT127
8a8f91d99d update the test_gitea
All checks were successful
Test Gitea Secrets / print-secret (push) Successful in 50s
2026-07-16 22:34:16 +07:00
DatTT127
2e439d2787 include the test gitea secrets
Some checks failed
Test Gitea Secrets / print-secret (push) Failing after 2m41s
2026-07-16 16:41:32 +07:00
David Tran
7af1553032 Merge pull request #7 from DTJ-Tran/poc_1_3
add the session memory
2026-07-15 23:30:27 +07:00
DatTT127
1f6b015815 add the session memory 2026-07-15 23:29:26 +07:00
David Tran
eed2d39c2d Merge pull request #6 from DTJ-Tran/poc_1_3
update the cv modal inference proxy server with optimization
2026-07-15 23:25:26 +07:00
DatTT127
1f5e71b46a update the cv modal inference proxy server with optimization 2026-07-15 23:24:34 +07:00
David Tran
3435cfefa0 Merge pull request #5 from DTJ-Tran/poc2
update session_memory - for refer to the passwork
2026-07-07 22:03:18 +07:00
DatTT127
3fbbca1eaa update session_memory - for refer to the passwork 2026-07-07 22:01:30 +07:00
David Tran
4d89e55d86 Merge pull request #4 from DTJ-Tran/poc2
update the requirements.txt
2026-07-07 16:17:22 +07:00
DatTT127
d862d1f7f8 update the requirements.txt 2026-07-07 16:14:50 +07:00
David Tran
2484597c7a Merge pull request #3 from DTJ-Tran/poc1
Poc1-Proof of Concept verison 1
2026-07-07 15:56:36 +07:00
DatTT127
1622dc8fc5 update the codebase poc ver1 2026-07-07 15:54:17 +07:00
HuyTa12112001
41e6a2a416 Refactor code predict imflammation 2026-06-27 16:25:43 +07:00
David Tran
41241489fb Merge pull request #2 from DTJ-Tran/update_
Update
2026-06-24 16:16:18 +07:00
David Tran
43f5c0f7da Merge pull request #1 from DTJ-Tran/update_
update the architecture & description
2026-06-24 10:35:20 +07:00
498 changed files with 89596 additions and 66524 deletions

View File

@@ -0,0 +1,38 @@
name: Test Gitea Secrets
on: workflow_dispatch
jobs:
deploy-to-modal:
runs-on: ubuntu-latest
container:
image: gitea-modal-runner:latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Debug Modal Credentials
run: |
echo "DEBUG: Token ID is: '${MODAL_TOKEN_ID}'"
echo "DEBUG: Token ID length is: ${#MODAL_TOKEN_ID}"
# Use 'cut' to get the first 5 characters safely
FIRST_5=$(echo "$MODAL_TOKEN_ID" | cut -c1-5)
echo "DEBUG: First 5 chars are: '$FIRST_5'"
env:
# This maps the Gitea Repository Secret to an Environment Variable
MODAL_TOKEN_ID: ${{ secrets.MODAL_KEY }}
MODAL_TOKEN_SECRET: ${{ secrets.MODAL_SECRET }}
- name: Deploy worker pipeline to Modal
run: |
echo "DEBUG: Token ID is: '${MODAL_TOKEN_ID}'"
echo "DEBUG: Token ID length is: ${#MODAL_TOKEN_ID}"
# Use 'cut' to get the first 5 characters safely
FIRST_5=$(echo "$MODAL_TOKEN_ID" | cut -c1-5)
echo "DEBUG: First 5 chars are: '$FIRST_5'"
cd workspace/sprint_1_2/CODEBASE/deps/implementation
modal deploy test_worker.py
env:
# This maps the Gitea Repository Secret to an Environment Variable
MODAL_TOKEN_ID: ${{ secrets.MODAL_KEY }}
MODAL_TOKEN_SECRET: ${{ secrets.MODAL_SECRET }}

View File

@@ -0,0 +1,28 @@
# What is the purpose of this workflows
# for a modal GPU instance that hosting the CV model's on Nvidia Triton
name: Triton Modal Trigger
on: [push]
jobs:
deploy-to-modal:
runs-on: ubuntu-latest
container:
image: gitea-modal-runner:latest # where build the code
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
sparse-checkout: |
workspace/sprint_1_2/CODEBASE/infra/implementation/triton_run/*
sparse-checkout-cone-mode: false
- name: Deploy Triton model to Modal
run: |
echo "Deploying Triton..."
cd workspace/sprint_1_2/CODEBASE/infra/implementation/triton_run
modal deploy modal_triton.py
env:
MODAL_TOKEN_ID: ${{ secrets.MODAL_KEY }}
MODAL_TOKEN_SECRET: ${{ secrets.MODAL_SECRET }}

28
.gitignore vendored
View File

@@ -212,3 +212,31 @@ secrets/
*.claude *.claude
*.kilo *.kilo
*.zip *.zip
tmp/
corpus/
corpus_db/
.locks/
models--gliner-community--gliner_small-v2.5/
models--microsoft--deberta-v3-small/
ner_model/
gemma_emb/
.cursor/
.husky/
Policy_Analysis/
.vscode/
package-lock.json
package.json
.env.development
run.sh
workspace/sprint_1_2/CODEBASE/knowledge/implementation/model/
*.onnx_data
*.onnx
*.bin
*.safetensors
*.pt
*.pth
*.pdf
*.xz
logs/
*.rdb
dist/

View File

@@ -17,3 +17,4 @@ Create up to 5 shareable agent skills under `PILOT_PROJECT/AGENT_SKILL/` that au
| interface_contract_hygiene | Contract files, versioning, breaking-change policy | | interface_contract_hygiene | Contract files, versioning, breaking-change policy |
| secrets_and_phi_safety | Secrets loading, PHI exclusion, CI safety, gitignore | | secrets_and_phi_safety | Secrets loading, PHI exclusion, CI safety, gitignore |
| design_pattern_compliance | Layered arch, DI, Zustand, Terraform IaC, pinning | | design_pattern_compliance | Layered arch, DI, Zustand, Terraform IaC, pinning |
| agent_tool_execution | Exa/Supabase/MedGemma agent tools, BFF proxy, MediaPipe output grammar |

View File

@@ -0,0 +1,53 @@
# Skill: agent_tool_execution
## Trigger Conditions
Any change involving Gemma-4-E2B agent tools, Exa search, Supabase RAG RPC, or MedGemma escalation.
## Rules
### 1. Tool catalog is authoritative
- Only use: `exa_search`, `supabase_query`, `escalate_medgemma`.
- Never invent tool names in prompts or handlers.
- Bump `TOOL_CATALOG_VERSION` on breaking schema changes.
### 2. Secrets and egress
- `EXA_API_KEY`, `SUPABASE_SERVICE_ROLE_KEY`, MedGemma keys: **backend only**.
- Browser calls BFF routes; never `https://api.exa.ai` directly from the client.
- PHI-scrub queries before any network egress ([`secrets_and_phi_safety`](../secrets_and_phi_safety/SKILL.md)).
### 3. Exa defaults (Pattern 1 — raw retrieval)
```json
{
"type": "auto",
"numResults": 10,
"contents": { "highlights": true }
}
```
- Canonical reference: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
- Do not use deprecated params (`useAutoprompt`, top-level `highlights`, `livecrawl: "always"`).
- `outputSchema` synthesis is BFF-only enrichment; not the default agent-loop path.
### 4. Retrieval policy (NFR-18)
- `supabase_query` first for in-corpus MOH guideline grounding.
- `exa_search` for external/recency queries.
- Final clinical answers must cite chunk IDs (Supabase) or URLs (Exa).
### 5. MedGemma escalation
- Requires consent + audit (NFR-16a) before `escalate_medgemma`.
- No direct Modal URLs from the browser.
### 6. MediaPipe agent output
- Parse tool calls from the **answer channel** when CoT is enabled.
- One block per turn: `<tool_call>` OR `<final>`.
## Package location
[`ml/implementation/nlp/agent_runtime/`](../ml/implementation/nlp/agent_runtime/)

View File

@@ -27,8 +27,6 @@ PILOT_PROJECT/
│ │ ├── LEGACY # The legacy material - for archieved / pre-existed project │ │ ├── LEGACY # The legacy material - for archieved / pre-existed project
│ │ ├── CODEBASE # The project codebase │ │ ├── CODEBASE # The project codebase
│ │ ├── Design_Material/ # The system design - API - and UIX design material │ │ ├── Design_Material/ # The system design - API - and UIX design material
│ │ ├── SOFTWARE_SYSTEM_DESIGN_FR_25.md # Detailed software design specification
│ │ ├── SOLUTION_ARCHITECTURE_SPEC.md # Solution architecture overview
│ │ └── VISUALIZATION/ # Charts, diagrams, and visual assets │ │ └── VISUALIZATION/ # Charts, diagrams, and visual assets
│ └── ... # Additional sprint folders as needed │ └── ... # Additional sprint folders as needed
├── secrets/ # **Developermanaged secrets** (NOT tracked by git) ├── secrets/ # **Developermanaged secrets** (NOT tracked by git)

View File

@@ -17,25 +17,30 @@ Build a reproducible, air-gapped-first musculoskeletal ultrasound analysis platf
### Functional (Sprint 1-2) ### Functional (Sprint 1-2)
- [ ] FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3 - [ ] FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3
- [ ] Grad-CAM overlay on primary viewport (zero extra clicks) - [ ] Grad-CAM overlay on primary viewport (zero extra clicks)
- [ ] Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing) - [ ] Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing) across 3-tier LLM system (Edge Gemma → Gemini → MedGemma)
- [ ] BERT drift monitor against baseline MOH corpus - [ ] BERT drift monitor against baseline MOH corpus
- [ ] RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks - [ ] RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks (mandatory for cloud tiers, lightweight for edge)
- [ ] Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware) - [ ] Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware)
- [ ] ladybugDB ontology traversal for anatomical entity disambiguation - [ ] ladybugDB ontology traversal for anatomical entity disambiguation
- [ ] GemmaE2B/MedGemma Vietnamese LLM consult (browser WebLLM local OR cloud Vertex AI) with MOH guideline citations - [ ] 3-model LLM consult system: Edge Gemma (browser WebLLM local), Gemini (GCP Vertex AI, orchestration/translation), MedGemma (Modal, clinical deep-reasoning) with MOH guideline citations and cost guarding (<20% MedGemma usage)
- [ ] Circular 46/2018 PDF report generation - [ ] Circular 46/2018 PDF report generation
- [ ] Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19) - [ ] Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19)
### Non-Functional (Critical) ### Non-Functional (Critical)
- [ ] NFR-4: 150 MB idle app bundle - [ ] NFR-4: 150 MB idle app bundle
- [ ] NFR-5: ≤1.5 s inference (on-prem Triton) - [ ] NFR-5: 1.5 s inference (on-prem Triton); cloud tiers targeted at <30s median
- [ ] NFR-7: ≤200 ms TTFT token streaming - [ ] NFR-7: 200 ms TTFT token streaming; progressive streaming with 25s timeout for cloud responses
- [ ] NFR-8: Fault-tolerant across Wi-Fi drops (state preserved) - [ ] NFR-8: Fault-tolerant across Wi-Fi drops (state preserved)
- [ ] NFR-10: Automated Generative Safety Guardrails: <90% verification prohibited, 100% of LLM-generated patient text explanations must pass verification. 3-tier guardrail: prompt rules + BERT edge detection (Tier 1) Vertex AI safety filters (Tier 2) RAG-Referee mandatory validation (Tier 3)
- [ ] NFR-11: Onboarding 45 minutes
- [ ] NFR-12: Zero-Friction Explainability Integration
- [ ] NFR-13: Spatial Layer-Activation Mapping
- [ ] NFR-14: No client-side GPU/neural accelerator required - [ ] NFR-14: No client-side GPU/neural accelerator required
- [ ] NFR-15: Circular 46 EMR compliance - [ ] NFR-15: Circular 46 EMR compliance
- [ ] NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit - [ ] NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit for cloud LLM tiers (Gemini + MedGemma)
- [ ] NFR-16a: Emergency Cloud Fallback with Redaction (PoC-SCOPE). Cloud LLM API (GCP Vertex AI Gemini + Modal MedGemma) invoked ONLY when browser-side WebLLM (Edge Gemma) is unavailable or orchestrates cloud task. FastAPI Redaction Middleware MUST strip all Decree 13 PII fields before egress. GCP CSEK + 30-day lifecycle + 1-year auto-delete. Audit-log commit + consent before egress. MedGemma usage <20% target with alerting. NFR-16a retired upon PoC sign-off.
- [ ] NFR-17: Immutable audit log - [ ] NFR-17: Immutable audit log
- [ ] NFR-18: 100% LLM text cites MOH protocol via RAG - [ ] NFR-18: 100% LLM text cites MOH protocol via RAG. Mandatory RAG pre-processing for all 3 tiers (not optional tool-calling). RAG-Referee validates MedGemma output.
- [ ] NFR-19: HITL digital signature before FINALIZED/ARCHIVED - [ ] NFR-19: HITL digital signature before FINALIZED/ARCHIVED
--- ---
@@ -79,7 +84,8 @@ System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") { System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
System_Boundary(edge_servers, "Application Server Cluster") { System_Boundary(edge_servers, "Application Server Cluster") {
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.") Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.")
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, GemmaE2B/MedGemma consult route, RAG-Referee validation.") Container(cloud_llm_gateway, "Cloud LLM Gateway", "FastAPI (Python 3.12, Uvicorn)", "Routes orchestration/translation to Gemini (GCP) and clinical deep-reasoning to MedGemma (Modal); NFR-16a redaction, consent, audit enforcement; consult_mode state machine extension; cost guarding for MedGemma usage.")
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, RAG-Referee validation for MedGemma output, mandatory RAG pre-processing for all tiers.")
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.") Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.")
Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.") Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.")
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.") Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.")
@@ -94,11 +100,12 @@ System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
} }
} }
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + embedding extraction.") System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + EmbeddingGemma RAG embeddings.")
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.") System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.")
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.") System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.")
System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.") System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.")
System_Ext(vertex_ai, "GCP Vertex AI", "GemmaE2B via REST", "PoC-only NFR-16a inference tier; redacted payloads only.") System_Ext(vertex_ai, "GCP Vertex AI (Gemini)", "Gemini via REST", "PoC-only NFR-16a inference tier: orchestration, translation, UI planning; redacted payloads only.")
System_Ext(modal_medgemma, "Modal MedGemma", "FastAPI + Transformers (T4 GPU)", "PoC-only NFR-16a clinical deep-reasoning endpoint; redacted payloads with RAG-Referee validation; keep-warm instances for latency budget.")
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443") Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443")
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH") Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH")
@@ -118,9 +125,16 @@ Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
Rel(edge_inference, auth_svc, "Token introspection", "OIDC") Rel(edge_inference, auth_svc, "Token introspection", "OIDC")
Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090") Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090")
Rel(edge_inference, cloud_llm_gateway, "Routes cloud consult", "HTTP 8000")
Rel(cloud_llm_gateway, vertex_ai, "Gemini proxy (NFR-16a redacted)", "HTTPS 443 / REST")
Rel(cloud_llm_gateway, modal_medgemma, "MedGemma proxy (NFR-16a redacted)", "HTTPS 443 / REST")
Rel(cloud_llm_gateway, redis, "Consult mode, consent, rate-limit", "TCP 6379")
Rel(cloud_llm_gateway, postgres, "Audit log append", "SQL 5432")
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432") Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379") Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API") Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API")
Rel(rag_svc, edge_inference, "RAG + RAG-Referee results", "HTTP")
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432") Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR") Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
@@ -130,7 +144,6 @@ Rel(edge_inference, emr, "Report push (via audit-svc wrapper)", "HL7/FHIR")
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM") Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)") Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)")
Rel(edge_inference, vertex_ai, "PoC-only cloud consult (NFR-16a redacted)", "HTTPS 443 / REST")
Rel(postgres, minio, "Backup checkpoint", "S3 API") Rel(postgres, minio, "Backup checkpoint", "S3 API")
@enduml @enduml
@@ -143,8 +156,14 @@ Container communication summary:
| PWA | NGINX | HTTPS 443 | All API requests | | PWA | NGINX | HTTPS 443 | All API requests |
| NGINX | Envoy | HTTP 8000 | Route upstream | | NGINX | Envoy | HTTP 8000 | Route upstream |
| Envoy | Edge Inference | HTTP 8000 | `/api/*` | | Envoy | Edge Inference | HTTP 8000 | `/api/*` |
| Envoy | Cloud LLM Gateway | HTTP 8000 | `/api/cloud-orchestrate`, `/api/cloud-consult` |
| Envoy | RAG Service | HTTP 8001 | `/rag/*` | | Envoy | RAG Service | HTTP 8001 | `/rag/*` |
| Envoy | Keycloak | HTTP 8080 | OIDC validation | | Envoy | Keycloak | HTTP 8080 | OIDC validation |
| Edge Inference | Cloud LLM Gateway | HTTP 8000 | Route cloud consult |
| Cloud LLM Gateway | Gemini (Vertex AI) | HTTPS 443 / REST | Orchestration, translation, UI planning (NFR-16a) |
| Cloud LLM Gateway | MedGemma (Modal) | HTTPS 443 / REST | Clinical deep-reasoning, report finalization (NFR-16a) |
| Cloud LLM Gateway | Redis | TCP 6379 | Consult mode, consent, rate-limit |
| Cloud LLM Gateway | Postgres | TCP 5432 | Audit log append |
| Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings | | Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings |
| Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW | | Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW |
| Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode | | Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode |
@@ -165,23 +184,24 @@ Container communication summary:
!include <C4/C4_Component> !include <C4/C4_Component>
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") { Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream") Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/cloud-orchestrate redirect")
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s") Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)") Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates") Component(router, "Inference Router", "consult_mode state", "Tier selection: Edge Gemma (tier_1) → Gemini (tier_2, orchestration) → MedGemma (tier_3, clinical) → Templates (tier_4)")
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates") Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR + Cloud LLM calls; fail-open to templates")
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement") Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA") Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED") Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation") Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; mandatory pre-generation for all tiers; enforce NFR-18 citation")
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold") Component(referee, "RAG-Referee", "BERT classifier", "Reject MedGemma text if citation confidence < threshold; mandatory for tier_3")
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable") Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override") Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override; cloud_llm_escalation events")
} }
Rel(api, stream, "delegates", "SSE") Rel(api, stream, "delegates", "SSE")
Rel(api, preproc, "validates image", "sync") Rel(api, preproc, "validates image", "sync")
Rel(api, router, "selects tier", "sync") Rel(api, router, "selects tier", "sync")
Rel(router, pipeline, "invokes", "gRPC") Rel(router, pipeline, "invokes", "gRPC")
Rel(router, api, "redirects cloud consult", "HTTP")
Rel(router, stream, "fallback text", "SSE") Rel(router, stream, "fallback text", "SSE")
Rel(api, gradcam, "requests overlay", "sync") Rel(api, gradcam, "requests overlay", "sync")
Rel(api, report, "generates PDF", "sync") Rel(api, report, "generates PDF", "sync")
@@ -192,39 +212,27 @@ Rel(api, audit, "writes event", "sync")
@enduml @enduml
``` ```
### 4.3 Component Diagrams (Tier 3) **Cloud LLM Gateway**
**Edge Inference Service (edge-inference-svc)**
```plantuml ```plantuml
@startuml @startuml
!include <C4/C4_Component> !include <C4/C4_Component>
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") { Container_Boundary(cloud_gw, "cloud-llm-gateway (FastAPI)") {
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream") Component(api, "Cloud API Controller", "REST + SSE", "/api/cloud-orchestrate, /api/cloud-consult, /api/cloud-consult/stream")
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s") Component(gateway, "Cloud LLM Router", "task_type matcher", "Routes orchestration/translation → Gemini (tier_2); clinical deep-reasoning → MedGemma (tier_3)")
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)") Component(consent, "Consent Enforcer", "Redis + NFR-16a checklist", "Validates consent token + redaction manifest before egress")
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates") Component(audit, "Cloud Audit Emitter", "PostgreSQL append-only", "egress_consent, egress_response, cloud_llm_escalation events")
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates") Component(cost, "Cost Guard", "Redis counter + Prometheus", "Tracks MedGemma usage ratio; alerts if >20% over 24h window")
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement") Component(referee, "RAG-Referee Trigger", "BERT classifier", "Mandatory validation for MedGemma output before SSE to PWA")
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA") Component(rag, "RAG Pre-processing", "pgvector SQL", "Mandatory top-k injection before any cloud LLM generation")
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation")
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
} }
Rel(api, stream, "delegates", "SSE") Rel(api, consent, "verifies", "sync")
Rel(api, router, "selects tier", "sync") Rel(api, gateway, "routes", "sync")
Rel(router, pipeline, "invokes", "gRPC") Rel(gateway, rag, "injects chunks", "sync")
Rel(router, stream, "fallback text", "SSE") Rel(gateway, referee, "validates output", "sync")
Rel(api, preproc, "validates image", "sync") Rel(gateway, audit, "logs egress", "sync")
Rel(api, gradcam, "requests overlay", "sync") Rel(gateway, cost, "increments counter", "sync")
Rel(api, report, "generates PDF", "sync")
Rel(api, rag, "queries MOH", "SQL")
Rel(rag, referee, "validates citations", "sync")
Rel(api, nlp, "scrubs output", "sync")
Rel(api, audit, "writes event", "sync")
@enduml @enduml
``` ```
@@ -309,13 +317,14 @@ app/
│ ├── auth.py # Keycloak OIDC validation │ ├── auth.py # Keycloak OIDC validation
│ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII │ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII
│ └── audit.py # Append-only event emitter │ └── audit.py # Append-only event emitter
├── routers/ ├── routers/ # FastAPI route handlers (existing + new cloud LLM)
│ ├── analyze.py # POST /api/analyze (sync 3-step pipeline) │ ├── analyze.py # POST /api/analyze (sync 3-step pipeline)
│ ├── consult.py # SSE /api/consult/stream (GemmaE2B/MedGemma + RAG + guardrail session management)
│ ├── pacs.py # C-MOVE proxy + DICOM upload │ ├── pacs.py # C-MOVE proxy + DICOM upload
│ ├── emr.py # HL7/FHIR push with outbox │ ├── emr.py # HL7/FHIR push with outbox
── admin.py # Model update, drift review, cache invalidation ── admin.py # Model update, drift review, cache invalidation
├── services/ │ ├── cloud_orchestrate.py # POST /api/cloud-orchestrate (Gemini proxy for orchestration/translation)
│ └── cloud_consult.py # POST /api/cloud-consult + GET /api/cloud-consult/stream (MedGemma proxy)
├── services/ # Business logic modules (existing + new cloud gateway)
│ ├── inference_router.py # consult_mode state machine │ ├── inference_router.py # consult_mode state machine
│ ├── triton_client.py # gRPC with retry decorator │ ├── triton_client.py # gRPC with retry decorator
│ ├── rag.py # pgvector top-k + citation formatter │ ├── rag.py # pgvector top-k + citation formatter
@@ -324,11 +333,12 @@ app/
│ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable │ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable
│ ├── report.py # Circular 46 PDF + HITL signature │ ├── report.py # Circular 46 PDF + HITL signature
│ ├── ontology.py # ladybugDB C++ bindings wrapper │ ├── ontology.py # ladybugDB C++ bindings wrapper
── audit_writer.py # WAL append ── audit_writer.py # WAL append
│ └── cloud_llm_gateway.py # Routes between Gemini (tier_2) and MedGemma (tier_3) based on task_type + consult_mode; NFR-16a consent/redaction/audit enforcement
├── models/ ├── models/
│ ├── dto.py # Request/response schemas (Pydantic v2) │ ├── dto.py # Request/response schemas (Pydantic v2)
│ ├── domain.py # Case, Session, Grade, Embedding entities │ ├── domain.py # Case, Session, Grade, Embedding entities
│ └── enums.py # ConsultMode, Grade, Tier │ └── enums.py # ConsultMode (tier_1..tier_4), Grade, Tier
└── infra/ └── infra/
├── cache.py # Redis client (5 data types only) ├── cache.py # Redis client (5 data types only)
├── db.py # SQLAlchemy async engine + pgvector ├── db.py # SQLAlchemy async engine + pgvector
@@ -443,22 +453,31 @@ Server-side FastAPI contracts:
| Method | Path | Auth | Request | Response | Notes | | Method | Path | Auth | Request | Response | Notes |
|--------|------|------|---------|----------|-------| |--------|------|------|---------|----------|-------|
| POST | `/api/analyze` | JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, 1.5 s target | | POST | `/api/analyze` | JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, 1.5 s target |
| GET | `/api/consult/stream` | JWT + consent | SSE text stream | NDJSON chunks | Token streaming ≤200 ms TTFT | | POST | `/api/cloud-orchestrate` | JWT + consent | JSON (task_type, prompt) | JSON text + tier | Gemini proxy; orchestration/translation/UI tasks |
| POST | `/api/cloud-consult` | JWT + consent | JSON (task_type, prompt) | JSON text + tier | MedGemma proxy; clinical deep-reasoning |
| GET | `/api/cloud-consult/stream` | JWT + consent | query params | SSE text stream | MedGemma streaming; 25s timeout, 30s budget |
| POST | `/api/emr/push` | JWT | case_id | 202 Accepted | Outbox if offline | | POST | `/api/emr/push` | JWT | case_id | 202 Accepted | Outbox if offline |
| POST | `/api/admin/models` | Admin | modelZip | 200 OK | K3s rolling restart | | POST | `/api/admin/models` | Admin | modelZip | 200 OK | K3s rolling restart |
| GET | `/api/rag/citations?q=` | JWT | query string | JSON top-5 chunks | NFR-18 | | GET | `/api/rag/citations?q=` | JWT | query string | JSON top-5 chunks | NFR-18 |
| POST | `/api/safety/guardrail-check` | JWT | payload | GuardrailResult | NFR-10 edge guardrail |
| GET | `/api/health` | | | HealthStatus | Liveness |
| GET | `/api/model-registry` | JWT | | ModelCatalog | Model registry |
| POST | `/api/models/register` | Admin | modelZip | RegistrationResult | Model upload |
### 6.2 SSE Consult Stream Contract ### 6.2 SSE Consult Stream Contract
``` ```
event: token event: token
data: {"text":"The","confidence":0.92} data: {"text":"The","confidence":0.92,"tier":"tier_2"}
event: citation event: citation
data: {"source":"MOH guideline 2024 §3.2","page":12} data: {"source":"MOH guideline 2024 §3.2","page":12,"tier":"tier_2"}
event: tier_change
data: {"from":"tier_1","to":"tier_2","reason":"edge_unavailable"}
event: done event: done
data: {"tier":"tier_2","latency_ms":340} data: {"tier":"tier_2","latency_ms":340,"model":"gemini-2.5-pro"}
``` ```
--- ---
@@ -475,6 +494,9 @@ data: {"tier":"tier_2","latency_ms":340}
| Redis | | OSS | **Use** | Scoped to 5 types; self-hosted | | Redis | | OSS | **Use** | Scoped to 5 types; self-hosted |
| MinIO | | OSS | **Use** | S3-compatible; self-hosted | | MinIO | | OSS | **Use** | S3-compatible; self-hosted |
| Auth | Build (Keycloak) | Auth0/Okta SaaS | **Build** | NFR-16: Keycloak on-prem, no SaaS identity | | Auth | Build (Keycloak) | Auth0/Okta SaaS | **Build** | NFR-16: Keycloak on-prem, no SaaS identity |
| Cloud LLM Gateway | Build | | **Build** | FastAPI service enforcing NFR-16a redaction/consent/audit; routes Gemini/MedGemma based on task_type |
| Gemini (Vertex AI) | | GCP PoC | **Use (NFR-16a)** | Orchestration, translation, UI planning; PoC only with redaction |
| MedGemma (Modal) | | Modal deploy | **Deploy (NFR-16a)** | Clinical deep-reasoning; PoC only with redaction + RAG-Referee |
| EMR integration | Build | Mirth Connect | **Build** | Thin HL7 wrapper FastAPI EMR; keeps surface area small | | EMR integration | Build | Mirth Connect | **Build** | Thin HL7 wrapper FastAPI EMR; keeps surface area small |
| CI/CD | Jenkins in K3s | SaaS GitHub Actions | **Build** | Jenkins inside LAN; cloud GitLab via SSH | | CI/CD | Jenkins in K3s | SaaS GitHub Actions | **Build** | Jenkins inside LAN; cloud GitLab via SSH |
| Issue tracking | Self-hosted Jira | Atlassian Cloud | **Build (NFR-16a)** | Cloud VM, compensating controls | | Issue tracking | Self-hosted Jira | Atlassian Cloud | **Build (NFR-16a)** | Cloud VM, compensating controls |
@@ -520,15 +542,20 @@ data: {"tier":"tier_2","latency_ms":340}
SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI. SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI.
- [ ] **T3-C RAG service + RAG-Referee** (M) - [ ] **T3-C RAG service + RAG-Referee** (M)
`/rag/query`; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail. `/rag/query`; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail. Mandatory for MedGemma (Tier 3) output; lightweight for Gemini (Tier 2) and Edge Gemma (Tier 1).
- [ ] **T3-D Browser WebLLM + Cloud MedGemma LLM endpoints** (L) - [ ] **T3-D 3-Model LLM System: Edge Gemma + Gemini + MedGemma** (L)
Browser: WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in separate WebWorker from CV pipeline. Tier 1 (Edge Gemma): Browser WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in dedicated WebWorker.
Cloud: MedGemma on GCP Vertex AI (NFR-16a governed); FastAPI wrapper with streaming + Decree 13 redaction middleware. Tier 2 (Gemini): GCP Vertex AI (GCP Vertex AI Gemini) for orchestration, translation, UI planning; FastAPI wrapper with streaming + Decree 13 redaction middleware.
Triton: EmbeddingGemma only (768-dim RAG embeddings), no LLM hosting. Tier 3 (MedGemma): Modal-deployed MedGemma large-24b for clinical deep-reasoning and report finalization; FastAPI gateway with keep-warm instances, 30s timeout, RAG-Referee validation.
Tier 4: MOH guideline templates (rule-based fallback).
Consult mode state machine extended: `tier_1` `tier_2` `tier_3` `tier_4`.
- [ ] **T3-E Decree 13 scrubber (client + server)** (M) - [ ] **T3-E Decree 13 scrubber (client + server)** (M)
Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens. Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens. Applied to all cloud tiers (Tier 2 + Tier 3).
- [ ] **T3-F Cloud LLM Gateway** (M)
FastAPI service (`/api/cloud-orchestrate`, `/api/cloud-consult`, `/api/cloud-consult/stream`); routes based on `task_type`; enforces consent + redaction + audit before egress; tracks MedGemma usage for <20% cost guard target; extends consult_mode state machine.
### Phase 4: Compliance & HITL ### Phase 4: Compliance & HITL
- [ ] **T4-A Immutable audit log** (M) - [ ] **T4-A Immutable audit log** (M)
@@ -561,13 +588,13 @@ Deliverables: K3s cluster up, DB VM ready, CI/CD pipeline green, PWA shell live.
Deliverables: `/api/analyze` end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure. Deliverables: `/api/analyze` end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure.
### Week 5-6: Phase 3 ### Week 5-6: Phase 3
Deliverables: RAG queries return MOH citations; GemmaE2B/MedGemma streams Vietnamese explanations; RAG-Referee blocks unmapped LLM text. Deliverables: RAG queries return MOH citations; Edge Gemma (WebLLM) primary, Gemini (Vertex AI) for orchestration/translation, MedGemma (Modal) for clinical deep-reasoning; RAG-Referee validates MedGemma output; consult_mode state machine extended to tier_1→tier_2→tier_3→tier_4.
### Week 7: Phase 4 ### Week 7: Phase 4
Deliverables: Audit log immutable; HITL signature enforces finalization; Circular 46 PDF exports. Deliverables: Audit log immutable with new event types (egress_consent_gemini, egress_consent_medgemma, cloud_llm_escalation); HITL signature enforces finalization; Circular 46 PDF exports; cost guarding (<20% MedGemma usage) enforced.
### Week 8: Phase 5 + Sprint Review ### Week 8: Phase 5 + Sprint Review
Deliverables: Dashboards, drift monitor, integration tests, demo-ready PWA. Deliverables: Dashboards (tier transitions, MedGemma usage count, latency p50/p99, consent events), drift monitor, integration tests, demo-ready PWA with 3-model routing status.
--- ---

View File

@@ -73,14 +73,19 @@ Guided by the above requirements and the solution_architect_corpus, the architec
### 4.1 High-Level Overview ### 4.1 High-Level Overview
The system adopts a hybrid edge-cloud (on-premise) architecture with the following layers: The system adopts a hybrid edge-cloud (on-premise) architecture with the following layers:
1. **Edge Inference Layer**: Located on-premise at hospitals/clinics, runs lightweight ML models for real-time DICOM processing (NFR-5,6). 1. **Edge Inference Layer**: Located on-premise at hospitals/clinics, runs lightweight ML models for real-time DICOM processing (NFR-5,6).
2. **Micro-Services Layer**: Containerized services orchestrated by K3s (Kubernetes-certified, lightweight distribution designed for edge and resource-constrained production environments). K3s selected over alternatives (Docker Swarm, Nomad, ECS Fargate, Cloud Run) on the following criteria: NFR-16 requires on-premise deployment eliminating ECS Fargate and Cloud Run (cloud-only platforms). Docker Swarm offers lowest PoC operational cost but is in maintenance mode with no viable scaling path; migration to production-grade orchestration requires complete rewrite of all deployment artifacts. Nomad is operationally viable but lacks ecosystem depth and has less documented migration path to multi-cluster federation. K3s is already production-grade at single-site scale, requires no migration to "production" it IS the production platform. The scaling path to N hospitals is multi-cluster federation via K3s cluster-api, not platform replacement. All manifests and service definitions are forward-compatible with upstream Kubernetes. 2. **3-Tier LLM System**:
3. **Data Layer**: On-premise databases (PostgreSQL + pgvector for relational + vector search, Redis for caching, MinIO for S3-compatible object storage). - **Tier 1 (Edge Gemma)**: Browser/WebLLM (GemmaE2B local in WASM) primary conversation, local inference, instant response, zero network cost. Orchestrates when to call cloud models via FastAPI middleware.
3. **Embedding Layer**: EmbeddingGemma (768-dim) for RAG retrieval vectors; separate from BERT which is reserved for drift monitoring and RAG-Referee classification only. - **Tier 2 (Gemini)**: GCP Vertex AI (Gemini) orchestration, translation, UI planning, formatting. Governed by NFR-16a with redaction, consent, and audit logging.
3. **Vector Layer (two-tier)**: - **Tier 3 (MedGemma)**: Modal-deployed MedGemma (large-24b) clinical deep-reasoning and report finalization. Governed by NFR-16a with redaction, consent, audit logging, and RAG-Referee validation.
- **Tier 4 (Templates)**: MOH guideline template responses rule-based, deterministic, safety floor when all inference tiers fail.
3. **Micro-Services Layer**: Containerized services orchestrated by K3s (Kubernetes-certified, lightweight distribution designed for edge and resource-constrained production environments). K3s selected over alternatives (Docker Swarm, Nomad, ECS Fargate, Cloud Run) on the following criteria: NFR-16 requires on-premise deployment eliminating ECS Fargate and Cloud Run (cloud-only platforms). Docker Swarm offers lowest PoC operational cost but is in maintenance mode with no viable scaling path; migration to production-grade orchestration requires complete rewrite of all deployment artifacts. Nomad is operationally viable but lacks ecosystem depth and has less documented migration path to multi-cluster federation. K3s is already production-grade at single-site scale, requires no migration to "production" it IS the production platform. The scaling path to N hospitals is multi-cluster federation via K3s cluster-api, not platform replacement. All manifests and service definitions are forward-compatible with upstream Kubernetes.
4. **Data Layer**: On-premise databases (PostgreSQL + pgvector for relational + vector search, Redis for caching, MinIO for S3-compatible object storage).
5. **Embedding Layer**: EmbeddingGemma (768-dim) for RAG retrieval vectors; separate from BERT which is reserved for drift monitoring and RAG-Referee classification only.
6. **Vector Layer (two-tier)**:
- **Hot + Warm: pgvector** (Postgres disk-backed HNSW index, ~15K MOH guideline vectors plus session and case embeddings serves both real-time RAG during consult and non-real-time analytical queries in a single store. NFR-18, NFR-12. NFR-16 compliant.) - **Hot + Warm: pgvector** (Postgres disk-backed HNSW index, ~15K MOH guideline vectors plus session and case embeddings serves both real-time RAG during consult and non-real-time analytical queries in a single store. NFR-18, NFR-12. NFR-16 compliant.)
- **Cold: S3 Vectors** (AWS ap-southeast-1, billion-scale archival, cross-facility bootstrap source, disaster recovery rebuild origin accessed under NFR-16a governance.) - **Cold: S3 Vectors** (AWS ap-southeast-1, billion-scale archival, cross-facility bootstrap source, disaster recovery rebuild origin accessed under NFR-16a governance.)
5. **Presentation Layer**: Progressive Web App (PWA) built with React, Web Workers, and service workers for offline capability and low memory footprint (NFR-4,14). 7. **Presentation Layer**: Progressive Web App (PWA) built with React, Web Workers, and service workers for offline capability and low memory footprint (NFR-4,14).
6. **Observability & Security Layer**: Centralized logging (immutable append-only), monitoring, and audit trails (NFR-10,17). 8. **Observability & Security Layer**: Centralized logging (immutable append-only), monitoring, and audit trails (NFR-10,17).
### 4.2 Data Flow ### 4.2 Data Flow
1. User captures DICOM image via PWA frontend. 1. User captures DICOM image via PWA frontend.
@@ -104,13 +109,12 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
[3] GCP CDN emergency fallback (asia-southeast1 = Singapore, nearest to Vietnam) activated ONLY when hospital intranet CDN is confirmed unreachable. GCP backend bucket (`cdn-backend-models-vkist`) wraps a Cloud Storage bucket (`gs://vkist-models-{site_id}`) with Cloud CDN enabled; signed-URL access control (no `allUsers` public exposure); cache mode `CACHE_ALL_STATIC`, default TTL 24h, max TTL 7d (matches model version lifecycle). Service Worker receives a 307 redirect to signed CDN URL on intranet CDN failure; PWA caches the fetched weights locally via Cache Storage as normal. Because model weights contain zero patient data they are static computational artifacts the GCP CDN fetch event is committed to the on-prem audit log (case_hash, timestamp, consult_mode = "cdn_fallback") but does NOT trigger the full NFR-16a redaction pipeline. The redaction pipeline applies only to Tier 3a inference calls carrying clinical payloads, not to non-clinical model binary fetches. [3] GCP CDN emergency fallback (asia-southeast1 = Singapore, nearest to Vietnam) activated ONLY when hospital intranet CDN is confirmed unreachable. GCP backend bucket (`cdn-backend-models-vkist`) wraps a Cloud Storage bucket (`gs://vkist-models-{site_id}`) with Cloud CDN enabled; signed-URL access control (no `allUsers` public exposure); cache mode `CACHE_ALL_STATIC`, default TTL 24h, max TTL 7d (matches model version lifecycle). Service Worker receives a 307 redirect to signed CDN URL on intranet CDN failure; PWA caches the fetched weights locally via Cache Storage as normal. Because model weights contain zero patient data they are static computational artifacts the GCP CDN fetch event is committed to the on-prem audit log (case_hash, timestamp, consult_mode = "cdn_fallback") but does NOT trigger the full NFR-16a redaction pipeline. The redaction pipeline applies only to Tier 3a inference calls carrying clinical payloads, not to non-clinical model binary fetches.
- **NFR-4 clarification**: NFR-4's 150MB ceiling covers the core application bundle (JS/TS code, UI assets, runtime state) at idle. Supra-150MB memory consumed by models, caches, and inference buffers is bounded separately under the WASM heap cap (1.5GB) and instrumented independently. - **NFR-4 clarification**: NFR-4's 150MB ceiling covers the core application bundle (JS/TS code, UI assets, runtime state) at idle. Supra-150MB memory consumed by models, caches, and inference buffers is bounded separately under the WASM heap cap (1.5GB) and instrumented independently.
- **Inference Fallback Chain** (LLM consult path only CV pipeline is always on-prem Triton): - **Inference Fallback Chain** (LLM consult path only CV pipeline is always on-prem Triton):
- **Tier 0** Browser WebLLM (GemmaE2B local in WASM, instant, zero network preferred default when device has 3GB RAM and WebGPU available). Consult mode: `browser_local`. Triggered automatically on page load if model weights are cached and WebGPU is detected. - **Tier 1** Browser WebLLM (Edge GemmaE2B local in WASM, instant, zero network preferred default when device has 3GB RAM and WebGPU available). Consult mode: `tier_1`. Triggered automatically on page load if model weights are cached and WebGPU is detected. Responsible for primary conversation and orchestrating cloud calls.
- **Tier 1** [REMOVED Triton does NOT host LLM. Triton hosts CV models + EmbeddingGemma only.]
- **Tier 2** On-prem Triton (hospital GPU node CV pipeline ONLY: angle inflammation segmentation + EmbeddingGemma for RAG embeddings). The LLM consult does NOT fall back to Triton. If browser WebLLM is unavailable, the next tier is cloud. - **Tier 2** On-prem Triton (hospital GPU node CV pipeline ONLY: angle inflammation segmentation + EmbeddingGemma for RAG embeddings). The LLM consult does NOT fall back to Triton. If browser WebLLM is unavailable, the next tier is cloud.
- **Tier 3a** GCP Vertex AI (MedGemma via REST API **PoC ONLY under NFR-16a**). Activated when Tier 0 (browser WebLLM) is unavailable (device unsupported, model not cached, memory exceeded) AND user explicitly consents per-session. All payloads MUST pass Decree 13 PII redaction gate (see §5.4) before leaving hospital boundary. GCP project configured with CSEK, 30-day storage lifecycle, 1-year project deletion timer. Consult mode: `cloud_vertex`. - **Tier 2** GCP Vertex AI (Gemini **PoC ONLY under NFR-16a**). Activated when Tier 1 (browser WebLLM) is unavailable AND Edge Gemma orchestrates a cloud call for task types: `orchestration`, `translation`, `ui_planning`, `formatting`. All payloads MUST pass Decree 13 PII redaction gate (see §5.4) before leaving hospital boundary. Consult mode: `tier_2`.
- **Tier 3b** Cloud LLM Arbiter (GCP Vertex AI MedGemma **PoC ONLY under NFR-16a, BERT-triggered correction**). Activated when BERT drift monitor (UC-74821) detects semantic impasse, logical contradiction, or contextual hallucination during Socratic dialogue, OR when radiologist explicitly requests "second opinion." Same Vertex AI endpoint as Tier 3a, but with arbiter system prompt + retrieved MOH chunks injected. ALL calls require prior audit-log commit of drift signal + user consent + Decree 13 redaction. Consult mode: `cloud_vertex_arbiter`. - **Tier 3** Modal MedGemma (clinical deep-reasoning & report finalization **PoC ONLY under NFR-16a**). Activated when RAG confidence < threshold, radiologist explicitly requests "clinical depth", or Edge Gemma detects complex medical reasoning requiring clinical expertise. Same NFR-16a governance as Tier 2. Consult mode: `tier_3`. Output validated by RAG-Referee before reaching radiologist.
- **Tier 3c** MOH guideline template responses (rule-based, no generative model, always available, deterministic, cited safety floor when all inference tiers fail). Consult mode: `templates`. - **Tier 4** MOH guideline template responses (rule-based, no generative model, always available, deterministic, cited safety floor when all inference tiers fail). Consult mode: `tier_4`.
- **Circuit Breaker**: PWA Feature Flag Manager (`consult_mode` state machine) attempts tiers sequentially: `browser_local` `cloud_vertex` `cloud_vertex_arbiter` `templates`. Emits tier-transition events to on-prem immutable audit log (NFR-17). Tier 3a/3b transitions require consent token + redaction manifest committed *before* the HTTP call. CDN fallback events (model weight distribution only) log lighter audit entry: no redaction required, no consent required (weights carry no PII). - **Circuit Breaker**: PWA Feature Flag Manager (`consult_mode` state machine) attempts tiers sequentially: `tier_1` `tier_2` `tier_3` `tier_4`. Emits tier-transition events to on-prem immutable audit log (NFR-17). Tier 2/3 transitions require consent token + redaction manifest committed *before* the HTTP call. CDN fallback events (model weight distribution only) log lighter audit entry: no redaction required, no consent required (weights carry no PII).
- **GCP CDN configuration** (emergency distribution path only): - **GCP CDN configuration** (emergency distribution path only):
- Cloud Storage bucket: `gs://vkist-models-{site_id}` Uniform bucket-level access, no `allUsers` public exposure. - Cloud Storage bucket: `gs://vkist-models-{site_id}` Uniform bucket-level access, no `allUsers` public exposure.
- Signed-URL generation: FastAPI generates time-limited signed URLs (signed by project service account private key, CSEK-managed) with a 15-minute expiry; returned to PWA as a `307` redirect on intranet CDN failure. - Signed-URL generation: FastAPI generates time-limited signed URLs (signed by project service account private key, CSEK-managed) with a 15-minute expiry; returned to PWA as a `307` redirect on intranet CDN failure.
@@ -133,7 +137,7 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
- **JWT session state** (TTL 1h, matches token expiry): validates active sessions without Postgres roundtrip; coalesced look-aside with single-flight mutex to prevent thundering herd on session-miss. - **JWT session state** (TTL 1h, matches token expiry): validates active sessions without Postgres roundtrip; coalesced look-aside with single-flight mutex to prevent thundering herd on session-miss.
- **MOH guideline chunks** (TTL 7d): version-key invalidation triggered by Postgres NOTIFY on guideline ingestion events; cache key = `guideline:{version}:{chunk_id}`, full key-space retired on version bump, no TTL races. - **MOH guideline chunks** (TTL 7d): version-key invalidation triggered by Postgres NOTIFY on guideline ingestion events; cache key = `guideline:{version}:{chunk_id}`, full key-space retired on version bump, no TTL races.
- **DICOM metadata per session** (TTL 12h, per-session scope): avoids repeated Postgres lookups of DICOM headers during the 3-step ML pipeline; evicted after session completion. - **DICOM metadata per session** (TTL 12h, per-session scope): avoids repeated Postgres lookups of DICOM headers during the 3-step ML pipeline; evicted after session completion.
- **Circuit-breaker / consult_mode state** (TTL 2h, matches session): consult_mode enum (tier_1/tier_2/tier_3a/tier_3b) keyed by session_hash. Written atomically on tier transitions. Read by: FastAPI routing layer, PWA (via SSE status push), Prometheus metrics exporter. Justifies Redis over Postgres: sub-ms reads required, writes are frequent (per BERT token stream during Socratic dialogue), data is volatile (expires with session), and multiple FastAPI workers behind NGINX need shared visibility in-process memory cannot cross worker boundaries. - **Circuit-breaker / consult_mode state** (TTL 2h, matches session): consult_mode enum (tier_1/tier_2/tier_3/tier_4) keyed by session_hash. Written atomically on tier transitions. Read by: FastAPI routing layer, PWA (via SSE status push), Prometheus metrics exporter. Justifies Redis over Postgres: sub-ms reads required, writes are frequent (per BERT token stream during Socratic dialogue), data is volatile (expires with session), and multiple FastAPI workers behind NGINX need shared visibility in-process memory cannot cross worker boundaries.
- **Rate-limit counters** (TTL 30s, sliding window): INCR on consult request entry. Self-clearing via EXPIRE. Prevents Triton GPU saturation under concurrent burst from multiple radiologists. Atomic Redis operation; Postgres row-lock alternative creates write amplification on hot path. - **Rate-limit counters** (TTL 30s, sliding window): INCR on consult request entry. Self-clearing via EXPIRE. Prevents Triton GPU saturation under concurrent burst from multiple radiologists. Atomic Redis operation; Postgres row-lock alternative creates write amplification on hot path.
- **Not cached in Redis** (served from primary store): - **Not cached in Redis** (served from primary store):
- Audit log: append-only, NFR-17 immutability requirement no intermediate cache layer. - Audit log: append-only, NFR-17 immutability requirement no intermediate cache layer.
@@ -248,9 +252,9 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
- **Application**: NeMo Guardrails and GuardrailsAI require substantial runtime scaffolding, rule engines, and WASM-compatible bindings that do not exist in the browser ecosystem at the model inference layer. The client-side cannot run a NeMo Rail or GuardrailsAI pipeline inside a WebWorker alongside WebLLM. Therefore, the edge guardrail tier relies on two lightweight mechanisms: - **Application**: NeMo Guardrails and GuardrailsAI require substantial runtime scaffolding, rule engines, and WASM-compatible bindings that do not exist in the browser ecosystem at the model inference layer. The client-side cannot run a NeMo Rail or GuardrailsAI pipeline inside a WebWorker alongside WebLLM. Therefore, the edge guardrail tier relies on two lightweight mechanisms:
1. **Robust prompt-based rules**: Hard system-prompt constraints injected at every consult turn. Boundaries: no diagnosis outside MOH protocol scope; mandatory citation for every clinical assertion; no free-text echoing of PII even if user inputs it; immediate refusal to follow instructions that attempt prompt injection or jailbreak; tone and scope locked to clinical decision-support. 1. **Robust prompt-based rules**: Hard system-prompt constraints injected at every consult turn. Boundaries: no diagnosis outside MOH protocol scope; mandatory citation for every clinical assertion; no free-text echoing of PII even if user inputs it; immediate refusal to follow instructions that attempt prompt injection or jailbreak; tone and scope locked to clinical decision-support.
2. **BERT hallucination / mal-intention detection**: Each candidate LLM token stream (or per-turn summary) is evaluated by a local Transformers.js BERT classifier running in a dedicated `guardrail.worker.ts` WebWorker. The classifier scores for: (a) hallucination probability (factual grounding vs retrieved MOH context); (b) mal-intention / prompt-injection indicators (user-context leaking, instruction override attempts, jailbreak markers); (c) scope violation (claims outside synovitis/MSK MOH guidelines). 2. **BERT hallucination / mal-intention detection**: Each candidate LLM token stream (or per-turn summary) is evaluated by a local Transformers.js BERT classifier running in a dedicated `guardrail.worker.ts` WebWorker. The classifier scores for: (a) hallucination probability (factual grounding vs retrieved MOH context); (b) mal-intention / prompt-injection indicators (user-context leaking, instruction override attempts, jailbreak markers); (c) scope violation (claims outside synovitis/MSK MOH guidelines).
- **Session termination and cloud mitigation**: If BERT edge guardrail returns a violation score above threshold on any axis FastAPI immediately terminates the WebLLM session (`stream.close()`), logs the trigger event to immutable audit, and opens a new mitigated consult session toward Tier 3a (cloud Vertex AI MedGemma). Mitigation path: the same user query + retrieved MOH chunks (already computed) + edge-detected-offense log are forwarded under NFR-16a governance (consent token confirmed, Decree 13 redaction re-applied, audit-log commit before egress). The radiologist sees: "Edge model restricted escalating to cloud consult (redacted)." - **Session termination and cloud mitigation**: If BERT edge guardrail returns a violation score above threshold on any axis FastAPI immediately terminates the WebLLM session (`stream.close()`), logs the trigger event to immutable audit, and routes to the appropriate cloud tier based on `task_type`: orchestration/translation tasks route to Tier 2 (Gemini), clinical deep-reasoning/report finalization routes to Tier 3 (MedGemma). Mitigation path: the same user query + retrieved MOH chunks (already computed) + edge-detected-offense log are forwarded under NFR-16a governance (consent token confirmed, Decree 13 redaction re-applied, audit-log commit before egress). The radiologist sees: "Edge model restricted escalating to cloud consult (redacted)."
- **WebWorker isolation**: `guardrail.worker.ts` runs in a separate WebWorker from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM). No shared WASM memory between workers. IndexedDB caches artifacts but is never injected into LLM context. Unload order on memory pressure: `llm.worker.ts` `guardrail.worker.ts` `cv.worker.ts`. - **WebWorker isolation**: `guardrail.worker.ts` runs in a separate WebWorker from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM). No shared WASM memory between workers. IndexedDB caches artifacts but is never injected into LLM context. Unload order on memory pressure: `llm.worker.ts` `guardrail.worker.ts` `cv.worker.ts`.
- **Cloud model guardrail**: Cloud Vertex AI MedGemma (Tier 3a/3b) uses natively supported Vertex AI Model Garden safety filters (tuned for healthcare) plus the same server-side FastAPI BERT RAG-Referee gate. No additional client-side framework required. - **Cloud model guardrail**: Cloud models use the server-side FastAPI BERT RAG-Referee gate as the primary safety mechanism. Tier 2 (Gemini) uses Vertex AI Model Garden safety filters (tuned for healthcare). Tier 3 (MedGemma) output is additionally validated by RAG-Referee before reaching the radiologist. No additional client-side framework required.
### 5.8 Edge Data Hygiene: Anonymization, Redaction & Server-Side Ground-Check ### 5.8 Edge Data Hygiene: Anonymization, Redaction & Server-Side Ground-Check
- **Pattern**: Edge-to-Server Redaction Pipeline with Ground-Truth Verification - **Pattern**: Edge-to-Server Redaction Pipeline with Ground-Truth Verification
@@ -269,22 +273,23 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
### 5.9 RAG as Essential Pipeline Step (Non-Optional) ### 5.9 RAG as Essential Pipeline Step (Non-Optional)
- **Pattern**: Mandatory RAG Pre-Processing for All LLM Consult Paths - **Pattern**: Mandatory RAG Pre-Processing for All LLM Consult Paths
- **Source**: NFR-18 (100% LLM text cites MOH protocol); NFR-12 (zero-friction explainability) - **Source**: NFR-18 (100% LLM text cites MOH protocol); NFR-12 (zero-friction explainability)
- **Application**: RAG retrieval is NOT an optional tool the LLM may choose to invoke. It is a mandatory pipeline step that executes before any LLM generation, for both edge (browser WebLLM) and cloud (Vertex AI MedGemma) tiers. - **Application**: RAG retrieval is NOT an optional tool the LLM may choose to invoke. It is a mandatory pipeline step that executes before any LLM generation, for all three-tier systems: Edge Gemma (browser WebLLM), Gemini (GCP Vertex AI), and MedGemma (Modal).
- **Pipeline order**: - **Pipeline order**:
1. User query arrives (post-redaction) 1. User query arrives (post-redaction)
2. `rag.query()` retrieves top-k MOH guideline chunks from pgvector HNSW 2. `rag.query()` retrieves top-k MOH guideline chunks from pgvector HNSW
3. Retrieved chunks + strict system prompt ("You are a clinical decision-support assistant. Answer ONLY using the provided MOH guidelines. Cite chunk IDs. Do not speculate.") injected into LLM context 3. Retrieved chunks + strict system prompt ("You are a clinical decision-support assistant. Answer ONLY using the provided MOH guidelines. Cite chunk IDs. Do not speculate.") injected into LLM context
4. LLM generates grounded, cited response 4. LLM generates grounded, cited response
5. `referee.validate()` (BERT RAG-Referee) checks citation correctness, logical cohesion, factual contestant status 5. `referee.validate()` (BERT RAG-Referee) checks citation correctness, logical cohesion, factual contestant status mandatory for MedGemma (Tier 3) output; optional but recommended for Gemini (Tier 2); lightweight validation for Edge Gemma (Tier 1)
6. If any axis fails reject fallback to MOH template 6. If any axis fails reject fallback to MOH template
- **Rationale**: If RAG were merely a tool call, a compromised or under-performing edge LLM could choose to skip retrieval and generate ungrounded text, violating NFR-18. Mandatory RAG pre-processing closes this vector. - **Rationale**: If RAG were merely a tool call, a compromised or under-performing edge LLM could choose to skip retrieval and generate ungrounded text, violating NFR-18. Mandatory RAG pre-processing closes this vector.
### 5.10 Tool-Calling Semantics for Edge and Cloud LLMs ### 5.10 Tool-Calling Semantics for Edge and Cloud LLMs
- **Pattern**: Function-Calling as Convenience Layer Over Mandatory RAG - **Pattern**: Function-Calling as Convenience Layer Over Mandatory RAG
- **Source**: Gemma Functions (Google DeepMind) + Vertex AI Function Calling - **Source**: Gemma Functions (Google DeepMind) + Vertex AI Function Calling + Modal MedGemma Function Calling
- **Application**: - **Application**:
- **Edge LLM (GemmaE2B via WebLLM)**: Supports Gemma Functions. The RAG retrieval is exposed as a function tool (`retrieve_moh_guidelines(query, top_k)`). HOWEVER, this tool is pre-invoked by the host pipeline per §5.9 before the first LLM token is generated. The function-calling interface is available for subsequent turns of Socratic dialogue where the model may need additional retrieval mid-conversation (e.g., user asks "what about contraindications?" mid-turn retrieval). The host pipeline enforces the mandatory RAG call on turn 0; subsequent per-turn retrieval is model-initiated via function calling. - **Edge LLM (Edge Gemma via WebLLM)**: Supports Gemma Functions. The RAG retrieval is exposed as a function tool (`retrieve_moh_guidelines(query, top_k)`). HOWEVER, this tool is pre-invoked by the host pipeline per §5.9 before the first LLM token is generated. The function-calling interface is available for subsequent turns of Socratic dialogue where the model may need additional retrieval mid-conversation (e.g., user asks "what about contraindications?" mid-turn retrieval). The host pipeline enforces the mandatory RAG call on turn 0; subsequent per-turn retrieval is model-initiated via function calling.
- **Cloud LLM (MedGemma on Vertex AI)**: Native Vertex AI Function Calling is enabled. Same semantics: server-side FastAPI injects top-k MOH chunks before first generation; model may request further retrieval via function call during multi-turn consult. - **Cloud LLM Tier 2 (Gemini on Vertex AI)**: Native Vertex AI Function Calling is enabled for orchestration/translation tasks. Same semantics: server-side FastAPI injects top-k MOH chunks before first generation; model may request further retrieval via function call during multi-turn consult.
- **Cloud LLM Tier 3 (MedGemma on Modal)**: Native function-calling interface for clinical reasoning tasks. Server-side FastAPI injects top-k MOH chunks and RAG-Referee validation results before first generation; model may request further retrieval via function call during multi-turn clinical consult.
- **Why not pure tool-calling**: Pure tool-calling delegates the decision to retrieve to the model. NFR-18 requires 100% citation; delegating retrieval to model choice creates a non-zero probability of unsupported generation. Mandatory pre-injection + optional per-turn tool call is the safer contract. - **Why not pure tool-calling**: Pure tool-calling delegates the decision to retrieve to the model. NFR-18 requires 100% citation; delegating retrieval to model choice creates a non-zero probability of unsupported generation. Mandatory pre-injection + optional per-turn tool call is the safer contract.
### 5.11 Output Filtering ### 5.11 Output Filtering
@@ -304,6 +309,11 @@ The system adopts a hybrid edge-cloud (on-premise) architecture with the followi
- `edge_guardrail_violation`: session_hash, violation_axis (hallucination|mal_intention|scope_breach), bert_score, user_query_hash, mitigation_target (cloud|template). - `edge_guardrail_violation`: session_hash, violation_axis (hallucination|mal_intention|scope_breach), bert_score, user_query_hash, mitigation_target (cloud|template).
- `rag_retrieval_event`: session_hash, query_hash, retrieved_chunk_ids, latency_ms. - `rag_retrieval_event`: session_hash, query_hash, retrieved_chunk_ids, latency_ms.
- `referee_decision`: session_hash, axis_results (attribution|cohesion|contestant), overall_pass, fallback_triggered. - `referee_decision`: session_hash, axis_results (attribution|cohesion|contestant), overall_pass, fallback_triggered.
- `egress_consent_gemini`: session_hash, user_id, task_type, redaction_hash before Gemini cloud call.
- `egress_consent_medgemma`: session_hash, user_id, task_type, redaction_hash before MedGemma cloud call.
- `egress_response_gemini`: session_hash, task_type, status, ts after Gemini response.
- `egress_response_medgemma`: session_hash, task_type, status, ts after MedGemma response.
- `cloud_llm_escalation`: session_hash, from_tier, to_tier, reason (reroute, referee_fail, token_limit) tracks GeminiMedGemma or any cross-tier escalation.
- **Audit events for this layer**: `redaction_edge_manifest` (client), `redaction_server_refine` (server-side cleaning pass), `redaction_failure` (server unable to clean error), `egress_consent` + `egress_redact_manifest` (before cloud call, NFR-16a). - **Audit events for this layer**: `redaction_edge_manifest` (client), `redaction_server_refine` (server-side cleaning pass), `redaction_failure` (server unable to clean error), `egress_consent` + `egress_redact_manifest` (before cloud call, NFR-16a).
- **Retention**: Audit events retained indefinitely (append-only, no UPDATE/DELETE). Raw LLM token streams retained for 90 days in MinIO (compressed, AES-256 at rest) for drift analysis. After 90 days, streams are deleted; only aggregated `referee_decision` and `rag_retrieval_event` summaries remain. - **Retention**: Audit events retained indefinitely (append-only, no UPDATE/DELETE). Raw LLM token streams retained for 90 days in MinIO (compressed, AES-256 at rest) for drift analysis. After 90 days, streams are deleted; only aggregated `referee_decision` and `rag_retrieval_event` summaries remain.
- **BERT drift monitor telemetry**: edge BERT classifier score distributions exported to Prometheus; alert if false-positive rate on hallucination detection exceeds 5% or false-negative rate exceeds 0.1% (clinical safety threshold). - **BERT drift monitor telemetry**: edge BERT classifier score distributions exported to Prometheus; alert if false-positive rate on hallucination detection exceeds 5% or false-negative rate exceeds 0.1% (clinical safety threshold).

View File

@@ -0,0 +1,110 @@
# DEMO_EXP — Functional Requirements (Demo-FRs)
> **Status:** Draft — Demo & Testing Only
> **Date:** June 25, 2026
> **Engineer:** Đạt Trần Tiến (Daves Tran)
> **Scope:** These Demo-FRs are experimental add-ons layered onto the existing VKIST FR architecture. They are **not** production FRs and shall be tracked separately from `FR_Engineer_DB_Mobile.csv`.
---
## Demo-FR-01: "Morning Cohort Ritual" — Synchronized Patient List Entry
| Field | Value |
| :--- | :--- |
| **Demo-FR ID** | Demo-FR-01 |
| **Title** | Morning Cohort Ritual — Synchronized Patient List Entry |
| **Platform** | Mobile Web (PWA) |
| **Component** | Patient Responsibility Dashboard (extends FR-26) |
| **User Profile** | UP5 (Diagnostic Radiologist) |
| **Interaction** | User-to-System |
| **Base FR** | FR-26 (`UC-48377`: View Patient List with Clinical Summary) |
| **Sprint** | Sprint 3 (The Collaborative Workspace) |
| **Priority** | Demo / Proof-of-Concept |
| **Precondition** | User has valid authentication and FR-26 patient roster is loaded. |
| **Postcondition** | Each patient card in the roster displays a context strip showing: (1) the last collective action taken on the case (with actor + timestamp), (2) which other roles are currently active on this case, and (3) any pending action required from the radiologist. No extra navigation is required. |
| **Trigger** | When the user opens the Patient Responsibility Dashboard (`UC-48377`), the system enriches each patient card entry with synchronized context data drawn from EMR sign-off logs, scan session state, and PT scheduling records. |
| **Stimulus** | User taps "My Patients" / "View Patient List" button on the Mobile Web PWA dashboard. |
| **System Response** | System retrieves the authenticated user's patient roster AND enriches each entry with: (a) last-signal metadata (who acted, when, what action), (b) active-role indicators (Surgeon? PT? both?), (c) pending-action flags (e.g., "Awaiting your grading on Scan #7"). Renders enriched cards in-place. |
| **NFR Constraints** | Zero extra clicks — context strip rendered inline on existing cards. All data passes through Decree 13 scrub layer. Compatible with zero-GPU fallback rendering. |
**Motivation:**
Transforms a sterile data lookup into a collective handoff moment. The radiologist opens the list and instantly understands *where they fit in the care chain* — who else is working on this case, what was the last decision, and what is pending their action.
---
## Demo-FR-02: "Agreement Moment" — Sign-Off Cascade
| Field | Value |
| :--- | :--- |
| **Demo-FR ID** | Demo-FR-02 |
| **Title** | Agreement Moment — Synchronized Sign-Off Cascade |
| **Platform** | Mobile Web (PWA) |
| **Component** | Report Finalization Layer (extends FR-25) |
| **User Profile** | UP5 (Diagnostic Radiologist), Surgeon, PT, Patient/Caregiver |
| **Interaction** | User-to-System (multi-role) |
| **Base FR** | FR-25 / `UC-92006` (Finalize & Sign Electronic Record) |
| **Sprint** | Sprint 3 (The Collaborative Workspace) |
| **Priority** | Demo / Proof-of-Concept |
| **Precondition** | Radiologist has completed grading review and is ready to sign the diagnostic report. All 4 roles have active accounts in the system. |
| **Postcondition** | Upon radiologist's cryptographic seal: (1) Surgeon's dashboard receives a real-time pulse notification with finalized report summary, (2) PT's read-only view is updated with the new PT protocol card, (3) Patient/Caregiver portal receives a plain-language notification of the result, (4) EMR log entry is created. All 3 downstream notifications fire within the same synchronization window. |
| **Trigger** | Radiologist taps "Finalize & Sign" on the diagnostic report screen. |
| **Stimulus** | User (UP5) confirms sign-off action. System displays a single confirmation: "This will notify the care team and patient. Proceed?" |
| **System Response** | Upon confirmation: (1) cryptographic seal applied, (2) EMR log entry created (`UC-02423`), (3) Surgeon dashboard pulse notification dispatched via existing event bus, (4) PT protocol card delivered to PT workspace via existing FR-30 send pipeline, (5) plain-language patient notification dispatched via existing FR-19 push infrastructure. All fires within <= 2 seconds of sign-off. |
| **NFR Constraints** | Zero extra workflow steps — sign-off is a single tap. Notification infrastructure reuses existing FR-19/FR-30 pipelines. All data passes through Decree 13 scrub. No external APIs — all sync over local K3s cluster. |
**Motivation:**
Elevates the act of signing from a bureaucratic checkbox into a *synchronized collective event*. The radiologist's single action of clinical judgment becomes the trigger that unlocks the next phase of care for every other participant. The radiologist *feels* that their signature has weight and consequence.
---
## Demo-FR-03: "Daily Progress Ritual" — Synchronized PTPatient Journal
| Field | Value |
| :--- | :--- |
| **Demo-FR ID** | Demo-FR-03 |
| **Title** | Daily Progress Ritual — Synchronized PTPatient Journal |
| **Platform** | Mobile Web (PWA) |
| **Component** | Patient Education / Care Logic Module (extends FR-28, FR-29) |
| **User Profile** | UP8 (MSK Patient & Family Caregiver), UP6 (Physical Therapist) |
| **Interaction** | User-to-System (dual-user, time-shifted) |
| **Base FR** | FR-28 (CREATE patient treatment journal), FR-29 (UPDATE patient treatment journal) |
| **Sprint** | Sprint 4 (Patient-Facing PWA) |
| **Priority** | Demo / Proof-of-Concept |
| **Precondition** | Patient treatment journal has been initialized and is within the active treatment monitoring period. |
| **Postcondition** | Patient journal entry is saved with timestamp and trend indicator. PT dashboard for that patient is updated with a soft indicator showing the latest entry timestamp, pain level delta, and trend direction. Patient receives positive visual feedback upon submission. |
| **Trigger** | Patient submits a journal update (pain slider adjustment, symptom checkbox, or free-text entry). |
| **Stimulus** | Patient interacts with the journal entry UI on their mobile device. |
| **System Response** | (1) Journal entry is saved with timestamp and anonymized patient ID, (2) Decree 13 scrub layer validates no PII in entry, (3) PT dashboard for that patient is updated with a soft indicator: latest entry timestamp, pain level delta (e.g., "from 7 to 3"), trend arrow, (4) Patient sees positive ritual feedback (e.g., progress arc animation, culturally appropriate encouragement text). |
| **NFR Constraints** | Journal data already scrubbed before PT view (Decree 13). Ritual UI overlay is a lightweight animation layer on existing journal UI — no new screens. Zero-GPU compatible. No open-ended communication loops — data flows one-way (patient -> PT), gated by existing FR-30 send pipeline. |
**Motivation:**
Transforms a repetitive, lonely self-reporting task into a *shared daily ritual*. The patient feels *seen* — their effort is acknowledged, not just logged. The PT enters the session already connected to the patient's lived experience, turning a generic protocol execution into a continuation of a shared story of progress.
---
## Demo-FR-04: "Journey Mirror" — Longitudinal Synchronized Progress View
| Field | Value |
| :--- | :--- |
| **Demo-FR ID** | Demo-FR-04 |
| **Title** | Journey Mirror — Longitudinal Synchronized Progress View |
| **Platform** | Mobile Web (PWA) |
| **Component** | Patient Portal / History Module (extends FR-16, FR-31, FR-12) |
| **User Profile** | UP8 (MSK Patient & Family Caregiver), UP5 (Diagnostic Radiologist), UP7 (Surgeon/Orthopedic) |
| **Interaction** | User-to-System (dual-lens, same data) |
| **Base FR** | FR-16 (TRA CUU lich su kham benh), FR-31 (INTERPRET diagnostic report from Clinic to Patient), FR-12 (SYNCHRONIZE hardware-adaptive musculoskeletal models) |
| **Sprint** | Sprint 4-5 (Patient-Facing PWA -> Feedback Pipeline & Hardening) |
| **Priority** | Demo / Proof-of-Concept |
| **Precondition** | Patient has >= 2 historical scan sessions in the system. FR-16 history data is accessible. FR-31 plain-language interpretation layer is active. |
| **Postcondition** | Both clinician and patient can view the same longitudinal timeline: (1) Clinician lens shows scan dates, AI grades, GradCAM overlays, EMR sign-off timestamps, surgical decisions, (2) Patient lens shows the same timeline nodes with 3D visit thumbnails and plain-language captions. Timeline updates are synchronized — when a clinician adds a new scan, the patient's mirror view reflects it. |
| **Trigger** | User (either role) opens the "My Journey" / "Patient Journey" timeline view. |
| **Stimulus** | User navigates to the longitudinal history view from their dashboard. |
| **System Response** | (1) System retrieves full patient visit history from FR-16 history store, (2) Renders timeline nodes: each node populated from scan session data (FR-25/UC-48376), signed report metadata (FR-25/UC-92006), and plain-language interpretation (FR-31), (3) Clinician view: renders with AI grades, GradCAM overlay thumbnails, EMR timestamps, (4) Patient view: renders with zero-GPU 3D sprite-sheet thumbnails (FR-12) and plain-language captions, (5) Both views reference the same timeline data — synchronized by patient ID and visit date. |
| **NFR Constraints** | Zero-GPU fallback renders sprite-sheet thumbnails on legacy devices (FR-12). Decree 13: clinician view shows anonymized ID; patient view shows only their own record. All rendering over local K3s cluster, no external CDN. |
**Motivation:**
Creates a **shared narrative** of the patient's healing journey. Both the clinician and the patient are telling the *same story* — the clinician through clinical precision, the patient through plain language and visual empathy. The timeline becomes a ritual object that both parties return to, reinforcing trust and shared understanding across the entire care arc.
---
*End of Functional Requirements — Demo-FRs (4 total)*

View File

@@ -0,0 +1,100 @@
# DEMO_EXP — Motivation: "Synchronized Ritual" Concept
> **Status:** Draft — Concept Note for Demo Session
> **Date:** June 25, 2026
> **Engineer:** Đạt Trần Tiến (Daves Tran)
> **Context:** VKIST-PILOT MSK Pilot Workspace (Sprint 1-6, June 2 - September 2, 2026)
---
## 1. The UX Concept: What Is a "Synchronized Ritual"?
In UX and digital design, a **Synchronized Ritual** is the intentional creation of shared digital experiences or synchronous interactions that connect users **across time and space**. It takes a basic, habitual app action and elevates it into an emotionally resonant, collective, or deeply personalized experience.
A ritual is not just a repeated action. A *synchronized* ritual is one where:
- **Multiple users** feel connected to the **same event** at the same moment, even if they are physically apart
- **Past, present, and future** are visible in a single view — the user feels the *arc* of what they are part of
- A **basic action** (open a list, tap a button) carries **emotional weight** beyond its functional outcome
---
## 2. Why This Concept Matters for VKIST Specifically
PILOT's user research reveals **three structural loneliness problems** that Synchronized Ritual directly addresses:
### Problem 1: The Isolated Radiologist
Radiologists work in **dark, quiet reading rooms**, detached from direct patient contact and from the downstream clinicians who consume their reports. Their work is invisible until a Surgeon or PT reads it — sometimes hours or days later.
> "They are the vital gatekeepers of clinical safety. Engagement and UX research should be focused entirely on invisible workflow optimization." — User Research Result, UP5 profile
**Synchronized Ritual response:** Create moments where the radiologist's action *immediately ripples outward*. When they sign a report, they see — in real time — that the Surgeon's dashboard has pulsed, the PT's session has updated, and the patient has been notified. The radiologist *feels* their work is part of a collective machine.
### Problem 2: The Anxious, Information-Starved Patient
Patients and their caregivers are **anxious and overwhelmed**, often turning to dangerous folk remedies (leaf-wrapping, aggressive manual adjustment) because no one has translated their scan into understandable terms. The hospital consultation is brief; the patient leaves confused.
> "If the AI is expertly presented as a high-tech, highly objective authority that visually validates their human doctor's hurried diagnosis, it can significantly and immediately increase institutional trust." — User Research Result, UP8 profile
**Synchronized Ritual response:** Transform the AI-generated report into a **shared artifact** — the doctor's clinical precision and the patient's emotional understanding, synchronized in one view. The patient sees their own healing arc as a narrative of progress, not a stack of confusing documents.
### Problem 3: The Information-Siloed PT
Physical Therapists receive only **brief text prescriptions** — they lack visibility into the radiologist's scan findings, the surgeon's protocol, or the patient's subjective experience. They execute treatment in semi-blindness.
> "Vietnamese Physiotherapists struggle to accurately target internal tissue pathologies during therapy because doctors rarely share full digital DICOM imaging data down the chain, providing only brief text prescription sheets, leading to semi-blind therapeutic execution." — User Research Result, UP6 profile
**Synchronized Ritual response:** Synchronize the PT's treatment session with the radiologist's scan finding and the surgeon's protocol. The PT enters the session already connected to the full clinical chain — not just a one-page prescription.
---
## 3. The Four Demo-FRs: How Ritual Maps to Existing FR Scope
Each Demo-FR is an **add-on layer** on top of existing Functional Requirements. No new backend pipelines are required — the rituals reuse data and infrastructure that already exists or is already planned.
| Demo-FR | Ritual Moment | What Becomes Shared | Base FR(s) |
| :--- | :--- | :--- | :--- |
| **01: Morning Cohort Ritual** | Opening the patient list | The *collective context* of each case — who else is working on it, what was last decided | FR-26, FR-25/UC-92006 |
| **02: Agreement Moment** | Signing a diagnostic report | The *act of clinical judgment* ripples to Surgeon, PT, Patient simultaneously | FR-25/UC-92006, FR-19, FR-30 |
| **03: Daily Progress Ritual** | Patient logging symptoms | The *daily narrative* of recovery, shared between patient and PT before the session begins | FR-28, FR-29, FR-30, FR-20 |
| **04: Journey Mirror** | Viewing the patient's history | The *longitudinal arc* of healing, seen through two synchronized lenses (clinical + patient) | FR-16, FR-31, FR-12 |
---
## 4. The Engineering Rationale: Why This Is Not "Just UX Polish"
The Synchronized Ritual concept is grounded in PILOT's **hard NFRs and constraints**, not in abstract UX theory:
| PILOT NFR | How Demo-FRs Respect It |
| :--- | :--- |
| **Zero Workflow Friction** — no extra clicks, no extra screens | All rituals are embedded in existing actions: opening a list, signing a report, logging a symptom. The ritual is the *enriched rendering* of data that already exists. |
| **Decree 13 / Data Privacy** — strict scrubbing | Ritual data (timestamps, actor names, status chips) is already available in scrubbed form in EMR logs and scan session metadata. No new PII is introduced. |
| **Hardware Heterogeneity** — low-spec phones, old GPUs | Ritual UI uses existing rendering pipelines: patient cards, status chips, sprite-sheet thumbnails. Zero-GPU fallback is already required by FR-12. |
| **Air-Gapped Infrastructure** — no cloud, no external APIs | All synchronization happens over the local K3s cluster / FastAPI backend. The "sync" in Synchronized Ritual is *local data propagation*, not cloud-based real-time networking. |
| **Sprint 6 Deadline** — only 6 sprints | Demo-FRs are achieved by **recomposing existing data** (EMR logs, scan sessions, journal entries) into new UI patterns — not by building new data pipelines or ML models. |
---
## 5. The Stakeholder Narrative: How to Present This
The Synchronized Ritual Demo is not a feature demo. It is a **values demo**.
> "We are not showing you what the system does. We are showing you how it makes every person in this care chain feel connected to each other's work. That is the difference between a tool that processes data and a platform that honors the clinical relationship."
**The demo opens with Demo-FR-02 (Agreement Moment)** — it has the highest emotional impact: one tap, four lives informed simultaneously. Then peel back to show how each layer (01, 03, 04) builds on the same principle.
**The closing argument:**
> "All of this runs locally at PILOT. No data leaves the hospital. No extra clicks were added to anyone's workflow. And yet — the radiologist who signs a report at 14:30 knows that at 14:30, a surgeon in the OR, a PT in the treatment room, and a patient at home all received a piece of that same moment. That is what we mean by Synchronized Ritual."
---
## 6. Risks & Guardrails
| Risk | Mitigation |
| :--- | :--- |
| Ritual UI perceived as "extra work" by busy clinicians | All rituals are **implicit** — they appear in existing views (list cards, sign-off confirmation, journal screen). No new buttons or new screens. |
| Patient-facing rituals trigger anxiety (e.g., "pain went up" notification) | Ritual feedback uses **positive framing** — progress arcs, encouragement text, trend arrows. No alarmist language. |
| Synchronized data introduces a privacy concern | All ritual data is already present in EMR logs and scan sessions. The ritual layer only *re-frames* existing data — it does not aggregate new data sources. |
| Demo feels "theatrical" rather than substantive | Demo scripts use **realistic synthetic data** (e.g., "Case #VN-2048", "Dr. Nguyen", "pain 7->3") and avoid cartoonish UI flourishes. The emotional weight comes from the *logic of connection*, not decorative animation. |
| Clinician rejection of "patient-facing" features | Demo-FR-02 and Demo-FR-04 are presented from the **clinician's perspective first** — the ritual is about *their* sense of impact, not about "making patients happy." |
---
*End of Motivation — Synchronized Ritual Concept for PILOT Demo Session*

View File

@@ -0,0 +1,167 @@
# DEMO_EXP — Use Cases (Demo UCs)
> **Status:** Draft — Demo & Testing Only
> **Date:** June 25, 2026
> **Engineer:** Đạt Trần Tiến (Daves Tran)
> **Scope:** These Use Cases describe the Demo-FR interactions for the "Synchronized Ritual" experiments. They are **not** production UCs and shall be tracked separately from `FR_25_UC_SPEC.md`.
---
## UC-48377: View Patient List with Clinical Summary
| Field | Value |
| :--- | :--- |
| **UC-ID** | UC-48377 |
| **Title [Verb + Noun]** | View Patient List with Clinical Summary |
| **Actor** | UP5 (Diagnostic Radiologist) |
| **System Actor** | System (Backend) |
| **Date Added** | June 25, 2026 |
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
| **Interaction** | User-to-System |
| **Platform** | Mobile Web (PWA) |
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — context strip), Demo-FR-02 (Agreement Moment — sign-off cascade trigger), Demo-FR-04 (Journey Mirror — longitudinal timeline entry point) |
**Goal:**
Access a structured list of patients under the user's clinical responsibility to monitor active diagnostic cases and treatment tracks.
**Preconditions:**
- User has valid authentication credentials and active session with the Mobile Web PWA.
- User has at least one patient case assigned under their clinical responsibility scope.
**Postconditions (Success State):**
- Patient roster list is fully rendered in the mobile viewport.
- Each list entry contains anonymized patient ID, scan type, diagnostic status, prescription summary, and workflow stage.
- *(Demo extension)* Each entry also shows synchronized context: last action actor + timestamp, active-role indicators, and pending-action flags.
**Stimulus:**
The user clicks the "View Patient List" button in the Mobile Web PWA dashboard.
**System Response:**
The system retrieves the authenticated user's patient roster and displays a scrollable list, each entry showing patient anonymized ID, most recent scan type, current diagnostic status (e.g., AI-graded, pending review, finalized), prescription summary, and active workflow stage.
*(Demo extension)* The system enriches each card with synchronized context data drawn from EMR sign-off logs, scan session state, and PT scheduling records.
**Verbose Form:**
The use case 'View Patient List with Clinical Summary' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to access a structured list of patients under the user's clinical responsibility to monitor active diagnostic cases and treatment tracks. This workflow is triggered when the user clicks the "View Patient List" button in the Mobile Web PWA dashboard, causing the system to respond by providing the system retrieves the authenticated user's patient roster and displays a scrollable list, each entry showing patient anonymized ID, most recent scan type, current diagnostic status, prescription summary, and active workflow stage.
**Main Success Scenario (Happy Path):**
1. Diagnostic Radiologist taps the "My Patients" navigation chip on the PWA dashboard.
2. System validates the active JWT token and queries the backend patient roster endpoint.
3. System retrieves the authenticated user's assigned patient list with associated clinical metadata (scan type, AI grade status, prescription summary, workflow stage).
4. System renders a scrollable, paginated list on the mobile viewport, with each card showing anonymized patient ID, most recent scan modality, current diagnostic status tag, and active treatment workflow indicator.
5. *(Demo extension)* System enriches each card entry with synchronized context strip: last action metadata, active-role indicators, and pending-action flags.
6. System enables pull-to-refresh to reload the roster without full page reload.
**Alternative & Exception Flows:**
- **Exception A: Empty Patient Roster** — If the authenticated user has no assigned patients, the system displays a placeholder state with a "No patients assigned" message and a contact administrator prompt.
- **Exception B: Network Timeout / Session Expiry** — If the backend request times out or the session token is invalid, the system displays a non-blocking connection error banner with a retry action.
**Demo Scenario:**
> "The radiologist opens the app on their mobile device. Instead of a flat list of anonymized IDs, each patient card now shows a thin contextual bar: 'Case #VN-2048 — Surgeon Dr. Nguyen reviewed protocol 14:22 today | PT session scheduled 16:00 | Awaiting your grading on Scan #7'. The radiologist immediately understands where they fit in the collective care chain, without clicking into anything."
---
## UC-48378: Filter Patient List by Clinical Status
| Field | Value |
| :--- | :--- |
| **UC-ID** | UC-48378 |
| **Title [Verb + Noun]** | Filter Patient List by Clinical Status |
| **Actor** | UP5 (Diagnostic Radiologist) |
| **System Actor** | System (Backend) |
| **Date Added** | June 25, 2026 |
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
| **Interaction** | User-to-System |
| **Platform** | Mobile Web (PWA) |
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — context strip preserved across filters), Demo-FR-02 (Agreement Moment — filtering before sign-off) |
**Goal:**
Narrow the patient roster to a specific diagnostic or treatment sub-population to reduce cognitive load during high-volume clinical sessions.
**Preconditions:**
- Full patient roster is loaded and displayed in the PWA viewport (`UC-48377`).
- At least one filter dimension is available (status, scan type, date range).
**Postconditions (Success State):**
- List viewport is updated to show only matching patient entries.
- Active filter chips are rendered to indicate current constraint state.
- Context strip from `UC-48377` is preserved on all filtered entries.
**Stimulus:**
The user applies a filter (by status, scan type, or date range) to the currently displayed patient list.
**System Response:**
The system immediately updates the list to show only patients matching the selected criteria, with visible filter chips indicating active constraints. Context strips remain intact on all visible entries.
**Verbose Form:**
The use case 'Filter Patient List by Clinical Status' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to narrow the patient roster to a specific diagnostic or treatment sub-population to reduce cognitive load during high-volume clinical sessions. This workflow is triggered when the user applies a filter (by status, scan type, or date range) to the currently displayed patient list, causing the system to respond by providing the system immediately updates the list to show only patients matching the selected criteria, with visible filter chips indicating active constraints.
**Main Success Scenario (Happy Path):**
1. Diagnostic Radiologist taps a filter chip or dropdown control above the patient list (e.g., "Pending Review", "AI-Graded", "Sup-Long", "Last 7 Days").
2. System captures the selected filter parameters and re-queries the patient roster endpoint with applied constraints.
3. System updates the rendered list in-place, preserving the scroll position and context strips on all matching entries.
4. System displays active filter chips above the list to provide clear visual feedback of current constraints.
5. System enables the user to stack additional filters or clear all filters with a single "Reset" action.
**Alternative & Exception Flows:**
- **Exception A: No Results After Filter** — If the applied filter yields zero matching patients, the system displays a "No patients match current filters" empty state with a direct "Clear Filters" action button.
- **Exception B: Partial Filter Failure** — If one of multiple selected filters fails to apply (e.g., server timeout on a date-range constraint), the system retains the last successfully filtered list state and highlights the failed filter chip in an error state.
---
## UC-48379: Navigate to Patient Detail Record
| Field | Value |
| :--- | :--- |
| **UC-ID** | UC-48379 |
| **Title [Verb + Noun]** | Navigate to Patient Detail Record |
| **Actor** | UP5 (Diagnostic Radiologist) |
| **System Actor** | System (Backend) |
| **Date Added** | June 25, 2026 |
| **Engineer** | Đạt Trần Tiến (Daves Tran) |
| **Interaction** | User-to-System |
| **Platform** | Mobile Web (PWA) |
| **FR Link** | FR-26 (VIEW PATIENT LIST WITH RESPONSIBILITY DETAILS) |
| **Demo-FR Extensions** | Demo-FR-01 (Morning Cohort Ritual — navigation from enriched list), Demo-FR-02 (Agreement Moment — detail view before sign-off), Demo-FR-04 (Journey Mirror — detail view as timeline entry point) |
**Goal:**
Drill into a specific patient's consolidated clinical workspace to review full diagnostic history, AI-generated overlays, and treatment context before rendering a final judgment.
**Preconditions:**
- Patient roster list is rendered in the PWA viewport (`UC-48377`).
- Patient record contains at least one diagnostic session entry accessible by the authenticated user.
**Postconditions (Success State):**
- Full patient workspace is loaded, showing consolidated diagnostic history, AI overlays, prescription summaries, and active treatment protocol notes.
- All displayed data is Decree 13-scrubbed (no raw PII visible to the user).
- *(Demo extension)* Longitudinal timeline entry point is accessible from the detail view, enabling navigation to Demo-FR-04 Journey Mirror.
**Stimulus:**
The user taps a specific patient entry from the list (filtered or unfiltered).
**System Response:**
The system navigates to the patient's full clinical workspace, displaying longitudinal diagnostic history, AI-segmented anatomical overlays with GradCAM heatmaps, prescription and diagnosis summaries, and current treatment protocol notes.
**Verbose Form:**
The use case 'Navigate to Patient Detail Record' defines a User-to-System interaction where the Diagnostic Radiologist (Rad) aims to drill into a specific patient's consolidated clinical workspace to review full diagnostic history, AI-generated overlays, and treatment context before rendering a final judgment. This workflow is triggered when the user taps a specific patient entry from the list (filtered or unfiltered), causing the system to respond by providing the system navigates to the patient's full clinical workspace, displaying longitudinal diagnostic history, AI-segmented anatomical overlays with GradCAM heatmaps, prescription and diagnosis summaries, and current treatment protocol notes.
**Main Success Scenario (Happy Path):**
1. Diagnostic Radiologist taps a patient card entry from the displayed roster list.
2. System sends a detail-record fetch request to the backend with the patient session identifier.
3. System retrieves longitudinal diagnostic history, including prior scan metadata, AI-generated segmentation overlays, GradCAM heatmaps, prescription summaries, and active treatment protocol notes.
4. System validates that all data has been through the Decree 13 scrubbing layer before rendering.
5. System renders the consolidated patient workspace, displaying a timeline of prior scans, the most recent AI-graded overlays, and current prescription and diagnosis summaries in a single rapid-consumption view.
6. *(Demo extension)* System enables navigation to the Journey Mirror timeline view (Demo-FR-04) from within the patient workspace.
7. System enables the radiologist to tap any historical scan entry to load the full diagnostic workspace (`UC-48376`).
**Alternative & Exception Flows:**
- **Exception A: Patient Record Locked or Restricted** — If the patient record is flagged with an access restriction (e.g., different responsible clinician), the system displays an "Access Restricted" dialog with a request access action rather than exposing partial data.
- **Exception B: Incomplete Scrubbing Detected** — If the scrubbing validation layer detects residual PII tokens in the payload, the system halts rendering and triggers a server-side re-scrubbing cycle before presenting the record.
**Demo Scenario:**
> "The radiologist taps a patient card. The system loads the full workspace: a timeline of past scans on the left, the most recent AI overlay in the center, prescription summary below. The radiologist reviews the GradCAM heatmap, sees the Surgeon's treatment protocol note from 14:22 today, and is ready to finalize — all without leaving the mobile viewport."
---
*End of Use Cases — Demo UCs (3 total)*

View File

@@ -0,0 +1,654 @@
# VKIST MSK Ultrasound Stack — Investment Proposal & System Specification
> **Audience:** CTO / Department Head / Executive Sponsor (technical-fluent, outcome-driven, risk-averse on healthcare investments)
> **Author:** VKIST MSK Pilot Engineering Team
> **Status:** Pilot proposal — active cycle June 2 September 2, 2026 (strict 3-month window)
> **Reading time:** Section 0 (Executive Summary) is a self-contained one-pager. Sections 18 provide the full technical and financial justification.
---
## Table of Contents
0. [Executive Summary (one page)](#0-executive-summary-one-page)
1. [Project Context — Who, What Pain, What Constraints](#1-project-context)
2. [How the System Addresses the Problem](#2-how-the-system-addresses-the-problem)
3. [Qualitative ROI — Behavioral & Workflow Change](#3-qualitative-roi--behavioral--workflow-change)
4. [Quantitative ROI — Projected Metrics & Measurement Plan](#4-quantitative-roi--projected-metrics--measurement-plan)
5. [Risk & Failure-Mode Handling — Why This Is Not Risky](#5-risk--failure-mode-handling--why-this-is-not-risky)
6. [Compliance & Data Governance](#6-compliance--data-governance)
7. [Roadmap & Milestones](#7-roadmap--milestones)
8. [Investment Ask & Scope Boundaries](#8-investment-ask--scope-boundaries)
---
## 0. Executive Summary (one page)
**The problem, in one sentence.** Vietnamese public-hospital musculoskeletal (MSK) care runs on clinicians processing 100+ evaluations per shift with no time for objective double-checks, no shared visibility across the radiologist → surgeon → physiotherapist → patient chain, and patients so information-starved that they turn to dangerous folk remedies — while VKIST's proven computer-vision research models sit idle as static academic checkpoints.
**What we are building.** The **VKIST MSK Ultrasound Stack** is an **air-gapped-first, single-hospital AI platform** that turns knee ultrasound scans into automated, explainable **synovitis grading (03)** with Vietnamese-language reasoning, every clinical statement **cited back to Ministry of Health (MOH) guidelines**, and a **human clinician signature required** before anything is finalized. It migrates VKIST's existing DeepLabv3/UNet/ConvNeXt vision models from research into a referenceable, production-grade medical software ecosystem.
**Why it is a sound investment (the four pillars):**
| Pillar | What it means for the leader |
| :--- | :--- |
| **Zero-friction adoption** | The AI is an *invisible background validation layer* — 0 extra clicks to see explanations (NFR-12/13). Clinicians keep their workflow; the system removes liability risk instead of adding work. |
| **Non-negotiable safety** | Human-in-the-Loop signature gate (NFR-19), 100% MOH-cited explanations (NFR-18), immutable audit log (NFR-17), and a 4-quadrant safety framework that explicitly defends against automation bias *and* AI hallucination. |
| **Regulatory sovereignty** | Runs inside the hospital LAN. Decree 13/2023 PII protection + Circular 46/2018 EMR compliance are built into the architecture, not bolted on. No patient data leaves the hospital in production. |
| **Capital efficiency** | Built on proven open-source (K3s, Triton, Postgres/pgvector, Keycloak, MinIO) and existing VKIST models. No SaaS lock-in, no per-seat licensing, delivered in a fixed 3-month pilot. |
**The ask.** Sponsorship of a fixed 3-month pilot on a single hospital's on-premise infrastructure (one Dell PowerEdge class node + one GPU node), a small cross-functional team, and a *governed, time-boxed* cloud allowance for the proof-of-concept LLM tiers (NFR-16a, retired at sign-off). See [Section 8](#8-investment-ask--scope-boundaries).
**The expected return.** A validated, referenceable clinical AI product that (projected) removes tedious measurement work from overloaded specialists, gives physiotherapists diagnostic visibility they have never had, converts patient anxiety into institutional trust, and establishes VKIST's operational credibility in the local healthcare community — with every claim instrumented and measurable during the pilot itself (see [Section 4](#4-quantitative-roi--projected-metrics--measurement-plan)).
> **Bottom line for the decision-maker:** This is a low-blast-radius, standards-compliant, single-site pilot that converts already-funded research into a safe, auditable clinical tool. The downside is bounded (one hospital, 3 months, open-source stack); the upside is a defensible product line and clinical-community trust.
---
## 1. Project Context
### 1.1 Who the System Is Designed For
The platform coordinates a **four-persona clinical care chain**. The pilot (Sprint 12, FR-25) is centered on the **Diagnostic Radiologist (UP5)** as the primary human actor, with the other three personas as governance/future-sprint stakeholders whose needs shape the non-functional requirements.
| Persona | Who they are | Environment | Core responsibility | What the system must respect |
| :--- | :--- | :--- | :--- | :--- |
| **UP5 — Diagnostic Radiologist** | Specialist MDs, age 2855, from digitally native attendings to department heads | Low-light reading rooms; massive daily DICOM backlogs; asynchronous, detached from patients | Translate images into definitive reports that gate all downstream care | *Invisible workflow preservation* — no extra clicks, no DICOM render lag, no uncompensated patient chat |
| **UP7 — Rheumatologist / Orthopedic Surgeon** | Treatment architects, age 3060; seniors control procurement | Split across OR, chaotic outpatient clinics (>100 patients/shift), inpatient wards | Synthesize fragmented data into a treatment plan; bear full legal/clinical liability | *Clinical force multiplier* — rapid dashboards, automated patient-education visuals, strictly gated communication |
| **UP6 — Physical Therapist** | Young (3640% aged 2029), highly stratified (≈54% vocational tier), digitally literate | Cramped (<20 m²), high-noise public units; 1120+ patients/shift; shared/scarce workstations | Execute physician scripts into kinetic rehab programs | *Digital exoskeleton* gel-proof knuckle-tap UI, zero-GPU mobile, localized language, objective progress data |
| **UP8 — MSK Patient & Family Caregiver** | Elderly patients (4580+) via younger caregiver proxies (2045) | Home, pharmacy, crowded waiting rooms; saturated with health misinformation | Manage chronic pain; adhere to protocols; make family-centric decisions | *Empathy + accessibility* jargon-free, high-contrast, large-font, dual-profile family access on legacy phones |
*Source: [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md), [CONTEXT_VISION_SCOPE.md §3](../PLAN/CONTEXT_VISION_SCOPE.md).*
### 1.2 The Pain Points That Drive the Need
These are not hypothetical they are drawn from ethnographic user research (professional forums, patient forums, and published studies).
- **Clinical exhaustion & volume:** Central referral hospitals (Bach Mai, Viet Duc) regularly exceed **100 evaluations per shift**, leaving no time for tedious measurements or objective double-checks. This creates the risk of *automation-bias blind-concurrence* a clinician rubber-stamping an AI estimate under time pressure.
- **The physiotherapist information bottleneck:** PTs receive only brief text prescriptions no DICOM, no scan findings forcing **semi-blind therapeutic execution** against estimated surface anatomy.
- **The physiotherapist body & literacy crisis:** **76.4%** WMSD prevalence (career-shortening physical strain); **84%** report a foreign-language barrier (only 25% adequate English); 96%/93%/86% rely on personal experience/peer chat/old textbooks rather than formal evidence.
- **Dangerous folk interventions:** Information-starved patients turn to leaf-wrapping or violent manual adjustment, causing tissue infection or permanent anatomical failure because no one translated their scan into understandable terms.
- **Stranded research value:** VKIST's CV models (DeepLabv3/UNet/ConvNeXt/MedViT) are proven in the lab but have no production pathway, so they generate zero operational or reputational value.
### 1.3 Constraints the System Must Meet
**Functional Requirements (pilot scope — FR-25 pipeline):**
| FR | Requirement |
| :--- | :--- |
| **FR-23/24/25** | Segment knee joint structures measure synovium thickness (mm) grade synovitis **03** |
| **FR-25 / Grad-CAM** | Spatial activation overlay on the primary viewport with **zero extra clicks** |
| **FR-25 / Safety loop** | Conversational Circuit Breaker + Socratic dialogue before finalization; BERT drift monitor; RAG-Referee arbitration |
| **FR-25 / Reporting** | Circular 46/2018 compliant PDF report generation |
| **FR-25 / Finalization** | Cryptographically signed, HITL-gated electronic record EMR |
**Non-Functional Requirements (the "why it won't fail" constraints):**
| NFR | Constraint | Why the leader cares |
| :--- | :--- | :--- |
| **NFR-4** | 150 MB idle app bundle | Runs on cheap, legacy hospital/consumer hardware |
| **NFR-5** | 1.5 s core inference (on-prem) | No workflow slowdown |
| **NFR-7** | 200 ms token streaming (TTFT) | Feels instantaneous to the clinician |
| **NFR-9** | 99.9% availability (≤ 45 s downtime/day) | Clinically dependable |
| **NFR-10** | 100% of patient-facing LLM text passes safety verification | No unsafe generative output reaches a human |
| **NFR-11** | Onboarding 45 minutes | Cheap to roll out to busy staff |
| **NFR-12/13** | 0 extra clicks for explainability; native spatial activation maps | Anti-black-box; builds clinician trust |
| **NFR-14** | No client-side GPU/accelerator required (Android 10+, 3 GB RAM) | Works on the fleet that actually exists |
| **NFR-15** | Circular 46/2018 EMR compliance | Legally deployable |
| **NFR-16** | Air-gapped primary (no PHI over public internet in production) | Data sovereignty (Decree 13) |
| **NFR-17** | Immutable audit log | Accountability & liability defense |
| **NFR-18** | 100% LLM clinical text cites MOH protocol via RAG | Every claim is traceable & defensible |
| **NFR-19** | HITL digital signature before FINALIZED/ARCHIVED | A licensed human always owns the decision |
*Source: [SOLUTION_ARCHITECTURE_SPEC.md §2](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md), [SOFTWARE_ARCHITECTURE_SPEC.md §2](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
### 1.4 System Context — C4 Level 1 (PlantUML)
The context diagram summarizes *who* and *what* the system boundary talks to. The pilot activates UP5 (and governance actors UP1/UP4); UP6/UP7/UP8 are shown as future-sprint boundaries.
```plantuml
@startuml VKIST_MSK_System_Context
!include <C4/C4_Context>
title System Context Diagram — VKIST MSK Ultrasound Stack (Pilot: FR-25 Synovitis Grading)
' === PRIMARY HUMAN ACTOR (in pilot scope) ===
Person(radiologist, "Diagnostic Radiologist (UP5)", "Loads knee ultrasound, reviews AI grading (0-3), confirms/overrides, engages Socratic circuit breaker, finalizes & signs the report, reads GradCAM + RAG evidence")
Person(admin, "System Administrator", "Deploys/monitors on-prem K3s, model updates, observability, NGINX failover")
' === GOVERNANCE / NFR-ALIGNMENT ACTORS ===
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Clinical protocol validation, model-threshold approval, pilot sponsor")
Person_Ext(support_staff, "Support Staff (UP4)", "Patient registration, case-queue management")
' === FUTURE-SPRINT PERSONAS (out of pilot scope) ===
Person_Ext(therapist, "Physical Therapist (UP6)", "Future: read-only diagnostic visibility, kinetic overlays, progress tracking")
Person_Ext(surgeon, "Surgeon / Rheumatologist (UP7)", "Future: aggregated dashboards, treatment planning")
Person_Ext(patient, "Patient & Caregiver (UP8)", "Future: patient portal, plain-language education, self-monitoring")
' === THE SYSTEM ===
System(mpps, "VKIST MSK Ultrasound Stack (FR-25)", "Air-gapped-first: knee ultrasound AI (angle -> inflammation -> segmentation), synovitis grading 0-3 with GradCAM, 4-quadrant clinical safety, MOH-cited Vietnamese explanations, HITL-signed Circular 46 reports")
' === EXTERNAL SYSTEMS ===
System_Ext(pacs, "Hospital PACS / Ultrasound Machine", "DICOM source (C-MOVE + direct capture)")
System_Ext(emr, "Hospital EMR / HIS", "Signed report storage, prescription/lab sync (HL7/FHIR)")
System_Ext(triton, "Triton Inference Server", "GPU node: 3-step ML pipeline + RAG embeddings (gRPC)")
System_Ext(knowledge, "Clinical Knowledge Systems", "ladybugDB ontology (SNOMED-CT), pgvector MOH guideline index, Gemma/MedGemma LLM tiers")
' === RELATIONSHIPS ===
Rel(radiologist, mpps, "Reviews grade, overrides, finalizes & signs, reads explanations", "HTTPS")
Rel(admin, mpps, "Deploys, monitors, updates models", "HTTPS / SSH")
Rel(senior_expert, mpps, "Validates protocols, approves thresholds", "HTTPS (governance)")
Rel(support_staff, mpps, "Registration, case queue", "HTTPS")
Rel(therapist, mpps, "Future sprint scope", "—")
Rel(surgeon, mpps, "Future sprint scope", "—")
Rel(patient, mpps, "Future sprint scope", "—")
Rel(mpps, pacs, "Imports DICOM", "DICOM / C-MOVE")
Rel(mpps, emr, "Pushes signed reports + audit records", "HL7/FHIR")
Rel(mpps, triton, "ML inference offload", "gRPC 8001")
Rel(mpps, knowledge, "RAG retrieval, ontology traversal, LLM generation", "SQL (pgvector) + in-process")
@enduml
```
*Adapted from [SOLUTION_ARCHITECTURE_SPEC.md §8.1](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md).*
### 1.5 Applicability — Use-Case View (PlantUML)
The pilot's functional scope is organized around a **4-Quadrant Human-AI Interaction Framework** that deliberately defends against both *automation bias* (blindly trusting AI) and *clinician subservience* (AI overriding a correct human). This is the applicability model what the radiologist can actually *do* with the system.
```plantuml
@startuml VKIST_MSK_UseCases
left to right direction
skinparam packageStyle rectangle
actor "Diagnostic Radiologist (UP5)" as Rad
actor "MSK Vision Grader" as Grader <<system>>
actor "LLM Explainer" as LLM <<system>>
actor "RAG-Referee Arbitrator" as Referee <<system>>
actor "Hospital EMR" as EMR <<system>>
rectangle "VKIST MSK Workspace (FR-25)" {
usecase "UC-48376\nLoad Patient Scan Session" as U1
usecase "UC-47988\nReview Suggested Synovitis Grade (0-3)" as U2
usecase "UC-92006\nFinalize & Sign Electronic Record" as U3
rectangle "Q1 — True Agreement (AI correct / Doctor correct)" {
usecase "UC-25776\nGradCAM + CoT Explanation" as Q1a
usecase "UC-02423\nLog High-Trust Concur Block" as Q1b
}
rectangle "Q2 — Automation Override (AI correct / Doctor oversight)" {
usecase "UC-22159\nTrigger Circuit Breaker" as Q2a
usecase "UC-55146\nSocratic Reasoning Dialogue" as Q2b
usecase "UC-74821\nBERT Context-Drift Monitor" as Q2c
usecase "UC-65473\nArbitrate via RAG-Referee" as Q2d
}
rectangle "Q3 — Clinician Subservience (AI hallucinates / Doctor correct)" {
usecase "UC-25637\nExpose Pixel-Level Activation" as Q3a
usecase "UC-60739\nIsolate Visual Noise/Artifacts" as Q3b
usecase "UC-62864\nCommit Validated Ground-Truth" as Q3c
}
rectangle "Q4 — Double-Blind Failure (AI faulty / edge case)" {
usecase "UC-35956\nActivate Clinical Investigation Mode" as Q4a
usecase "UC-47796\nStructured Morphology Annotation" as Q4b
usecase "UC-01580\nSerialize to Telemetry Queue" as Q4c
}
}
Rad --> U1
Rad --> U2
Rad --> U3
Rad --> Q2b : Argue observations
Rad --> Q3b : Tag artifacts
Rad --> Q4b : Document manual findings
Grader --> U1 : Feeds vision tensors + scores
Grader --> U2 : Generates prediction
LLM --> Q1a : Generates clinical CoT
LLM --> Q2b : Generates Socratic checks
Referee --> Q2d : Retrieves MOH guidelines
U3 --> EMR : Syncs signed report
Q1b --> EMR : Stores consensus log
Q4c --> EMR : Routes anomaly to engineering telemetry
U1 ..> Q1a : <<include>>
Q1a ..> Q1b : <<include>>
U2 <.. Q2a : <<extend>> (friction detected)
Q2a ..> Q2b : <<include>>
Q2a ..> Q2c : <<include>>
Q2c ..> Q2d : <<extend>> (impasse/drift)
U2 <.. Q3a : <<extend>> (clinician contests score)
Q3a ..> Q3b : <<include>>
Q3b ..> Q3c : <<include>>
U2 <.. Q4a : <<extend>> (low confidence + empty RAG)
Q4a ..> Q4b : <<include>>
Q4b ..> Q4c : <<include>>
@enduml
```
*Adapted from [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md) and [CONTEXT_FR_25_UC_ELIT.md](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/CONTEXT_FR_25_UC_ELIT.md).*
---
## 2. How the System Addresses the Problem
### 2.1 Technology Choices — Problem → Solution → Why Optimal → Limitation
Each technology is selected against a specific user problem or NFR, not for novelty. The table states the honest limitation of each choice so the leader sees a balanced picture.
| Technology | User problem / NFR it solves | Why it is the optimal choice | Limitation / trade-off |
| :--- | :--- | :--- | :--- |
| **React PWA + LiteRT (MediaPipe) + WebLLM edge** | NFR-14 (no client GPU), NFR-4 (150 MB), zero-friction on legacy phones | Installable, offline-capable, runs a local angle pre-classifier + edge LLM with no server round-trip; degrades gracefully on weak hardware | Browser memory ceilings; edge LLM is small (GemmaE2B-Q4 ~1.3 GB) so deep reasoning must escalate |
| **FastAPI edge-inference-svc (Python 3.12)** | FR-25 pipeline orchestration, Decree 13 middleware, SSE streaming | Async, lightweight, tight control over PII redaction + audit at the boundary; in-house domain logic | Python GIL for CPU-bound work mitigated by offloading inference to Triton |
| **Triton Inference Server (ONNX/TensorRT)** | NFR-5 (≤1.5 s inference), reuse of VKIST CV models | Industry-standard, GPU-efficient, self-hosted (no SaaS); serves the 3-step ensemble + embeddings | Requires a GPU node (capex); one more component to operate |
| **Postgres + pgvector (HNSW)** | NFR-18 MOH-cited RAG, transactional consistency of clinical records | One store for records *and* vectors (~15K guideline vectors) ~520 ms retrieval, no separate vector DB to run | Not optimal beyond ~100K vectors / 500 QPS (documented Phase-2 path to Qdrant) |
| **RAG-Referee (BERT classifier)** | NFR-10/18 reject unsupported LLM claims | Validates attribution + logical cohesion of every explanation before it reaches a human | Adds latency to the cloud tier; thresholds need clinical tuning |
| **3-Tier LLM (Edge Gemma → Gemini → MedGemma)** | Vietnamese explanations, deep clinical reasoning, cost control | Cheapest-capable-tier routing: local first, cloud only when needed, <20% MedGemma usage cap | Cloud tiers are PoC-only (NFR-16a) and must be redacted retired at sign-off |
| **Keycloak (OIDC/RBAC) on-prem** | NFR-16 sovereignty, role separation (PT read-only) | No SaaS identity; realm-scoped RBAC inside the LAN | Operational overhead vs. managed Auth0/Okta (accepted for sovereignty) |
| **K3s on Dell PowerEdge** | NFR-16 air-gap, NFR-9 availability | Lightweight Kubernetes for a single hospital; rolling updates, self-healing pods; NGINX+Keepalived VIP failover (≤2 s) | Single-site scale; not a multi-region HA design (by design for the pilot) |
| **MinIO + Redis (5 scoped types)** | Object/blob store + session/cache | S3-compatible, self-hosted, no cloud egress; Redis scoped to prevent sprawl | Self-managed durability (mitigated by backup cron to MinIO) |
**Honest, system-level limitations (stated up front for the leader):**
- **Single-hospital scale by design.** The pilot targets one site; multi-site federation is a documented Phase-2 evolution, not a pilot promise.
- **Cloud LLM tiers are proof-of-concept only** under NFR-16a, gated by redaction + consent + audit, and **formally retired at PoC sign-off** production reverts to fully air-gapped NFR-16.
- **Edge model is intentionally small** deep reasoning escalates through the tier chain, which is why the cost guard (<20% MedGemma) and RAG-Referee exist.
*Source: [SOFTWARE_ARCHITECTURE_SPEC.md §7 Build-vs-Buy](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md), [SOLUTION_ARCHITECTURE_SPEC.md §5](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md).*
### 2.2 Deployment Environment — C4 Container Diagram (PlantUML)
The system runs entirely inside the hospital LAN. The only external touchpoints are the governed, PoC-scoped cloud fallbacks (NFR-16a).
```plantuml
@startuml VKIST_MSK_Containers
!include <C4/C4_Container>
title C4 Container Diagram — VKIST MSK Ultrasound Stack
Person(radiologist, "Diagnostic Radiologist (UP5)", "Loads DICOM, reviews grading, finalizes reports")
Person(admin, "System Administrator", "K3s ops, model updates, observability, failover")
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Protocol validation, threshold approval")
Person_Ext(support_staff, "Support Staff (UP4)", "Registration, case queue")
System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, <=10 Mbps)") {
Container(pwa, "React PWA Frontend", "React 18, TS, Zustand, LiteRT, MediaPipe, Dexie.js", "View-angle validation (WASM); DICOM preview; Grad-CAM overlay; Decree 13 scrubber; encrypted IndexedDB sessions")
Container(nginx, "Active-Passive Gateway", "NGINX + Keepalived", "SSL termination, VIP load balancing, <=2 s failover")
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12)", "DICOM ingest, 3-step ML orchestration, Grad-CAM, report assembly, SSE streaming")
Container(cloud_llm_gateway, "Cloud LLM Gateway", "FastAPI (Python 3.12)", "Routes to Gemini (orchestration) / MedGemma (clinical); NFR-16a redaction, consent, audit, cost guard")
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (asyncpg)", "pgvector top-k, ladybugDB ontology, RAG-Referee validation, mandatory RAG pre-processing")
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit; HL7/FHIR EMR push with outbox retry")
Container(api_gw, "API Gateway", "Envoy", "TLS, rate limiting, routing, OIDC pass-through")
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk")
Container(obs_stack, "Observability Stack", "Prometheus + Grafana", "Metrics, dashboards, alerting")
}
System_Boundary(db_vm, "Database VM") {
ContainerDb(postgres, "PostgreSQL + pgvector", "PostgreSQL 16 + pgvector", "EMR records, spatial markers, MOH HNSW index, embeddings, audit ledger")
ContainerDb(redis, "Redis Cache", "Redis 7 (AOF+RDB)", "Sessions, guideline chunks, DICOM metadata, consult_mode state, rate-limit")
ContainerDb(minio, "MinIO Object Store", "MinIO S3-compatible", "DICOM payloads, overlays, Grad-CAM heatmaps, report PDFs, model staging")
}
}
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton (ONNX/TensorRT)", "3-step ML pipeline + EmbeddingGemma RAG embeddings")
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Signed report storage, prescription sync")
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM / C-MOVE", "Image capture & retrieval")
System_Ext(vertex_ai, "GCP Vertex AI (Gemini)", "REST", "PoC-only NFR-16a: orchestration, translation; redacted payloads only")
System_Ext(modal_medgemma, "Modal MedGemma", "T4 GPU", "PoC-only NFR-16a: clinical deep-reasoning; redacted + RAG-Referee validated")
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes, reads explanations", "HTTPS 443")
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS / SSH")
Rel(senior_expert, pwa, "Validates protocols, approves thresholds", "HTTPS 443")
Rel(support_staff, pwa, "Registration, case queue", "HTTPS 443")
Rel(pwa, nginx, "API requests + pre-validated DICOM", "HTTPS 443")
Rel(nginx, api_gw, "Routes upstream", "HTTP 8000")
Rel(api_gw, edge_inference, "Forwards /api/*", "HTTP 8000")
Rel(api_gw, rag_svc, "Forwards /rag/*", "HTTP 8001")
Rel(api_gw, auth_svc, "Validates OIDC tokens", "HTTP 8080")
Rel(edge_inference, triton, "3-step inference + embeddings", "gRPC 8001")
Rel(edge_inference, postgres, "Reads guidelines; writes cases/embeddings/audit", "SQL 5432")
Rel(edge_inference, redis, "Session, rate-limit, consult_mode", "TCP 6379")
Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
Rel(edge_inference, cloud_llm_gateway, "Routes cloud consult", "HTTP 8000")
Rel(cloud_llm_gateway, vertex_ai, "Gemini proxy (NFR-16a redacted)", "HTTPS 443")
Rel(cloud_llm_gateway, modal_medgemma, "MedGemma proxy (NFR-16a redacted)", "HTTPS 443")
Rel(cloud_llm_gateway, postgres, "Audit log append", "SQL 5432")
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
@enduml
```
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.2](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
### 2.3 Deployment Topology — C4 Deployment Diagram (PlantUML)
Physical placement: everything on hospital hardware; the only external node is the governed NFR-16a fallback.
```plantuml
@startuml VKIST_MSK_Deployment
!include <C4/C4_Deployment>
LAYOUT_LEFT_RIGHT()
Deployment_Node(hw, "Dell PowerEdge (Hospital)") {
Deployment_Node(k3s, "K3s Cluster") {
Deployment_Node(pod_edge, "Pod: edge-inference-svc") {
Container(ci, "FastAPI", "Python 3.12, Uvicorn")
}
Deployment_Node(pod_rag, "Pod: rag-svc") {
Container(cr, "FastAPI", "Python 3.12, asyncpg")
}
Deployment_Node(pod_audit, "Pod: audit-svc") {
Container(ca, "Node.js", "WAL writer")
}
Deployment_Node(pod_gw, "Pod: api-gateway") {
Container(cg, "Envoy", "TLS, rate limit")
}
Deployment_Node(pod_auth, "Pod: auth-svc") {
Container(ck, "Keycloak", "OIDC, RBAC")
}
Deployment_Node(pod_obs, "Pod: observability") {
Container(cp, "Prometheus")
Container(cf, "Grafana")
}
}
Deployment_Node(db_vm, "DB VM") {
ContainerDb(pg, "PostgreSQL + pgvector HNSW")
ContainerDb(rd, "Redis")
ContainerDb(mn, "MinIO")
}
}
Deployment_Node(triton_node, "GPU Node (Triton)", "NVIDIA A10/T4, 24 GB VRAM") {
Container(tc, "Triton Inference Server", "ONNX/TensorRT, gRPC 8001")
}
Deployment_Node(ext, "External (NFR-16a governed, PoC only)") {
ContainerDb(s3v, "GCP Vertex AI / Modal MedGemma / CDN fallback")
}
Rel(ci, pg, "SQL", "TCP 5432")
Rel(ci, rd, "Cache", "TCP 6379")
Rel(ci, mn, "Blobs", "S3 API")
Rel(ci, tc, "Inference", "gRPC 8001")
Rel(ci, ck, "Auth", "OIDC")
Rel(ci, ca, "Audit", "HTTP")
Rel(k3s, db_vm, "SQL / Cache / Blobs", "TCP")
@enduml
```
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.4](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
### 2.4 Components & Interactions — C4 Component Diagram (PlantUML)
Inside the core `edge-inference-svc`, each component maps to a specific functional requirement. The **Inference Router** implements the tiered LLM escalation; the **Circuit Breaker** guarantees the system fails safely to MOH templates.
```plantuml
@startuml VKIST_MSK_Component_EdgeInference
!include <C4/C4_Component>
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
Component(api, "API Controller", "REST + SSE", "/api/analyze, cloud-orchestrate redirect")
Component(stream, "SSE Token Streamer", "StreamingResponse", "200 ms TTFT, 30 s heartbeat")
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub")
Component(router, "Inference Router", "consult_mode state", "Tier selection: Edge Gemma -> Gemini -> MedGemma -> Templates")
Component(breaker, "Circuit Breaker", "pybreaker", "Wraps Triton + EMR + Cloud LLM; fail-open to templates")
Component(pipeline, "ML Pipeline", "gRPC client", "Angle -> Inflammation -> Segmentation -> Measurement")
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay -> base64 PNG")
Component(report, "Report Builder", "WeasyPrint", "Circular 46 PDF; HITL signature gate")
Component(rag, "RAG Service", "pgvector SQL", "Top-5 MOH chunks; mandatory pre-generation; NFR-18")
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; clean residual PII")
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
}
Rel(api, stream, "delegates", "SSE")
Rel(api, preproc, "validates image", "sync")
Rel(api, router, "selects tier", "sync")
Rel(router, breaker, "guards calls", "sync")
Rel(router, pipeline, "invokes", "gRPC")
Rel(router, stream, "fallback text", "SSE")
Rel(api, gradcam, "requests overlay", "sync")
Rel(api, report, "generates PDF", "sync")
Rel(api, rag, "queries MOH", "SQL")
Rel(rag, referee, "validates citations", "sync")
Rel(api, nlp, "scrubs output", "sync")
Rel(api, audit, "writes event", "sync")
@enduml
```
The **Cloud LLM Gateway** enforces the NFR-16a governance boundary nothing egresses without consent, redaction, RAG grounding, and an audit record.
```plantuml
@startuml VKIST_MSK_Component_CloudGateway
!include <C4/C4_Component>
Container_Boundary(cloud_gw, "cloud-llm-gateway (FastAPI)") {
Component(capi, "Cloud API Controller", "REST + SSE", "/api/cloud-orchestrate, /api/cloud-consult")
Component(gateway, "Cloud LLM Router", "task_type matcher", "Orchestration/translation -> Gemini; clinical -> MedGemma")
Component(consent, "Consent Enforcer", "Redis + NFR-16a checklist", "Validates consent token + redaction manifest before egress")
Component(caudit, "Cloud Audit Emitter", "PostgreSQL append-only", "egress_consent, egress_response, escalation events")
Component(cost, "Cost Guard", "Redis + Prometheus", "Tracks MedGemma usage; alerts if >20% over 24h")
Component(creferee, "RAG-Referee Trigger", "BERT classifier", "Mandatory validation of MedGemma output")
Component(crag, "RAG Pre-processing", "pgvector SQL", "Mandatory top-k injection before generation")
}
Rel(capi, consent, "verifies", "sync")
Rel(capi, gateway, "routes", "sync")
Rel(gateway, crag, "injects chunks", "sync")
Rel(gateway, creferee, "validates output", "sync")
Rel(gateway, caudit, "logs egress", "sync")
Rel(gateway, cost, "increments counter", "sync")
@enduml
```
*Reused from [SOFTWARE_ARCHITECTURE_SPEC.md §4.3](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md).*
---
## 3. Qualitative ROI — Behavioral & Workflow Change
The core value thesis: the system does not add a tool to a workflow it **enriches the existing workflow and reconnects a fragmented care chain**. The following before/after captures the behavioral change per persona.
| Persona | Before (current behavior) | After (with the system) | Behavioral shift |
| :--- | :--- | :--- | :--- |
| **UP5 Radiologist** | Manual, repetitive measurements under fatigue; risk of missing subtle findings; isolated in a reading room; liability anxiety | AI pre-measures + grades; Grad-CAM shows *why* with 0 extra clicks; a Socratic circuit breaker catches rushed sign-offs; every report is MOH-cited and audit-backed | From lone, fatigued gatekeeper to a *validated* decision-maker shielded from liability |
| **UP7 Surgeon** | Synthesizes fragmented silos by hand; no time for patient education; fears uncompensated patient messaging | Aggregated view of signed findings; auto-generated patient-friendly visuals; strictly gated (no open chat) communication | From data-assembler to *clinical force multiplier* with protected time |
| **UP6 Physiotherapist** | Semi-blind execution from a one-line prescription; no scan visibility; language barrier; manual charting | Read-only diagnostic visibility down the chain; objective progress metrics; gel-proof, localized, zero-GPU UI | From guesswork executor to *data-backed* practitioner with professional standing |
| **UP8 Patient/Caregiver** | Anxiety + confusion folk remedies; crowdsources raw scans on social media | Scan translated into plain-language, high-tech visual validation of the doctor's diagnosis | From anxious misinformation-seeker to a *trusting, informed* participant |
**System-level qualitative returns:**
- **Institutional trust & credibility:** A high-trust, objective, MOH-anchored tool displaces both dangerous folk interventions (patients) and unsecured external messaging (clinicians bypassing the system).
- **Academic-to-production migration:** VKIST research models become a live, referenceable clinical asset establishing the team's operational value in the local healthcare community.
- **Care-chain cohesion:** The radiologist's sign-off *ripples* to the surgeon, PT, and patient turning isolated actions into a connected clinical relationship (the "Synchronized Ritual" concept).
*Source: [Motivation.md](../DEMO_EXP/Motivation.md), [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md).*
---
## 4. Quantitative ROI — Projected Metrics & Measurement Plan
> **Important framing for the leader:** The pilot has not yet deployed (runs through Sep 2026), so the figures below are **modeled projections**, not measured outcomes. Each row states its **baseline source** and its **assumption**, and — critically — **how the pilot will actually measure it**. The value of the pilot is precisely to *convert these projections into evidence*.
### 4.1 Projected Before → After Metrics
| Metric | Baseline (before) | Target (after) | Assumption / basis | How the pilot measures it |
| :--- | :--- | :--- | :--- | :--- |
| **Tedious measurement time per scan** | Manual synovium measurement + grading, several minutes/scan under load | Automated pre-measurement; radiologist verifies | FR-25 automates FR-23/24 (segment + measure) | Time-on-task instrumentation (session telemetry, NFR audit timestamps) |
| **Core inference latency** | N/A (no automated pipeline today) | 1.5 s (on-prem) | NFR-5 target on Triton GPU node | Prometheus p50/p99 latency dashboards |
| **Explainability access cost** | Not available / manual literature lookup | **0 extra clicks** (Grad-CAM + citation inline) | NFR-12/13 design mandate | UI event audit (clicks-to-explanation = 0) |
| **Clinical claim traceability** | Ad-hoc, uncited | **100%** of LLM clinical text cites MOH protocol | NFR-18 mandatory RAG | RAG-Referee pass rate; citation coverage log |
| **Unsafe generative output reaching a human** | N/A | **0%** (100% pass safety verification) | NFR-10 3-tier guardrail | Guardrail rejection log; audit events |
| **Onboarding time per staff member** | Ad-hoc, hours | 45 minutes | NFR-11 target | Timed onboarding sessions during rollout |
| **System availability** | Dependent on manual workflow | 99.9% (≤ 45 s downtime/day) | NFR-9 target | Uptime monitoring (Prometheus/Grafana) |
| **Cloud (MedGemma) usage ratio** | N/A | < 20% of consults | Cost-guard target (capital efficiency) | Cost Guard Redis counter + alert |
| **PT diagnostic visibility** | ~0 (text prescription only) | Read-only access to signed findings | RBAC role (future sprint) | Access-log coverage per PT-linked case |
| **Auditability of AI accept/override** | None | 100% immutable capture | NFR-17 | Immutable audit ledger completeness check |
### 4.2 Behavioral-Change Indicators (Adoption Proxies)
Beyond raw performance, the pilot measures whether behavior actually changed the true ROI signal for a leader:
- **Retention / anti-bypass:** % of target clinicians using the structured workspace instead of unsecured external messaging (a stated success criterion).
- **Override rate & ground-truth capture:** frequency of clinician overrides (Q3) feeding the retraining corpus proves the human-AI loop works both ways.
- **Circuit-breaker engagement:** how often the Socratic breaker prevents a rushed sign-off (a direct automation-bias safety signal).
- **Compliance events:** zero unencrypted PHI leak events (Decree 13) a binary go/no-go for production.
*Baselines from [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md); targets from NFRs in [SOLUTION_ARCHITECTURE_SPEC.md §2.2](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md); success criteria from [CONTEXT_VISION_SCOPE.md §6](../PLAN/CONTEXT_VISION_SCOPE.md).*
---
## 5. Risk & Failure-Mode Handling — Why This Is Not Risky
The single most important message for an investment decision: **this system is engineered around the assumption that either the AI *or* the human can be wrong.** It handles all four combinations explicitly via the **4-Quadrant Human-AI Interaction Framework**.
```plantuml
@startuml VKIST_MSK_Risk_Quadrants
skinparam packageStyle rectangle
title 4-Quadrant Human-AI Safety Framework (FR-25)
rectangle "Q1 — True Agreement\n(AI correct / Doctor correct)" as Q1
rectangle "Q2 — Automation Override\n(AI correct / Doctor oversight)" as Q2
rectangle "Q3 — Clinician Subservience\n(AI hallucinates / Doctor correct)" as Q3
rectangle "Q4 — Double-Blind Failure\n(AI faulty / edge case)" as Q4
Q1 --> Q1 : GradCAM + CoT -> High-Trust Concur log
Q2 --> Q2 : Circuit Breaker -> Socratic -> BERT drift -> RAG-Referee
Q3 --> Q3 : Expose activations -> isolate artifacts -> commit ground-truth
Q4 --> Q4 : Investigation mode -> manual annotation -> telemetry queue
@enduml
```
| Risk quadrant | The danger | How the system mitigates it |
| :--- | :--- | :--- |
| **Q1 — True Agreement** | Even correct consensus needs a trace for liability | Grad-CAM + Chain-of-Thought explanation, then an immutable High-Trust Concur log block |
| **Q2 — Automation Bias** | Overloaded doctor rubber-stamps AI | Conversational Circuit Breaker intercepts rushed finalization Socratic dialogue BERT drift monitor RAG-Referee arbitration against MOH guidelines |
| **Q3 — AI Hallucination** | AI is wrong, doctor is right, but AI "authority" pressures the human | Expose pixel-level activation logic; let the clinician mask artifacts and **commit the corrected ground-truth** (which also feeds retraining) |
| **Q4 — Double-Blind Edge Case** | Both AI confidence and guideline coverage are low | Auto-switch to strict manual **Clinical Investigation Mode**; force structured morphology annotation; route the anomaly to an engineering telemetry queue (bypassing the EMR) |
**Cross-cutting safety guarantees:**
- **HITL gate (NFR-19):** No AI output becomes FINALIZED/ARCHIVED without a licensed clinician's cryptographic signature. *A human always owns the decision.*
- **Fail-safe by design:** The Circuit Breaker fails **open to deterministic MOH templates** if Triton, EMR, or a cloud LLM is down, the system degrades to a safe rule-based floor, not to an error.
- **Immutable audit (NFR-17):** Every accept/override/tier-transition/consent event is append-only and tamper-proof a complete liability defense record.
- **RAG-Referee (NFR-18):** No uncited clinical claim reaches a human.
*Source: [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md), and the Q1Q4 scenario docs under [FR_25_UC_DESIGN/](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/).*
---
## 6. Compliance & Data Governance
Healthcare compliance is treated as a first-class architectural constraint, not an afterthought.
| Regulation | Requirement | How the architecture satisfies it |
| :--- | :--- | :--- |
| **Decree 13/2023/ND-CP** (personal health data) | Sensitive PII must be protected; data sovereignty | Client-side WebCrypto scrubbing + FastAPI Presidio redaction middleware; role-hash tokens `[BN:7f3a2c]`; all production data resides in-hospital / sovereign VN data centers |
| **Circular 46/2018/TT-BYT** (EMR) | National EMR compliance for stored records | Circular-46-compliant PDF reports; HL7/FHIR EMR push; NFR-15 (not relaxed even under NFR-16a) |
| **Air-gap (NFR-16, permanent)** | No diagnostic/identifiable data over public internet in production | Everything runs on the hospital LAN; K3s network policies isolate workloads; local-only container registries |
| **Governed cloud PoC (NFR-16a, temporary)** | Cloud LLM only as an emergency/PoC fallback | Invoked only when edge LLM unavailable **and** user consents; mandatory pre-egress redaction + audit commit; CSEK keys, 30-day lifecycle, 1-year auto-delete; **retired at PoC sign-off** |
**The compliance headline for a leader:** in production, **no patient data leaves the hospital**, every clinical statement is cited, every action is audit-logged, and a human signature is legally required to finalize anything.
*Source: [SOLUTION_ARCHITECTURE_SPEC.md §2.2, §5.8](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md), [CONTEXT_VISION_SCOPE.md §4](../PLAN/CONTEXT_VISION_SCOPE.md).*
---
## 7. Roadmap & Milestones
A strict, time-boxed 3-month cycle (June 2 September 2, 2026), executed as a hybrid of waterfall macro-phases and agile sprints. This bounds the investment and provides clear go/no-go checkpoints - However it can be tentative to change based on the actual resource of development team.
```mermaid
gantt
title VKIST MSK Pilot - 3-Month Delivery Roadmap
dateFormat YYYY-MM-DD
axisFormat %b %d
section Foundation
Sprint 0 Pitch and Architecture Spike :done, sp0, 2026-06-02, 2026-06-12
Win project pitch :milestone, m1, 2026-06-09
Sprint 1 Fast PoC Baseline :active, sp1, 2026-06-15, 2026-06-26
section Core Build
Sprint 2 Multi-Modal and NLP Integration :sp2, 2026-06-29, 2026-07-10
Sprint 3 Collaborative Workspace :sp3, 2026-07-13, 2026-07-24
section Patient and Hardening
Sprint 4 Patient-Facing PWA :sp4, 2026-07-27, 2026-08-07
Sprint 5 Feedback Pipeline and Hardening :sp5, 2026-08-10, 2026-08-21
Sprint 6 Beta Release and Evolution :sp6, 2026-08-24, 2026-09-02
Pilot cycle complete :milestone, m2, 2026-09-02
```
| Sprint | Window | Focus & key deliverable |
| :--- | :--- | :--- |
| **0** | Jun 0212 | Pitch & architecture clearance; VKIST ML model audit; CI/CD baseline |
| **1** | Jun 1526 | Fast end-to-end PoC: FastAPI pipeline + inference mask preview |
| **2** | Jun 29Jul 10 | Multi-modal NLP + Decree 13 client-side scrubbing |
| **3** | Jul 1324 | Collaborative canvas; RBAC (PT read-only); usability testing |
| **4** | Jul 27Aug 07 | Installable PWA; zero-GPU fallback for legacy phones |
| **5** | Aug 1021 | Feedback instrumentation; performance/caching; cleanup |
| **6** | Aug 24Sep 02 | Beta launch to MSK cohort; telemetry analysis; Phase-2 backlog |
**Definition of success (pilot go/no-go):** zero workflow friction added; deterministic cross-device stability (incl. CPU-bound rendering); zero Decree 13 leak events; sustained clinician retention over external messaging; successful academic-to-production model migration.
*Source: [CONTEXT_VISION_SCOPE.md §56](../PLAN/CONTEXT_VISION_SCOPE.md).*
---
## 8. Investment Ask & Scope Boundaries
### 8.1 The Resource Envelope (qualitative)
| Category | Pilot ask | Rationale |
| :--- | :--- | :--- |
| **On-prem compute** | 1 × Dell PowerEdge-class node (K3s + DB VM) + 1 × GPU node (A10/T4, 24 GB VRAM) for Triton | Single-site air-gapped deployment; capex-only, no recurring SaaS |
| **Team** | Small cross-functional squad (backend/ML, frontend, infra/DevOps) + clinical champion (UP1/UP5) for validation | Matches the 7-sprint scope; clinical sponsor is essential for governance & buy-in |
| **Software** | Open-source stack (K3s, Triton, Postgres/pgvector, Redis, MinIO, Keycloak, Prometheus/Grafana) | Zero licensing cost; no vendor lock-in |
| **Governed cloud (PoC only)** | Time-boxed GCP Vertex AI (Gemini) + Modal (MedGemma) budget, capped at <20% consult usage | NFR-16a fallback; **retired at sign-off** not a recurring production cost |
| **Data** | Access to de-identified MSK ultrasound + MOH guideline corpus | Needed to tune grading thresholds and RAG index |
### 8.2 In Scope vs. Out of Scope (Pilot)
| In scope (pilot) | Out of scope (Phase 2+) |
| :--- | :--- |
| FR-25 knee synovitis grading (UP5 primary) | Multi-hospital federation / multi-region HA |
| Grad-CAM explainability, Socratic safety loop, RAG-Referee | Full UP7 surgeon planning & UP8 patient portal at production scale |
| Decree 13 + Circular 46 compliance, HITL sign-off, audit log | Predictive analytics (cartilage degeneration, hardware failure forecasting) |
| Single-hospital air-gapped deployment | Qdrant hot-tier vector cache (only if >100K vectors / >500 QPS) |
| PoC cloud LLM tiers (NFR-16a, retired at sign-off) | Permanent cloud dependency (explicitly prohibited by NFR-16) |
### 8.3 Go / No-Go Decision Criteria
At the end of the 3-month cycle, the leader decides on Phase 2 based on objective evidence:
1. **Safety:** zero Decree 13 leak events; 100% HITL-gated finalizations; 100% MOH-cited clinical text.
2. **Performance:** inference ≤ 1.5 s; availability ≥ 99.9%; onboarding ≤ 45 min.
3. **Adoption:** measurable clinician retention within the structured workspace vs. external messaging.
4. **Strategic:** VKIST models successfully running in a referenceable production form.
> **The investment is bounded and reversible:** one hospital, one quarter, open-source stack, no production cloud dependency. If the go/no-go criteria are not met, the blast radius is a single controlled pilot — not an enterprise commitment.
---
## Appendix — Source Documents
- Vision, personas, constraints, milestones: [CONTEXT_VISION_SCOPE.md](../PLAN/CONTEXT_VISION_SCOPE.md)
- Deep personas + quantitative baselines: [User_Research_Result.md](../Requirement_Analysis/USER_RESEARCH/User_Research_Result.md)
- Architecture, FR/NFR, C4 diagrams, build-vs-buy: [SOFTWARE_ARCHITECTURE_SPEC.md](../ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md), [SOLUTION_ARCHITECTURE_SPEC.md](../ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md)
- Use cases + 4-quadrant framework: [full_usecase_planuml.md](../../workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/Use_Case/full_usecase_planuml.md), [CONTEXT_FR_25_UC_ELIT.md](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/CONTEXT_FR_25_UC_ELIT.md)
- Risk scenarios: [FR_25_UC_DESIGN/](../Requirement_Analysis/UC_Design/FR_25_UC_DESIGN/) (Q1Q4 scenario docs)
- Impact narrative: [Motivation.md](../DEMO_EXP/Motivation.md)
*End of proposal.*

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -1,510 +0,0 @@
<!-- image -->
## Architectural design
## Objectives
The objective of this chapter is to introduce the concepts of software architecture and architectural design. When you have read the chapter, you will:
- ■ understand why the architectural design of software is important;
- ■ understand the decisions that have to be made about the system architecture during the architectural design process;
- ■ have been introduced to the idea of architectural patterns, well-tried ways of organizing system architectures, which can be reused in system designs;
- ■ know the architectural patterns that are often used in different types of application system, including transaction processing systems and language processing systems.
## Contents
- 6.1 Architectural design decisions
- 6.2 Architectural views
- 6.3 Architectural patterns
- 6.4 Application architectures
Architectural design is concerned with understanding how a system should be organized and designing the overall structure of that system. In the model of the software development process, as shown in Chapter 2, architectural design is the first stage in the software design process. It is the critical link between design and requirements engineering, as it identifies the main structural components in a system and the relationships between them. The output of the architectural design process is an architectural model that describes how the system is organized as a set of communicating components.
In agile processes, it is generally accepted that an early stage of the development process should be concerned with establishing an overall system architecture. Incremental development of architectures is not usually successful. While refactoring components in response to changes is usually relatively easy, refactoring a system architecture is likely to be expensive.
To help you understand what I mean by system architecture, consider Figure 6.1. This shows an abstract model of the architecture for a packing robot system that shows the components that have to be developed. This robotic system can pack different kinds of object. It uses a vision component to pick out objects on a conveyor, identify the type of object, and select the right kind of packaging. The system then moves objects from the delivery conveyor to be packaged. It places packaged objects on another conveyor. The architectural model shows these components and the links between them.
In practice, there is a significant overlap between the processes of requirements engineering and architectural design. Ideally, a system specification should not include any design information. This is unrealistic except for very small systems. Architectural decomposition is usually necessary to structure and organize the specification. Therefore, as part of the requirements engineering process, you might propose an abstract system architecture where you associate groups of system functions or features with large-scale components or sub-systems. You can then use this decomposition to discuss the requirements and features of the system with stakeholders.
You can design software architectures at two levels of abstraction, which I call architecture in the small and architecture in the large :
1. Architecture in the small is concerned with the architecture of individual programs. At this level, we are concerned with the way that an individual program is decomposed into components. This chapter is mostly concerned with program architectures.
2. Architecture in the large is concerned with the architecture of complex enterprise systems that include other systems, programs, and program components. These enterprise systems are distributed over different computers, which may be owned and managed by different companies. I cover architecture in the large in Chapters 18 and 19, where I discuss distributed systems architectures.
Figure 6.1 The architecture of a packing robot control system Software architecture is important because it affects the performance, robustness, distributability, and maintainability of a system (Bosch, 2000). As Bosch discusses, individual components implement the functional system requirements. The nonfunctional requirements depend on the system architecture-the way in which these components are organized and communicate. In many systems, non-functional requirements are also influenced by individual components, but there is no doubt that the architecture of the system is the dominant influence.
<!-- image -->
Bass et al. (2003) discuss three advantages of explicitly designing and documenting software architecture:
1. Stakeholder communication The architecture is a high-level presentation of the system that may be used as a focus for discussion by a range of different stakeholders.
2. System analysis Making the system architecture explicit at an early stage in the system development requires some analysis. Architectural design decisions have a profound effect on whether or not the system can meet critical requirements such as performance, reliability, and maintainability.
3. Large-scale reuse A model of a system architecture is a compact, manageable description of how a system is organized and how the components interoperate. The system architecture is often the same for systems with similar requirements and so can support large-scale software reuse. As I explain in Chapter 16, it may be possible to develop product-line architectures where the same architecture is reused across a range of related systems.
Hofmeister et al. (2000) propose that a software architecture can serve firstly as a design plan for the negotiation of system requirements, and secondly as a means of structuring discussions with clients, developers, and managers. They also suggest that it is an essential tool for complexity management. It hides details and allows the designers to focus on the key system abstractions.
System architectures are often modeled using simple block diagrams, as in Figure 6.1. Each box in the diagram represents a component. Boxes within boxes indicate that the component has been decomposed to sub-components. Arrows mean that data and or control signals are passed from component to component in the direction of the arrows. Y ou can see many examples of this type of architectural model in Booch's software architecture catalog (Booch, 2009).
Block diagrams present a high-level picture of the system structure, which people from different disciplines, who are involved in the system development process, can readily understand. However, in spite of their widespread use, Bass et al. (2003) dislike informal block diagrams for describing an architecture. They claim that these informal diagrams are poor architectural representations, as they show neither the type of the relationships among system components nor the components' externally visible properties.
The apparent contradictions between practice and architectural theory arise because there are two ways in which an architectural model of a program is used:
1. As a way of facilitating discussion about the system design A high-level architectural view of a system is useful for communication with system stakeholders and project planning because it is not cluttered with detail. Stakeholders can relate to it and understand an abstract view of the system. They can then discuss the system as a whole without being confused by detail. The architectural model identifies the key components that are to be developed so managers can start assigning people to plan the development of these systems.
2. As a way of documenting an architecture that has been designed The aim here is to produce a complete system model that shows the different components in a system, their interfaces, and their connections. The argument for this is that such a detailed architectural description makes it easier to understand and evolve the system.
Block diagrams are an appropriate way of describing the system architecture during the design process, as they are a good way of supporting communications between the people involved in the process. In many projects, these are often the only architectural documentation that exists. However, if the architecture of a system is to be thoroughly documented then it is better to use a notation with well-defined semantics for architectural description. However, as I discuss in Section 6.2, some people think that detailed documentation is neither useful, nor really worth the cost of its development.
## 6.1 Architectural design decisions
Architectural design is a creative process where you design a system organization that will satisfy the functional and non-functional requirements of a system. Because it is a creative process, the activities within the process depend on the type of system being developed, the background and experience of the system architect, and the specific requirements for the system. It is therefore useful to think of architectural design as a series of decisions to be made rather than a sequence of activities.
During the architectural design process, system architects have to make a number of structural decisions that profoundly affect the system and its development process. Based on their knowledge and experience, they have to consider the following fundamental questions about the system:
1. Is there a generic application architecture that can act as a template for the system that is being designed?
2. How will the system be distributed across a number of cores or processors?
3. What architectural patterns or styles might be used?
4. What will be the fundamental approach used to structure the system?
5. How will the structural components in the system be decomposed into subcomponents?
6. What strategy will be used to control the operation of the components in the system?
7. What architectural organization is best for delivering the non-functional requirements of the system?
8. How will the architectural design be evaluated?
9. How should the architecture of the system be documented?
Although each software system is unique, systems in the same application domain often have similar architectures that reflect the fundamental concepts of the domain. For example, application product lines are applications that are built around a core architecture with variants that satisfy specific customer requirements. When designing a system architecture, you have to decide what your system and broader application classes have in common, and decide how much knowledge from these application architectures you can reuse. I discuss generic application architectures in Section 6.4 and application product lines in Chapter 16.
For embedded systems and systems designed for personal computers, there is usually only a single processor and you will not have to design a distributed architecture for the system. However, most large systems are now distributed systems in which the system software is distributed across many different computers. The choice of distribution architecture is a key decision that affects the performance and reliability of the system. This is a major topic in its own right and I cover it separately in Chapter 18.
The architecture of a software system may be based on a particular architectural pattern or style. An architectural pattern is a description of a system organization (Garlan and Shaw, 1993), such as a client-server organization or a layered architecture. Architectural patterns capture the essence of an architecture that has been used in different software systems. You should be aware of common patterns, where they can be used, and their strengths and weaknesses when making decisions about the architecture of a system. I discuss a number of frequently used patterns in Section 6.3.
Garlan and Shaw's notion of an architectural style (style and pattern have come to mean the same thing) covers questions 4 to 6 in the previous list. You have to choose the most appropriate structure, such as client-server or layered structuring, that will enable you to meet the system requirements. To decompose structural system units, you decide on the strategy for decomposing components into sub-components. The approaches that you can use allow different types of architecture to be implemented. Finally, in the control modeling process, you make decisions about how the execution of components is controlled. You develop a general model of the control relationships between the various parts of the system.
Because of the close relationship between non-functional requirements and software architecture, the particular architectural style and structure that you choose for a system should depend on the non-functional system requirements:
1. Performance If performance is a critical requirement, the architecture should be designed to localize critical operations within a small number of components, with these components all deployed on the same computer rather than distributed across the network. This may mean using a few relatively large components rather than small, fine-grain components, which reduces the number of component communications. You may also consider run-time system organizations that allow the system to be replicated and executed on different processors.
2. Security If security is a critical requirement, a layered structure for the architecture should be used, with the most critical assets protected in the innermost layers, with a high level of security validation applied to these layers.
3. Safety If safety is a critical requirement, the architecture should be designed so that safety-related operations are all located in either a single component or in a small number of components. This reduces the costs and problems of safety validation and makes it possible to provide related protection systems that can safely shut down the system in the event of failure.
4. Availability If availability is a critical requirement, the architecture should be designed to include redundant components so that it is possible to replace and update components without stopping the system. I describe two fault-tolerant system architectures for high-availability systems in Chapter 13.
5. Maintainability If maintainability is a critical requirement, the system architecture should be designed using fine-grain, self-contained components that may
readily be changed. Producers of data should be separated from consumers and shared data structures should be avoided.
Obviously there is potential conflict between some of these architectures. For example, using large components improves performance and using small, fine-grain components improves maintainability. If both performance and maintainability are important system requirements, then some compromise must be found. This can sometimes be achieved by using different architectural patterns or styles for different parts of the system.
Evaluating an architectural design is difficult because the true test of an architecture is how well the system meets its functional and non-functional requirements when it is in use. However, you can do some evaluation by comparing your design against reference architectures or generic architectural patterns. Bosch's (2000) description of the non-functional characteristics of architectural patterns can also be used to help with architectural evaluation.
## 6.2 Architectural views
I explained in the introduction to this chapter that architectural models of a software system can be used to focus discussion about the software requirements or design. Alternatively, they may be used to document a design so that it can be used as a basis for more detailed design and implementation, and for the future evolution of the system. In this section, I discuss two issues that are relevant to both of these:
1. What views or perspectives are useful when designing and documenting a system's architecture?
2. What notations should be used for describing architectural models?
It is impossible to represent all relevant information about a system's architecture in a single architectural model, as each model only shows one view or perspective of the system. It might show how a system is decomposed into modules, how the run-time processes interact, or the different ways in which system components are distributed across a network. All of these are useful at different times so, for both design and documentation, you usually need to present multiple views of the software architecture.
There are different opinions as to what views are required. Krutchen (1995), in his well-known 4+1 view model of software architecture, suggests that there should be four fundamental architectural views, which are related using use cases or scenarios. The views that he suggests are:
1. A logical view, which shows the key abstractions in the system as objects or object classes. It should be possible to relate the system requirements to entities in this logical view.
2. A process view, which shows how, at run-time, the system is composed of interacting processes. This view is useful for making judgments about nonfunctional system characteristics such as performance and availability.
3. A development view, which shows how the software is decomposed for development, that is, it shows the breakdown of the software into components that are implemented by a single developer or development team. This view is useful for software managers and programmers.
4. A physical view, which shows the system hardware and how software components are distributed across the processors in the system. This view is useful for systems engineers planning a system deployment.
Hofmeister et al. (2000) suggest the use of similar views but add to this the notion of a conceptual view. This view is an abstract view of the system that can be the basis for decomposing high-level requirements into more detailed specifications, help engineers make decisions about components that can be reused, and represent a product line (discussed in Chapter 16) rather than a single system. Figure 6.1, which describes the architecture of a packing robot, is an example of a conceptual system view.
In practice, conceptual views are almost always developed during the design process and are used to support architectural decision making. They are a way of communicating the essence of a system to different stakeholders. During the design process, some of the other views may also be developed when different aspects of the system are discussed, but there is no need for a complete description from all perspectives. It may also be possible to associate architectural patterns, discussed in the next section, with the different views of a system.
There are differing views about whether or not software architects should use the UML for architectural description (Clements, et al., 2002). A survey in 2006 (Lange et al., 2006) showed that, when the UML was used, it was mostly applied in a loose and informal way. The authors of that paper argued that this was a bad thing. I disagree with this view. The UML was designed for describing object-oriented systems and, at the architectural design stage, you often want to describe systems at a higher level of abstraction. Object classes are too close to the implementation to be useful for architectural description.
I don't find the UML to be useful during the design process itself and prefer informal notations that are quicker to write and which can be easily drawn on a whiteboard. The UML is of most value when you are documenting an architecture in detail or using model-driven development, as discussed in Chapter 5.
A number of researchers have proposed the use of more specialized architectural description languages (ADLs) (Bass et al., 2003) to describe system architectures. The basic elements of ADLs are components and connectors, and they include rules and guidelines for well-formed architectures. However, because of their specialized nature, domain and application specialists find it hard to understand and use ADLs. This makes it difficult to assess their usefulness for practical software engineering. ADLs designed for a particular domain (e.g., automobile systems) may be used as a
| Name | MVC (Model-View-Controller) |
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | Separates presentation and interaction from the system data. The system is structured into three logical components that interact with each other. The Model component manages the system data and associated operations on that data. The View component defines and manages how the data is presented to the user. The Controller component manages user interaction (e.g., key presses, mouse clicks, etc.) and passes these interactions to the View and the Model. See Figure 6.3. |
| Example | Figure 6.4 shows the architecture of a web-based application system organized using the MVC pattern. |
| When used | Used when there are multiple ways to view and interact with data. Also used when the future requirements for interaction and presentation of data are unknown. |
| Advantages | Allows the data to change independently of its representation and vice versa. Supports presentation of the same data in different ways with changes made in one representation shown in all of them. |
| Disadvantages | Can involve additional code and code complexity when the data model and interactions are simple. |
Figure 6.2 The modelview-controller (MVC) pattern basis for model-driven development. However, I believe that informal models and notations, such as the UML, will remain the most commonly used ways of documenting system architectures.
Users of agile methods claim that detailed design documentation is mostly unused. It is, therefore, a waste of time and money to develop it. I largely agree with this view and I think that, for most systems, it is not worth developing a detailed architectural description from these four perspectives. You should develop the views that are useful for communication and not worry about whether or not your architectural documentation is complete. However, an exception to this is when you are developing critical systems, when you need to make a detailed dependability analysis of the system. You may need to convince external regulators that your system conforms to their regulations and so complete architectural documentation may be required.
## 6.3 Architectural patterns
The idea of patterns as a way of presenting, sharing, and reusing knowledge about software systems is now widely used. The trigger for this was the publication of a book on object-oriented design patterns (Gamma et al., 1995), which prompted the development of other types of pattern, such as patterns for organizational design (Coplien and Harrison, 2004), usability patterns (Usability Group, 1998), interaction (Martin and Sommerville, 2004), configuration management (Berczuk and Appleton, 2002), and so on. Architectural patterns were proposed in the 1990s under the name 'architectural styles' (Shaw and Garlan, 1996), with a five-volume series of handbooks on pattern-oriented software architecture published between 1996 and 2007 (Buschmann et al., 1996; Buschmann et al., 2007a; Buschmann et al., 2007b; Kircher and Jain, 2004; Schmidt et al., 2000).
Figure 6.3 The organization of the MVC
<!-- image -->
In this section, I introduce architectural patterns and briefly describe a selection of architectural patterns that are commonly used in different types of systems. For more information about patterns and their use, you should refer to published pattern handbooks.
You can think of an architectural pattern as a stylized, abstract description of good practice, which has been tried and tested in different systems and environments. So, an architectural pattern should describe a system organization that has been successful in previous systems. It should include information of when it is and is not appropriate to use that pattern, and the pattern's strengths and weaknesses.
For example, Figure 6.2 describes the well-known Model-View-Controller pattern. This pattern is the basis of interaction management in many web-based systems. The stylized pattern description includes the pattern name, a brief description (with an associated graphical model), and an example of the type of system where the pattern is used (again, perhaps with a graphical model). You should also include information about when the pattern should be used and its advantages and disadvantages. Graphical models of the architecture associated with the MVC pattern are shown in Figures 6.3 and 6.4. These present the architecture from different views-Figure 6.3 is a conceptual view and Figure 6.4 shows a possible run-time architecture when this pattern is used for interaction management in a web-based system.
In a short section of a general chapter, it is impossible to describe all of the generic patterns that can be used in software development. Rather, I present some selected examples of patterns that are widely used and which capture good architectural design principles. I have included some further examples of generic architectural patterns on the book's web pages.
Figure 6.4 Web application architecture using the MVC pattern
<!-- image -->
<!-- image -->
## 6.3.1 Layered architecture
The notions of separation and independence are fundamental to architectural design because they allow changes to be localized. The MVC pattern, shown in Figure 6.2, separates elements of a system, allowing them to change independently. For example, adding a new view or changing an existing view can be done without any changes to the underlying data in the model. The layered architecture pattern is another way of achieving separation and independence. This pattern is shown in Figure 6.5. Here, the system functionality is organized into separate layers, and each layer only relies on the facilities and services offered by the layer immediately beneath it.
This layered approach supports the incremental development of systems. As a layer is developed, some of the services provided by that layer may be made available to users. The architecture is also changeable and portable. So long as its interface is unchanged, a layer can be replaced by another, equivalent layer. Furthermore, when layer interfaces change or new facilities are added to a layer, only the adjacent layer is affected. As layered systems localize machine dependencies in inner layers, this makes it easier to provide multi-platform implementations of an application system. Only the inner, machine-dependent layers need be re-implemented to take account of the facilities of a different operating system or database.
Figure 6.6 is an example of a layered architecture with four layers. The lowest layer includes system support software-typically database and operating system support. The next layer is the application layer that includes the components concerned with the application functionality and utility components that are used by other application components. The third layer is concerned with user interface
| Name | Layered architecture |
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | Organizes the system into layers with related functionality associated with each layer. A layer provides services to the layer above it so the lowest-level layers represent core services that are likely to be used throughout the system. See Figure 6.6. |
| Example | A layered model of a system for sharing copyright documents held in different libraries, as shown in Figure 6.7. |
| When used | Used when building new facilities on top of existing systems; when the development is spread across several teams with each team responsibility for a layer of functionality; when there is a requirement for multi-level security. |
| Advantages | Allows replacement of entire layers so long as the interface is maintained. Redundant facilities (e.g., authentication) can be provided in each layer to increase the dependability of the system. |
| Disadvantages | In practice, providing a clean separation between layers is often difficult and a high-level layer may have to interact directly with lower-level layers rather than through the layer immediately below it. Performance can be a problem because of multiple levels of interpretation of a service request as it is processed at each layer. |
## Figure 6.5 The layered architecture pattern
management and providing user authentication and authorization, with the top layer providing user interface facilities. Of course, the number of layers is arbitrary. Any of the layers in Figure 6.6 could be split into two or more layers.
Figure 6.7 is an example of how this layered architecture pattern can be applied to a library system called LIBSYS, which allows controlled electronic access to copyright material from a group of university libraries. This has a five-layer architecture, with the bottom layer being the individual databases in each library.
You can see another example of the layered architecture pattern in Figure 6.17 (found in Section 6.4). This shows the organization of the system for mental healthcare (MHC-PMS) that I have discussed in earlier chapters.
<!-- image -->
Figure 6.7 The architecture of the LIBSYS system
<!-- image -->
## 6.3.2 Repository architecture
The layered architecture and MVC patterns are examples of patterns where the view presented is the conceptual organization of a system. My next example, the Repository pattern (Figure 6.8), describes how a set of interacting components can share data.
Figure 6.8 The repository pattern The majority of systems that use large amounts of data are organized around a shared database or repository. This model is therefore suited to applications in which data is generated by one component and used by another. Examples of this type of system include command and control systems, management information systems, CAD systems, and interactive development environments for software.
| Name | Repository |
|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | All data in a system is managed in a central repository that is accessible to all system components. Components do not interact directly, only through the repository. |
| Example | Figure 6.9 is an example of an IDE where the components use a repository of system design information. Each software tool generates information which is then available for use by other tools. |
| When used | You should use this pattern when you have a system in which large volumes of information are generated that has to be stored for a long time. You may also use it in data-driven systems where the inclusion of data in the repository triggers an action or tool. |
| Advantages | Components can be independent-they do not need to know of the existence of other components. Changes made by one component can be propagated to all components. All data can be managed consistently (e.g., backups done at the same time) as it is all in one place. |
| Disadvantages | The repository is a single point of failure so problems in the repository affect the whole system. May be inefficiencies in organizing all communication through the repository. Distributing the repository across several computers may be difficult. |
<!-- image -->
Figure 6.9 is an illustration of a situation in which a repository might be used. This diagram shows an IDE that includes different tools to support model-driven development. The repository in this case might be a version-controlled environment (as discussed in Chapter 25) that keeps track of changes to software and allows rollback to earlier versions.
Organizing tools around a repository is an efficient way to share large amounts of data. There is no need to transmit data explicitly from one component to another. However, components must operate around an agreed repository data model. Inevitably, this is a compromise between the specific needs of each tool and it may be difficult or impossible to integrate new components if their data models do not fit the agreed schema. In practice, it may be difficult to distribute the repository over a number of machines. Although it is possible to distribute a logically centralized repository, there may be problems with data redundancy and inconsistency.
In the example shown in Figure 6.9, the repository is passive and control is the responsibility of the components using the repository. An alternative approach, which has been derived for AI systems, uses a 'blackboard' model that triggers components when particular data become available. This is appropriate when the form of the repository data is less well structured. Decisions about which tool to activate can only be made when the data has been analyzed. This model is introduced by Nii (1986). Bosch (2000) includes a good discussion of how this style relates to system quality attributes.
## 6.3.3 Client-server architecture
The repository pattern is concerned with the static structure of a system and does not show its run-time organization. My next example illustrates a very commonly used run-time organization for distributed systems. The Client-server pattern is described in Figure 6.10.
| Name | Client-server |
|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | In a client-server architecture, the functionality of the system is organized into services, with each service delivered from a separate server. Clients are users of these services and access servers to make use of them. |
| Example | Figure 6.11 is an example of a film and video/DVD library organized as a client-server system. |
| When used | Used when data in a shared database has to be accessed from a range of locations. Because servers can be replicated, may also be used when the load on a system is variable. |
| Advantages | The principal advantage of this model is that servers can be distributed across a network. General functionality (e.g., a printing service) can be available to all clients and does not need to be implemented by all services. |
| Disadvantages | Each service is a single point of failure so susceptible to denial of service attacks or server failure. Performance may be unpredictable because it depends on the network as well as the system. May be management problems if servers are owned by different organizations. |
Figure 6.10 The client-server pattern A system that follows the client-server pattern is organized as a set of services and associated servers, and clients that access and use the services. The major components of this model are:
1. A set of servers that offer services to other components. Examples of servers include print servers that offer printing services, file servers that offer file management services, and a compile server, which offers programming language compilation services.
2. A set of clients that call on the services offered by servers. There will normally be several instances of a client program executing concurrently on different computers.
3. A network that allows the clients to access these services. Most client-server systems are implemented as distributed systems, connected using Internet protocols.
Client-server architectures are usually thought of as distributed systems architectures but the logical model of independent services running on separate servers can be implemented on a single computer. Again, an important benefit is separation and independence. Services and servers can be changed without affecting other parts of the system.
Clients may have to know the names of the available servers and the services that they provide. However, servers do not need to know the identity of clients or how many clients are accessing their services. Clients access the services provided by a server through remote procedure calls using a request-reply protocol such as the http protocol used in the WWW. Essentially, a client makes a request to a server and waits until it receives a reply.
Figure 6.11 A clientserver architecture for a film library
Figure 6.12 The pipe and filter pattern
<!-- image -->
Figure 6.11 is an example of a system that is based on the client-server model. This is a multi-user, web-based system for providing a film and photograph library. In this system, several servers manage and display the different types of media. Video frames need to be transmitted quickly and in synchrony but at relatively low resolution. They may be compressed in a store, so the video server can handle video compression and decompression in different formats. Still pictures, however, must be maintained at a high resolution, so it is appropriate to maintain them on a separate server.
The catalog must be able to deal with a variety of queries and provide links into the web information system that includes data about the film and video clips, and an e-commerce system that supports the sale of photographs, film, and video clips. The
| Name | Pipe and filter |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | The processing of the data in a system is organized so that each processing component (filter) is discrete and carries out one type of data transformation. The data flows (as in a pipe) from one component to another for processing. |
| Example | Figure 6.13 is an example of a pipe and filter system used for processing invoices. |
| When used | Commonly used in data processing applications (both batch- and transaction-based) where inputs are processed in separate stages to generate related outputs. |
| Advantages | Easy to understand and supports transformation reuse. Workflow style matches the structure of many business processes. Evolution by adding transformations is straightforward. Can be implemented as either a sequential or concurrent system. |
| Disadvantages | The format for data transfer has to be agreed upon between communicating transformations. Each transformation must parse its input and unparse its output to the agreed form. This increases system overhead and may mean that it is impossible to reuse functional transformations that use incompatible data structures. |
Figure 6.13 An example of the pipe and filter architecture
<!-- image -->
client program is simply an integrated user interface, constructed using a web browser, to access these services.
The most important advantage of the client-server model is that it is a distributed architecture. Effective use can be made of networked systems with many distributed processors. It is easy to add a new server and integrate it with the rest of the system or to upgrade servers transparently without affecting other parts of the system. I discuss distributed architectures, including client-server architectures and distributed object architectures, in Chapter 18.
## 6.3.4 Pipe and filter architecture
My final example of an architectural pattern is the pipe and filter pattern. This is a model of the run-time organization of a system where functional transformations process their inputs and produce outputs. Data flows from one to another and is transformed as it moves through the sequence. Each processing step is implemented as a transform. Input data flows through these transforms until converted to output. The transformations may execute sequentially or in parallel. The data can be processed by each transform item by item or in a single batch.
The name 'pipe and filter' comes from the original Unix system where it was possible to link processes using 'pipes'. These passed a text stream from one process to another. Systems that conform to this model can be implemented by combining Unix commands, using pipes and the control facilities of the Unix shell. The term 'filter' is used because a transformation 'filters out' the data it can process from its input data stream.
Variants of this pattern have been in use since computers were first used for automatic data processing. When transformations are sequential with data processed in batches, this pipe and filter architectural model becomes a batch sequential model, a common architecture for data processing systems (e.g., a billing system). The architecture of an embedded system may also be organized as a process pipeline, with each process executing concurrently. I discuss the use of this pattern in embedded systems in Chapter 20.
An example of this type of system architecture, used in a batch processing application, is shown in Figure 6.13. An organization has issued invoices to customers. Once a week, payments that have been made are reconciled with the invoices. For
<!-- image -->
## Architectural patterns for control
There are specific architectural patterns that reflect commonly used ways of organizing control in a system. These include centralized control, based on one component calling other components, and event-based control, where the system reacts to external events.
http://www.SoftwareEngineering-9.com/Web/Architecture/ArchPatterns/
those invoices that have been paid, a receipt is issued. For those invoices that have not been paid within the allowed payment time, a reminder is issued.
Interactive systems are difficult to write using the pipe and filter model because of the need for a stream of data to be processed. Although simple textual input and output can be modeled in this way, graphical user interfaces have more complex I/O formats and a control strategy that is based on events such as mouse clicks or menu selections. It is difficult to translate this into a form compatible with the pipelining model.
## 6.4 Application architectures
Application systems are intended to meet a business or organizational need. All businesses have much in common-they need to hire people, issue invoices, keep accounts, and so on. Businesses operating in the same sector use common sectorspecific applications. Therefore, as well as general business functions, all phone companies need systems to connect calls, manage their network, issue bills to customers, etc. Consequently, the application systems used by these businesses also have much in common.
These commonalities have led to the development of software architectures that describe the structure and organization of particular types of software systems. Application architectures encapsulate the principal characteristics of a class of systems. For example, in real-time systems, there might be generic architectural models of different system types, such as data collection systems or monitoring systems. Although instances of these systems differ in detail, the common architectural structure can be reused when developing new systems of the same type.
The application architecture may be re-implemented when developing new systems but, for many business systems, application reuse is possible without reimplementation. We see this in the growth of Enterprise Resource Planning (ERP) systems from companies such as SAP and Oracle, and vertical software packages (COTS) for specialized applications in different areas of business. In these systems, a generic system is configured and adapted to create a specific business application.
<!-- image -->
## Application architectures
There are several examples of application architectures on the book's website. These include descriptions of batch data-processing systems, resource allocation systems, and event-based editing systems.
http://www.SoftwareEngineering-9.com/Web/Architecture/AppArch/
For example, a system for supply chain management can be adapted for different types of suppliers, goods, and contractual arrangements.
As a software designer, you can use models of application architectures in a number of ways:
1. As a starting point for the architectural design process If you are unfamiliar with the type of application that you are developing, you can base your initial design on a generic application architecture. Of course, this will have to be specialized for the specific system being developed, but it is a good starting point for design.
2. As a design checklist If you have developed an architectural design for an application system, you can compare this with the generic application architecture. You can check that your design is consistent with the generic architecture.
3. As a way of organizing the work of the development team The application architectures identify stable structural features of the system architectures and in many cases, it is possible to develop these in parallel. You can assign work to group members to implement different components within the architecture.
4. As a means of assessing components for reuse If you have components you might be able to reuse, you can compare these with the generic structures to see whether there are comparable components in the application architecture.
5. As a vocabulary for talking about types of applications If you are discussing a specific application or trying to compare applications of the same types, then you can use the concepts identified in the generic architecture to talk about the applications.
There are many types of application system and, in some cases, they may seem to be very different. However, many of these superficially dissimilar applications actually have much in common, and thus can be represented by a single abstract application architecture. I illustrate this here by describing the following architectures of two types of application:
1. Transaction processing applications Transaction processing applications are database-centered applications that process user requests for information and update the information in a database. These are the most common type of interactive business systems. They are organized in such a way that user actions can't interfere with each other and the integrity of the database is maintained. This
<!-- image -->
<!-- image -->
- class of system includes interactive banking systems, e-commerce systems, information systems, and booking systems.
2. Language processing systems Language processing systems are systems in which the user's intentions are expressed in a formal language (such as Java). The language processing system processes this language into an internal format and then interprets this internal representation. The best-known language processing systems are compilers, which translate high-level language programs into machine code. However, language processing systems are also used to interpret command languages for databases and information systems, and markup languages such as XML (Harold and Means, 2002; Hunter et al., 2007).
I have chosen these particular types of system because a large number of webbased business systems are transaction-processing systems, and all software development relies on language processing systems.
## 6.4.1 Transaction processing systems
Transaction processing (TP) systems are designed to process user requests for information from a database, or requests to update a database (Lewis et al., 2003). Technically, a database transaction is sequence of operations that is treated as a single unit (an atomic unit). All of the operations in a transaction have to be completed before the database changes are made permanent. This ensures that failure of operations within the transaction does not lead to inconsistencies in the database.
From a user perspective, a transaction is any coherent sequence of operations that satisfies a goal, such as 'find the times of flights from London to Paris'. If the user transaction does not require the database to be changed then it may not be necessary to package this as a technical database transaction.
An example of a transaction is a customer request to withdraw money from a bank account using an ATM. This involves getting details of the customer's account, checking the balance, modifying the balance by the amount withdrawn, and sending commands to the ATM to deliver the cash. Until all of these steps have been completed, the transaction is incomplete and the customer accounts database is not changed.
Transaction processing systems are usually interactive systems in which users make asynchronous requests for service. Figure 6.14 illustrates the conceptual architectural structure of TP applications. First a user makes a request to the system through an I/O processing component. The request is processed by some applicationspecific logic. A transaction is created and passed to a transaction manager, which is usually embedded in the database management system. After the transaction manager has ensured that the transaction is properly completed, it signals to the application that processing has finished.
Figure 6.15 The software architecture of an ATM system
<!-- image -->
<!-- image -->
Transaction processing systems may be organized as a 'pipe and filter' architecture with system components responsible for input, processing, and output. For example, consider a banking system that allows customers to query their accounts and withdraw cash from an ATM. The system is composed of two cooperating software components-the ATM software and the account processing software in the bank's database server. The input and output components are implemented as software in the ATM and the processing component is part of the bank's database server. Figure 6.15 shows the architecture of this system, illustrating the functions of the input, process, and output components.
## 6.4.2 Information systems
All systems that involve interaction with a shared database can be considered to be transaction-based information systems. An information system allows controlled access to a large base of information, such as a library catalog, a flight timetable, or the records of patients in a hospital. Increasingly, information systems are web-based systems that are accessed through a web browser.
Figure 6.16 a very general model of an information system. The system is modeled using a layered approach (discussed in Section 6.3) where the top layer supports the user interface and the bottom layer is the system database. The user communications layer handles all input and output from the user interface, and the information retrieval layer includes application-specific logic for accessing and updating the database. As we shall see later, the layers in this model can map directly onto servers in an Internet-based system.
As an example of an instantiation of this layered model, Figure 6.17 shows the architecture of the MHC-PMS. Recall that this system maintains and manages details of patients who are consulting specialist doctors about mental health problems. I have Figure 6.16 Layered information system architecture added detail to each layer in the model by identifying the components that support user communications and information retrieval and access:
<!-- image -->
1. The top layer is responsible for implementing the user interface. In this case, the UI has been implemented using a web browser.
2. The second layer provides the user interface functionality that is delivered through the web browser. It includes components to allow users to log in to the system and checking components that ensure that the operations they use are allowed by their role. This layer includes form and menu management components that present information to users, and data validation components that check information consistency.
3. The third layer implements the functionality of the system and provides components that implement system security, patient information creation and updating, import and export of patient data from other databases, and report generators that create management reports.
<!-- image -->
4. Finally, the lowest layer, which is built using a commercial database management system, provides transaction management and persistent data storage.
Information and resource management systems are now usually web-based systems where the user interfaces are implemented using a web browser. For example, e-commerce systems are Internet-based resource management systems that accept electronic orders for goods or services and then arrange delivery of these goods or services to the customer. In an e-commerce system, the application-specific layer includes additional functionality supporting a 'shopping cart' in which users can place a number of items in separate transactions, then pay for them all together in a single transaction.
The organization of servers in these systems usually reflects the four-layer generic model presented in Figure 6.16. These systems are often implemented as multi-tier client server/architectures, as discussed in Chapter 18:
1. The web server is responsible for all user communications, with the user interface implemented using a web browser;
2. The application server is responsible for implementing application-specific logic as well as information storage and retrieval requests;
3. The database server moves information to and from the database and handles transaction management.
Using multiple servers allows high throughput and makes it possible to handle hundreds of transactions per minute. As demand increases, servers can be added at each level to cope with the extra processing involved.
## 6.4.3 Language processing systems
Language processing systems translate a natural or artificial language into another representation of that language and, for programming languages, may also execute the resulting code. In software engineering, compilers translate an artificial programming language into machine code. Other language-processing systems may translate an XML data description into commands to query a database or to an alternative XML representation. Natural language processing systems may translate one natural language to another e.g., French to Norwegian.
A possible architecture for a language processing system for a programming language is illustrated in Figure 6.18. The source language instructions define the program to be executed and a translator converts these into instructions for an abstract machine. These instructions are then interpreted by another component that fetches the instructions for execution and executes them using (if necessary) data from the environment. The output of the process is the result of interpreting the instructions on the input data.
<!-- image -->
Of course, for many compilers, the interpreter is a hardware unit that processes machine instructions and the abstract machine is a real processor. However, for dynamically typed languages, such as Python, the interpreter may be a software component.
Programming language compilers that are part of a more general programming environment have a generic architecture (Figure 6.19) that includes the following components:
1. A lexical analyzer, which takes input language tokens and converts them to an internal form.
2. A symbol table, which holds information about the names of entities (variables, class names, object names, etc.) used in the text that is being translated.
3. A syntax analyzer, which checks the syntax of the language being translated. It uses a defined grammar of the language and builds a syntax tree.
4. A syntax tree, which is an internal structure representing the program being compiled.
<!-- image -->
Figure 6.19 A pipe and filter compiler architecture
<!-- image -->
## Reference architectures
Reference architectures capture important features of system architectures in a domain. Essentially, they include everything that might be in an application architecture although, in reality, it is very unlikely that any individual application would include all the features shown in a reference architecture. The main purpose of reference architectures is to evaluate and compare design proposals, and to educate people about architectural characteristics in that domain.
http://www.SoftwareEngineering-9.com/Web/Architecture/RefArch.html
5. A semantic analyzer that uses information from the syntax tree and the symbol table to check the semantic correctness of the input language text.
6. A code generator that 'walks' the syntax tree and generates abstract machine code.
Other components might also be included which analyze and transform the syntax tree to improve efficiency and remove redundancy from the generated machine code. In other types of language processing system, such as a natural language translator, there will be additional components such as a dictionary, and the generated code is actually the input text translated into another language.
There are alternative architectural patterns that may be used in a language processing system (Garlan and Shaw, 1993). Compilers can be implemented using a composite of a repository and a pipe and filter model. In a compiler architecture, the symbol table is a repository for shared data. The phases of lexical, syntactic, and semantic analysis are organized sequentially, as shown in Figure 6.19, and communicate through the shared symbol table.
This pipe and filter model of language compilation is effective in batch environments where programs are compiled and executed without user interaction; for example, in the translation of one XML document to another. It is less effective when a compiler is integrated with other language processing tools such as a structured editing system, an interactive debugger or a program prettyprinter. In this situation, changes from one component need to be reflected immediately in other components. It is better, therefore, to organize the system around a repository, as shown in Figure 6.20.
This figure illustrates how a language processing system can be part of an integrated set of programming support tools. In this example, the symbol table and syntax tree act as a central information repository. Tools or tool fragments communicate through it. Other information that is sometimes embedded in tools, such as the grammar definition and the definition of the output format for the program, have been taken out of the tools and put into the repository. Therefore, a syntax-directed editor can check that the syntax of a program is correct as it is being typed and a prettyprinter can create listings of the program in a format that is easy to read.
<!-- image -->
<!-- image -->
## KEY POINTS
- ■ A software architecture is a description of how a software system is organized. Properties of a system such as performance, security, and availability are influenced by the architecture used.
- ■ Architectural design decisions include decisions on the type of application, the distribution of the system, the architectural styles to be used, and the ways in which the architecture should be documented and evaluated.
- ■ Architectures may be documented from several different perspectives or views. Possible views include a conceptual view, a logical view, a process view, a development view, and a physical view.
- ■ Architectural patterns are a means of reusing knowledge about generic system architectures. They describe the architecture, explain when it may be used, and discuss its advantages and disadvantages.
- ■ Commonly used architectural patterns include Model-View-Controller, Layered Architecture, Repository, Client-server, and Pipe and Filter.
- ■ Generic models of application systems architectures help us understand the operation of applications, compare applications of the same type, validate application system designs, and assess large-scale components for reuse.
- ■ Transaction processing systems are interactive systems that allow information in a database to be remotely accessed and modified by a number of users. Information systems and resource management systems are examples of transaction processing systems.
- ■ Language processing systems are used to translate texts from one language into another and to carry out the instructions specified in the input language. They include a translator and an abstract machine that executes the generated language.
## FURTHER READING
Software Architecture: Perspectives on an Emerging Discipline. This was the first book on software architecture and has a good discussion on different architectural styles. (M. Shaw and D. Garlan, Prentice-Hall, 1996.)
Software Architecture in Practice, 2nd ed. This is a practical discussion of software architectures that does not oversell the benefits of architectural design. It provides a clear business rationale explaining why architectures are important. (L. Bass, P. Clements and R. Kazman, Addison-Wesley, 2003.)
'The Golden Age of Software Architecture' This paper surveys the development of software architecture from its beginnings in the 1980s through to its current usage. There is little technical content but it is an interesting historical overview. (M. Shaw and P. Clements, IEEE Software, 21 (2), March-April 2006.) http://dx.doi.org/10.1109/MS.2006.58.
Handbook of Software Architecture . This is a work in progress by Grady Booch, one of the early evangelists for software architecture. He has been documenting the architectures of a range of software systems so you can see reality rather than academic abstraction. Available on the Web and intended to appear as a book. http://www.handbookofsoftwarearchitecture.com/.
## EXERCISES
- 6.1. When describing a system, explain why you may have to design the system architecture before the requirements specification is complete.
- 6.2. You have been asked to prepare and deliver a presentation to a non-technical manager to justify the hiring of a system architect for a new project. Write a list of bullet points setting out the key points in your presentation. Naturally, you have to explain what is meant by system architecture.
- 6.3. Explain why design conflicts might arise when designing an architecture for which both availability and security requirements are the most important non-functional requirements.
- 6.4. Draw diagrams showing a conceptual view and a process view of the architectures of the following systems:
An automated ticket-issuing system used by passengers at a railway station.
A computer-controlled video conferencing system that allows video, audio, and computer data to be visible to several participants at the same time.
A robot floor cleaner that is intended to clean relatively clear spaces such as corridors. The cleaner must be able to sense walls and other obstructions.
<!-- image -->
- 6.5. Explain why you normally use several architectural patterns when designing the architecture of a large system. Apart from the information about patterns that I have discussed in this chapter, what additional information might be useful when designing large systems?
- 6.6. Suggest an architecture for a system (such as iTunes) that is used to sell and distribute music on the Internet. What architectural patterns are the basis for this architecture?
- 6.7. Explain how you would use the reference model of CASE environments (available on the book's web pages) to compare the IDEs offered by different vendors of a programming language such as Java.
- 6.8. Using the generic model of a language processing system presented here, design the architecture of a system that accepts natural language commands and translates these into database queries in a language such as SQL.
- 6.9. Using the basic model of an information system, as presented in Figure 6.16, suggest the components that might be part of an information system that allows users to view information about flights arriving and departing from a particular airport.
- 6.10. Should there be a separate profession of 'software architect' whose role is to work independently with a customer to design the software system architecture? A separate software company would then implement the system. What might be the difficulties of establishing such a profession?
## REFERENCES
Bass, L., Clements, P. and Kazman, R. (2003). Software Architecture in Practice, 2nd ed. Boston: Addison-Wesley.
Berczuk, S. P. and Appleton, B. (2002). Software Configuration Management Patterns: Effective Teamwork, Practical Integration. Boston: Addison-Wesley.
Booch, G. (2009). 'Handbook of software architecture'. Web publication. http:/ /www.handbookofsoftwarearchitecture.com/.
Bosch, J. (2000). Design and Use of Software Architectures. Harlow, UK: Addison-Wesley.
Buschmann, F., Henney, K. and Schmidt, D. C. (2007a). Pattern-oriented Software Architecture Volume 4: A Pattern Language for Distributed Computing. New York: John Wiley &amp; Sons.
Buschmann, F., Henney, K. and Schmidt, D. C. (2007b). Pattern-oriented Software Architecture Volume 5: On Patterns and Pattern Languages. New York: John Wiley &amp; Sons.
Buschmann, F., Meunier, R., Rohnert, H. and Sommerlad, P. (1996). Pattern-oriented Software Architecture Volume 1: A System of Patterns. New York: John Wiley &amp; Sons.
Clements, P., Bachmann, F., Bass, L., Garlan, D., Ivers, J., Little, R., Nord, R. and Stafford, J. (2002). Documenting Software Architectures: Views and Beyond. Boston: Addison-Wesley.
Coplien, J. H. and Harrison, N. B. (2004). Organizational Patterns of Agile Software Development. Englewood Cliffs, NJ: Prentice Hall.
Gamma, E., Helm, R., Johnson, R. and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Reading, Mass.: Addison-Wesley.
Garlan, D. and Shaw, M. (1993). 'An introduction to software architecture'. Advances in Software Engineering and Knowledge Engineering, 1 1-39.
Harold, E. R. and Means, W. S. (2002). XML in a Nutshell. Sebastopol. Calif.: O'Reilly.
Hofmeister, C., Nord, R. and Soni, D. (2000). Applied Software Architecture. Boston: AddisonWesley.
Hunter, D., Rafter, J., Fawcett, J. and Van Der Vlist, E. (2007). Beginning XML, 4th ed. Indianapolis, Ind.: Wrox Press.
Kircher, M. and Jain, P. (2004). Pattern-Oriented Software Architecture Volume 3: Patterns for Resource Management. New York: John Wiley &amp; Sons.
Krutchen, P. (1995). 'The 4+1 view model of software architecture'. IEEE Software, 12 (6), 42-50.
Lange, C. F. J., Chaudron, M. R. V. and Muskens, J. (2006). 'UML software description and architecture description'. IEEE Software, 23 (2), 40-6.
Lewis, P. M., Bernstein, A. J. and Kifer, M. (2003). Databases and Transaction Processing: An Application-oriented Approach. Boston: Addison-Wesley.
Martin, D. and Sommerville, I. (2004). 'Patterns of interaction: Linking ethnomethodology and design'. ACM Trans. on Computer-Human Interaction, 11 (1), 59-89.
Nii, H. P . (1986). 'Blackboard systems, parts 1 and 2'. AI Magazine, 7 (3 and 4), 38-53 and 62-9.
Schmidt, D., Stal, M., Rohnert, H. and Buschmann, F. (2000). Pattern-Oriented Software Architecture Volume 2: Patterns for Concurrent and Networked Objects. New York: John Wiley &amp; Sons.
Shaw, M. and Garlan, D. (1996). Software Architecture: Perspectives on an Emerging Discipline. Englewood Cliffs, NJ: Prentice Hall.
Usability group. (1998). 'Usability patterns'. Web publication. http://www.it.bton.ac.uk/cil/usability/patterns/.

View File

@@ -1,36 +0,0 @@
from pathlib import Path
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, AcceleratorOptions
from docling.document_converter import DocumentConverter, PdfFormatOption
def main():
input_doc_path = Path("se_arch.pdf")
pipeline_options = PdfPipelineOptions()
pipeline_options.do_ocr = True
pipeline_options.do_table_structure = True
accelerator_options = AcceleratorOptions(device="cpu")
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options,
)
}
)
conversion_result = converter.convert(input_doc_path)
doc = conversion_result.document
md = doc.export_to_markdown()
output_path = Path("se_arch.md")
output_path.write_text(md, encoding="utf-8")
# doc_conversion_secs = conversion_result.timings["pipeline_total"].times
print(f"Saved markdown to {output_path}")
# print(f"Conversion secs: {doc_conversion_secs}")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,73 @@
# Environment Variables Template
# Copy this file to `.env` and fill in real values.
# Do NOT commit `.env` — it is ignored by .gitignore.
# ============================================================
# Cloud LLM Gateway (Backend)
# ============================================================
# Vertex AI / Gemini
VERTEX_AI_PROJECT=vkist-project
VERTEX_AI_LOCATION=asia-southeast1
VERTEX_AI_GEMINI_ENDPOINT=https://asia-southeast1-aiplatform.googleapis.com/v1/projects/vkist-project/locations/asia-southeast1/publishers/google/models/gemini-2.5-pro:generateContent
VERTEX_AI_MODEL=medgemma
# GCP access token loaded from secrets/gcp_access_token.txt or via GCP_ACCESS_TOKEN_FILE
# MedGemma Modal endpoint (Ollama web server)
# Full /api/chat URL also accepted — base is normalized automatically.
MODAL_MEDGEMMA_ENDPOINT=https://dtj-tran--ollama-medgemma-ollamaserver-web.modal.run
MEDGEMMA_MODEL=medgemma:4b
PORT=8080
# MEDGEMMA_API_KEY loaded from secrets/modal_api_key.txt or via MEDGEMMA_API_KEY_FILE
# MODAL_API_KEY is required by the Modal SDK itself
# ============================================================
# CORS
# ============================================================
# Comma-separated list of allowed frontend origins
CORS_ORIGINS=http://localhost:3000,http://localhost:5173,http://localhost:4173
# ============================================================
# Storage / Triton
# ============================================================
TRITON_ENDPOINT=http://localhost:8080
TEMP_DIR=/tmp/analysis_jobs
# ============================================================
# Redis
# ============================================================
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
# ============================================================
# Image Processing
# ============================================================
CLAHE_CLIP_LIMIT=2.0
CLAHE_TILE_SIZE=8,8
# ============================================================
# Exa Web Search (Agent Tools BFF)
# ============================================================
EXA_API_KEY=
# Canonical API guide: https://docs.exa.ai/reference/search-api-guide-for-coding-agents
# ============================================================
# Supabase — Knowledge Semantic Vector DB
# ============================================================
SUPABASE_URL=https://<project-ref>.supabase.co
SUPABASE_SERVICE_ROLE_KEY=
# Optional: authenticated read + RPC from backend with user JWT
SUPABASE_ANON_KEY=
# PoC only: return zero vector for supabase_query embedding (not for production RAG quality)
# EMBED_QUERY_MOCK=1
# ============================================================
# AWS S3 (if used)
# ============================================================
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_SESSION_TOKEN=
AWS_REGION=
AWS_DEFAULT_REGION=
AWS_ENDPOINT_URL=

170
secrets_template/README.md Normal file
View File

@@ -0,0 +1,170 @@
# Secrets Template — Developer Self-Setup Guide
This folder is the **tracked scaffold** for the local `secrets/` directory described in the [repository root README](../README.md#getting-started). It is safe to commit; it contains **no real credentials**.
Each developer creates and maintains their own `secrets/` folder at the repository root. That folder is **gitignored** and never pushed to Git.
---
## Quick Start
From the repository root (`PILOT_PROJECT/`):
```bash
# 1. Create your local secrets directory
mkdir -p secrets/aws_secret
# 2. Copy the environment template
cp secrets_template/.env.example secrets/aws_secret/.env
# 3. Create plaintext secret files (see tables below)
touch secrets/gcp_access_token.txt secrets/modal_api_key.txt
```
Then:
1. Open `secrets/aws_secret/.env` and fill in values for the services you need.
2. Paste single-value secrets into the corresponding `.txt` files (one secret per file, no quotes, no trailing newline required).
3. Confirm `secrets/` is ignored: `git check-ignore -v secrets/` should report a match.
Optional — add a local pointer file (also gitignored):
```bash
cp secrets_template/README.md secrets/README.md
```
---
## Expected Layout
After setup, your local tree should look like this:
```
PILOT_PROJECT/
├── secrets/ # gitignored — your machine only
│ ├── gcp_access_token.txt # Vertex AI / Gemini bearer token
│ ├── modal_api_key.txt # MedGemma Modal endpoint API key
│ ├── aws_secret/
│ │ └── .env # consolidated env vars (Supabase, Exa, AWS, Modal SDK, …)
│ ├── key # optional — SSH private key (infra)
│ ├── key.pub # optional — SSH public key
│ ├── init_ssh.sh # optional — SSH setup helper
│ └── server_vkist.sh # optional — server bootstrap script
└── secrets_template/ # tracked — templates & docs only
├── README.md # this file
├── .env.example # copy → secrets/aws_secret/.env
└── gemini_api_key.txt # production reference (not a local key file)
```
Only create the files you need for the components you are working on. Missing secrets for unused services will not block unrelated work.
---
## Secret Inventory
### Plaintext files (`secrets/` root)
| File | Env var | Used by | How to obtain |
|------|---------|---------|---------------|
| `gcp_access_token.txt` | `GCP_ACCESS_TOKEN` | Backend Cloud LLM Gateway (Vertex AI / Gemini) | `gcloud auth print-access-token` (short-lived; refresh as needed) |
| `modal_api_key.txt` | `MEDGEMMA_API_KEY` | Backend + Modal MedGemma endpoint auth | Project lead or Modal dashboard → API keys |
**Override:** point to a file outside `secrets/` with `{NAME}_FILE`, e.g. `GCP_ACCESS_TOKEN_FILE=/path/to/token.txt`. The loader checks `*_FILE` first, then `secrets/<filename>`.
Reference: `workspace/sprint_1_2/CODEBASE/backend/implementation/config.py`
### Environment bundle (`secrets/aws_secret/.env`)
Copy from `secrets_template/.env.example`. This file is auto-loaded by agent-tool smoke tests and the knowledge ingestion pipeline when present.
| Variable | Required for | Notes |
|----------|--------------|-------|
| `SUPABASE_URL` | Knowledge RAG, agent `supabase_query` | Supabase project URL |
| `SUPABASE_SERVICE_ROLE_KEY` | Backend writes / ingestion upload | Dashboard → Project Settings → API |
| `SUPABASE_PUBLISHABLE_KEY` / `SUPABASE_ANON_KEY` | Client-side or read-only flows | Same dashboard section |
| `SUPABASE_DB_URL` | Ingestion pipeline (PostgreSQL path) | Dashboard → Database → Connection string |
| `EXA_API_KEY` | Agent `exa_search` tool | [Exa dashboard](https://docs.exa.ai/) |
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` | S3 uploads, corpus storage | IAM user or role credentials |
| `MODAL_KEY`, `MODAL_SECRET` | Modal SDK CLI / deploys | Modal account settings |
| `HF_TOKEN` | Hugging Face model downloads | huggingface.co → Settings → Access Tokens |
| `FIGMA_ACCESS_TOKEN` | Design automation (optional) | Figma → Personal access tokens |
Non-secret configuration (endpoints, ports, CORS) lives in `workspace/sprint_1_2/CODEBASE/.env.example` — copy that to `CODEBASE/.env` separately; do **not** commit either `.env` file.
Reference paths:
- `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/config.py``secrets/aws_secret/.env`
- `workspace/sprint_1_2/CODEBASE/backend/tests/smoke_agent_tools_server.py` → same path
- `workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md` → smoke test auto-load
### Optional infra files
| File | Purpose |
|------|---------|
| `key` / `key.pub` | SSH key pair for remote servers or CI |
| `init_ssh.sh` | Helper to register SSH keys with `ssh-agent` |
| `server_vkist.sh` | Server-side bootstrap (project-specific) |
---
## Gemini API Key (Production Reference)
`secrets_template/gemini_api_key.txt` documents where the **production** Gemini key lives (GCP Secret Manager: `gemini-api-key` in project `vkist-project`). It is **not** a file you paste a key into for local dev.
For local backend work, use `gcp_access_token.txt` (Vertex AI bearer token) instead.
---
## Verify Your Setup
### Backend secrets (GCP + Modal)
```bash
cd workspace/sprint_1_2/CODEBASE
conda activate vkist_ultra # or your project env
PYTHONPATH=. python -c "from backend.implementation import config; print('GCP token loaded:', bool(config.GCP_ACCESS_TOKEN))"
```
### Agent tools smoke (Exa + Supabase, no GCP)
```bash
cd workspace/sprint_1_2/CODEBASE
PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py
# In another terminal:
cd workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools && npm run smoke:bff
```
### Knowledge ingestion
See `workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md` — requires `secrets/aws_secret/.env` with Supabase credentials.
---
## Security Rules
1. **Never commit** `secrets/` or any file containing real keys, tokens, or passwords.
2. The root `.gitignore` already excludes `secrets/`. Before adding new secret paths, confirm they remain ignored.
3. Do not paste secrets into source code, logs, issues, or PR comments.
4. Share credentials only through your team's approved secret manager (1Password, Vault, GCP Secret Manager, etc.).
5. Rotate tokens if they are ever exposed. GCP access tokens are short-lived; treat `.env` and API keys as long-lived secrets.
For agent and CI guardrails, see `AGENT_SKILL/secrets_and_phi_safety/SKILL.md`.
---
## Getting Credentials
If you do not have access to a required service:
1. Ask the project lead for onboarding to the relevant cloud account (GCP, Modal, Supabase, AWS).
2. Use the tables above to identify which variables or files you need for your sprint task.
3. Start with the smoke tests in [Verify Your Setup](#verify-your-setup) to confirm only the secrets you need are wired correctly.
---
## Related Documentation
- [Repository README — Secrets Management](../README.md#secrets-management-important)
- [CODEBASE `.env.example`](../workspace/sprint_1_2/CODEBASE/.env.example) — non-secret runtime configuration
- [Agent tools smoke tests](../workspace/sprint_1_2/CODEBASE/ml/tests/agent_tools/README.md)
- [Knowledge ingestion prerequisites](../workspace/sprint_1_2/CODEBASE/knowledge/implementation/ingestion/README.md)

View File

@@ -0,0 +1,14 @@
# Gemini API Key Reference
# This file documents the secret reference for the Gemini API key used by the Cloud LLM Gateway.
#
# Production secret location: GCP Secret Manager
# Secret name: gemini-api-key
# Project: vkist-project
# Region: asia-southeast1
#
# Access pattern:
# FastAPI Cloud LLM Gateway fetches key from GCP Secret Manager at startup
# Key never written to disk; accessed via GCP Workload Identity or service account
#
# Fallback: modal.Secret.from_name("gcp-secrets") in Modal deployment
# allows MedGemma container to authenticate to GCP Vertex AI if needed.

View File

@@ -0,0 +1 @@
# Replace with actual MedGemma Modal endpoint API key (never commit real secrets)

View File

@@ -0,0 +1,3 @@
modal secret create gitea-runner-secrets \
MODAL_KEY=your-modal-key \
MODAL_SECRET=your-modal-secret

View File

@@ -0,0 +1,37 @@
# Session Memory
## Current task
Created Kilo skill `cicd-architect-grill`.
## Created skill
- Skill name: `cicd-architect-grill`
- Skill file: `.kilocode/skills/cicd-architect-grill/SKILL.md`
- Skill line count: 49 lines after long draft compression.
## Skill purpose
Design, redesign, migrate, debate CI/CD pipelines for one software project. No cloud/language/runtime/platform assumptions.
## Skill behavior
- Inspect repo context before questions.
- Extract FRs, NFRs, constraints, success criteria, Definition of Done.
- Use C4 modeling only as deep as needed.
- Choose CI/CD pattern from situation, not vendor defaults.
- Cross-check: build reproducibility, tests, scans, artifacts, secrets, promotion, approvals, rollback, observability, cost, ownership.
- Grill one decision at a time when requested.
- Output draft/final plans as YAML.
## Repository context
- Workspace root: `/Users/davestran/Downloads/vkist_internship`
- Kilo config: `kilo.json`
- User-facing style: caveman style active.
- Existing skills inspected under `/Users/davestran/Downloads/vkist_internship/.kilocode/skills/`.
## Important constraints
- Do not assume GCP, AWS, Azure, Kubernetes, serverless, GitHub Actions, GitLab CI, Jenkins, Terraform unless user asks.
- Do not implement infrastructure until user approves design plan + implementation path.
- Do not ask many questions when repo evidence answers.
## Current status
- New skill directory exists.
- `SKILL.md` written and inspected.
- No tests run: Kilo skill definition, not executable code.

View File

@@ -0,0 +1,43 @@
# Session Memory
## Current task
Create session memory for 13 Jun 26.
## Work done
- Refined `API_CONTRACT_DRAFT.md` to v0.2.0 for FR-25 Synovitis Grading Workspace.
- Added public clinical routes + internal/local safety routes from API contract.
- Added async analysis job model, model registry, GradCAM, rationale, Socratic, BERT drift, RAG evidence, activations, artifact masks, ground truth, escalation, morphology, telemetry, report sign, EMR sync.
- Updated error codes, data types, OpenAPI skeleton, PlantUML sequence.
- Ran `git diff --check`; no whitespace errors.
## Architecture review
- Loaded `improve-codebase-architecture` skill.
- Read `LANGUAGE.md`; no repo `CONTEXT.md`; no `docs/adr`.
- Explore agent inspected `vkist-ultrasound/app.py`, `pdf_service.py`, `templates/js/script.js`, API docs, FR-25 use cases.
- Found 6 deepening candidates:
- Clinical Workflow State Machine Module — Strong.
- Model Registry + Runtime Adapter Module — Strong.
- Measurement + Severity Calibration Module — Strong.
- Privacy + Artifact + Audit Module — Strong.
- Finalization + Report + EMR State Module — Strong.
- Safety + Telemetry + Feedback Module — Worth exploring.
- HTML report write was attempted, but tool aborted before file landed. Report not completed.
## Repository context
- Workspace root: `/Users/davestran/Downloads/vkist_internship`
- User-facing style: caveman style active.
- Main code area: `PILOT_PROJECT/vkist-ultrasound`
- API contract target: `PILOT_PROJECT/workspace/sprint_1_2/Design_Material/API-docs/API_CONTRACT_DRAFT.md`
## Important constraints
- Do not propose interfaces before user picks architecture candidate.
- If user picks candidate, enter grilling loop.
- If naming new deepened module after concept missing from `CONTEXT.md`, update `CONTEXT.md`.
- If user rejects candidate with load-bearing reason, offer ADR only if future reviewers need it.
- Do not commit unless explicitly asked.
## Current status
- API contract file updated.
- Architecture candidates identified.
- HTML architecture report still pending due aborted write.
- No tests run for architecture review; no code changed during review.

View File

@@ -0,0 +1,60 @@
# Session Memory
## Current task
Create/update `grill_bento_codebase` architecture skill.
## Work done
- Loaded `grill-with-docs`, `grill-bento-codebase`, `skill-creator`.
- Inspected:
- `.kilocode/skills/grill_bento_codebase/SKILL.md`
- `.kilocode/skills/grill_bento_codebase/GEMINI.md`
- `.kilocode/skills/grill_bento_codebase/BENTO_DESIGN_GUIDELINE.md`
- `.kilocode/skills/grill_bento_codebase/bento_template.html`
- `.kilocode/rules/bento_template.html`
- `.kilocode/rules/BENTO_DESIGN_GUIDELINE.md`
- Updated `grill_bento_codebase` skill assets.
- Added Bento depth constraint:
- one Bento room may contain multiple sub-components/sub-projects
- max Bento depth = 3
- depth 1 = root Bento
- depth 2 = child sub-project
- depth 3 = grandchild sub-project
- depth 4+ invalid
- Added fallback rule: detail beyond depth 3 stays inside owning Bento as local `spec/`, `spec/docs/`, `implementation/`, or `tests/`.
- Moved skill-local source refs to:
- `BENTO_DESIGN_GUIDELINE.md`
- `bento_template.html`
- Bumped guideline version to `0.5.0`.
## Skill output contract
- Context lock from FRS, NFRS, user-interaction, Software_Arch, Solution_Arch.
- Bento-BEV HTML blueprint from `bento_template.html`.
- Markdown tree-view for dev-team flatout navigation.
- Room cards.
- Interface contracts.
- Dependency map.
- Traceability matrix.
- Open decisions, one at a time.
## Validation done
- Re-read full `.kilocode/skills/grill_bento_codebase/BENTO_DESIGN_GUIDELINE.md`.
- Confirmed depth rule appears in:
- `BENTO_DESIGN_GUIDELINE.md`
- `SKILL.md`
- `GEMINI.md`
- Grep checked no stale `.kilocode/rules` refs inside `.kilocode/skills/grill_bento_codebase`.
## Repository context
- Workspace root: `/Users/davestran/Downloads/vkist_internship`
- User-facing style: caveman style active.
- Skill area: `.kilocode/skills/grill_bento_codebase`
## Important constraints
- No Bento tree depth > 3.
- No nested Bento beyond grandchild depth.
- Extra detail stays local to owning Bento room.
- No commits unless explicitly asked.
## Current status
- `SKILL.md`, `GEMINI.md`, `BENTO_DESIGN_GUIDELINE.md` updated.
- No tests run: skill docs, not executable code.

View File

@@ -0,0 +1,71 @@
# Session Knowledge — 2026-07-15
## Goal
Debug why Celery batch API results are not displaying on the client UI and fix the full pipeline: server startup, worker task registration, Triton endpoint routing, polling storm, stale result re-submission.
## Constraints & Preferences
- Client should not hammer backend with polls; need request coalescing and caching.
- Already-completed batch results should not be re-submitted or re-queried on every page view.
- Worker must target Modal Triton endpoint, not localhost.
## Progress
### Done
- Fixed `GroupResult.restore()` returning `None` by adding `result.save()` after `apply_async()` in `submit_celery_batch()`.
- Fixed server startup `ModuleNotFoundError: No module named 'backend'` by correcting `CODEBASE_ROOT` path math in `backend/routers/run_cv_inference.sh` (changed `SCRIPT_DIR/..``SCRIPT_DIR/../..`).
- Added `celery_app.autodiscover_tasks(["backend.implementation.tasks"])` to `backend/services/celery_app.py` so worker registers `run_cv_chunk`.
- Added `export TRITON_ENDPOINT=...modal.run` to `backend/start_workers.sh` so worker targets Modal instead of `localhost:8000`.
- Added server-side TTL poll cache (`_batch_poll_cache`) in `backend/services/cv_celery_service.py` with default 2000ms TTL to coalesce rapid repeat polls.
- Increased client poll interval from 1000ms → 2000ms and removed unused `pollIntervalMs` parameter from `getCvAnalyzeResultsForProfileCelery()` in `frontend/implementation/src/lib/cvAnalyzeApi.ts`.
- Fixed poll endpoint HTTP status semantics: `GET /api/test/analyze/batch/celery/{job_id}` now returns `202 Accepted` for pending, `404 Not Found` for unknown, `200 OK` for completed/failed. This removes the misleading-200 problem for in-flight jobs.
- Added backend timing logs in `cv_celery_service.py`: submission logs chunk/image counts; completion/failure logs `duration_ms` from submission to terminal state.
- Added frontend timing logs in `getCvAnalyzeResultsForProfileCelery()` for completed/failed/unknown outcomes.
- Migrated frontend config from `.env`/`.env.development` to single YAML source of truth at `config/frontend.config.yaml`, loaded via `js-yaml` in `vite.config.ts` using Vite `define`. Removed old `.env` files. Added `VITE_MODAL_OLLAMA_TARGET` to YAML so proxy target is config-driven.
- Fixed TypeScript errors in `vite.config.ts` by adding `"types": ["node"]` and including `vite.config.ts` in `tsconfig.json`.
- Added client-side in-flight coalescing + completed-result cache in `getCvAnalyzeResultsForProfileCelery()` so repeat page views dont re-submit identical work.
### In Progress
- Worker needs force-restart to pick up `autodiscover_tasks` + `TRITON_ENDPOINT` changes.
- One submitted job reached `completed: 1/2` but never finished — need worker restart + fresh batch test.
- Client-side ETA estimation for pending batch jobs is planned but not yet implemented.
### Blocked
- Worker process restart — old PID/log still active without the new fixes applied.
## Key Decisions
- Used `result.save()` because Celery does not auto-persist `GroupResult` metadata to Redis.
- Added `autodiscover_tasks` instead of manual imports because worker is launched via `-A backend.services.celery_app` and never imports `cv_tasks.py` otherwise.
- Set worker `TRITON_ENDPOINT` in `start_workers.sh` because worker is separate process that doesnt run `cv_inference_server.py`s `os.environ.setdefault()`.
- Poll cache TTL set to 2000ms to match 2s client poll interval.
- Single YAML config replaces `.env` + `.env.development` to eliminate precedence confusion.
- `VITE_MODAL_OLLAMA_TARGET` lives in YAML so proxy target is editable without touching `vite.config.ts` code.
## Next Steps
- Force-restart Celery worker (`kill -9` stale PID, remove `celery.pid`, run `./start_workers.sh start`).
- Verify worker log shows `backend.implementation.tasks.cv_tasks.run_cv_chunk` under `[tasks]`.
- Submit fresh batch and confirm it transitions `pending``completed` with actual results.
- Implement ETA estimation in backend poll response based on observed chunk throughput.
- Address client behavior that submits new jobs while previous jobs are still `pending` (7+ concurrent jobs observed in logs).
## Critical Context
- Latest server logs show 7+ concurrent jobs all stuck at `pending` with `completed: 0/2` or `1/2`, plus new POST submissions every ~10s — indicating worker is not processing tasks.
- Worker log still shows stale startup at `16:22:18` with empty `[tasks]`; fixes in `celery_app.py` and `start_workers.sh` have not been loaded by running worker.
- One job (`13fd014e`) reached `completed: 1/2, progress: 0.5` and stalled — second chunk likely failed or was never picked up.
- Modal Triton endpoint: `https://dtj-tran--triton-s3-service-unified-triton-server.modal.run`.
- Redis broker/backend: `redis://localhost:6379/0`.
- Chunk size default: 4 frames per chunk (`CELERY_CHUNK_SIZE`).
- `cv_result_cache.py` already exists with in-flight coalescing pattern; new batch poll cache follows same design.
- Frontend not currently in Celery mode (`VITE_USE_CV_CELERY=false` in YAML).
- Direct batch route still shows noisy fallback logs when Triton returns 502 on batched inference — fallback works, but should log at `info` not `warning`.
## Relevant Files
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/services/cv_celery_service.py`: Contains `submit_celery_batch()` (fixed with `result.save()`), `get_celery_batch_result()` (now with 2000ms TTL poll cache + timing logs), and poll cache constants.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/services/celery_app.py`: Added `autodiscover_tasks(["backend.implementation.tasks"])`; routes `cv_inference.run_chunk` to `cv-inference` queue.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/start_workers.sh`: Added `TRITON_ENDPOINT` export so worker targets Modal.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/routers/run_cv_inference.sh`: Fixed `CODEBASE_ROOT` path math (`SCRIPT_DIR/../..`) so `PYTHONPATH` resolves `backend` package.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/implementation/tasks/cv_tasks.py`: Defines `run_cv_chunk` Celery task.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/routers/cv_inference.py`: Contains `/api/test/analyze/batch/celery` endpoints. Poll endpoint now returns 202/404/200.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation/src/lib/cvAnalyzeApi.ts`: Poll interval 2000ms; client-side cache + in-flight coalescing; timing logs.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation/src/hooks/useSegmentationOverlay.ts`: Calls `getCvAnalyzeResultsForProfileCelery()`.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation/config/frontend.config.yaml`: Single source of truth for frontend feature flags and URLs.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation/vite.config.ts`: Loads YAML config and injects as `import.meta.env.*`.
- `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation/tsconfig.json`: Added `"types": ["node"]` and included `vite.config.ts`.

View File

@@ -0,0 +1,45 @@
---
title: Session Memory 18 Jun 26
date: 2026-06-18
status: active
---
## Summary
Compressed 2026-06-18 session work: `cv_code` split, PlantUML Chen ER skill, session memory.
## cv_code module split
- Split legacy `legcode/app.py` into `ml/implementation/cv/code/` into three modules:
- `config.py`
- `framework.py`
- `server.py`
- Legacy file untouched until new modules pass import checks.
- `python -m py_compile` passed all files.
- `server.py` import blocked by missing `pdf_service`.
- `framework.py` path bug: `cv_root = Path(__file__).resolve().parents[2]` skips too far. Correct: `parents[1]`.
## skill-creator inspection
- Loaded `.config/kilo/.kilo/skills/skill-creator/SKILL.md`.
- Skill structure: `SKILL.md` required; optional `GEMINI.md`, `EXAMPLES.md`, `REFERENCE.md`, `scripts/`, `assets/`.
- Template description rules apply.
## er-chen-diagram skill
- Created `.kilocode/skills/er-chen-diagram/` scaffold.
- Files written: `SKILL.md`, `EXAMPLES.md`, `REFERENCE.md`.
- Skill contents:
- Mandatory `@startchen` / `@endchen`.
- Attributes only inside `{}`.
- Primary keys use `<<key>>`.
- Relationships empty blocks, then connect lines.
- Copy exact code shells verbatim.
## Session memory housekeeping
- Listed existing memory files: `12_Jun_26.md`, `13_Jun_26.md`, `14_Jun_26.md`.
- Recorded activity in this file.
## Relational DB Design
- Created relational DB design spec per plan: ER/schema PlantUML and Chen notation in RELATIONAL_DB_SCHEMA_SPEC.md.

View File

@@ -0,0 +1,159 @@
# Coding Agent Instruction: Resolve 17 Remaining Structural Gaps in Phase 0 & Phase 1
You have already fixed the TypeScript build failures. The 17 unchecked plan items below must be addressed structurally. Each item is a checklist entry in `.kilo/plans/35-figma-designs-implementation-plan.md`. Only mark an item `[x]` in that plan file after you have implemented it AND it compiles cleanly.
## Reference Files
- Implementation Plan: `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/.kilo/plans/35-figma-designs-implementation-plan.md`
- Frontend root: `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend`
- Severity rule: Do NOT mark `[x]` unless you have implemented the item AND verified it.
## Work Order: Exact Sequence with Acceptance Tests
### 0.1 Environment Tooling
1. Configure ESLint + Prettier
- Add `.eslintrc.cjs` and `.prettierrc` (or extend recommended configs).
- Add scripts to `package.json`: `"lint": "eslint src --ext .ts,.tsx"`.
- Acceptance: `npm run lint` runs without failing on the current src tree (warnings OK, but no auto-fixable errors left hanging).
2. Set up Husky
- Install `husky` + `lint-staged`.
- Add `prepare` script: `"prepare": "husky"`.
- Add `.husky/pre-commit` to run `lint-staged`.
- Acceptance: `git add src/ && git commit` triggers Husky and blocks on lint failure.
3. Configure Vitest
- Install: `vitest`, `@testing-library/react`, `@testing-library/jest-dom`, `jsdom`.
- Add `vitest.config.ts` with `test.environment = 'jsdom'` and `include: ['src/**/*.{test,spec}.{js,ts,jsx,tsx}']`.
- Add script: `"test": "vitest"`.
- Acceptance: `npx vitest run` executes (0 tests is OK for now; framework must be correct).
4. Set up Playwright
- Install `@playwright/test`.
- Add `playwright.config.ts` with baseURL `http://localhost:5173` and webServer using `vite preview` or same.
- Add script: `"test:e2e": "playwright test"`.
- Acceptance: `npx playwright test --list` does not error (no tests yet is OK).
5. Configure Storybook
- Install `@storybook/react-vite` and run `storybook init`.
- Add 1 story for `Button` atom and 1 story for `TopBar` organism.
- Acceptance: `npm run storybook` builds the UI and the 2 stories render.
### 0.3 Component Inventory
6. Analyze 35 Figma Files
- Inspect Figma paths under `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/Design_Material/` - section: `Figma Design Source Files`.
- Produce an inventory mapping file: `spec/figma-component-specs.md` is insufficient; append `spec/component-inventory.md` with:
- Unique screen count (should be 35).
- Reusable component list by atomic depth (atoms, molecules, organisms, templates).
- One dependency graph (Mermaid is fine).
- Acceptance: `spec/component-inventory.md` exists, has 35 distinct screens, and has a Mermaid dependency graph.
7. Build Storybook Stories for Primitives
- Add stories for `Button`, `Input`, `Icon`, `Badge`, `Label`, `Toggle`, `Tooltip`.
- Acceptance: All primitives render in Storybook.
### 0.4 API Contracts
8. Cross-Reference API Contracts
- Open `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/Design_Material/API_docs/API_CONTRACT_DRAFT.md` & the `API spec from the backend-file` in `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/backend/spec/interface-contract.md` & the data-layer definition - check the reference from context in the dirrectory (a folder) at `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/data/spec`
- Cross-check each endpoint against `src/services/*.ts`.
- Acceptance: Each service file has JSDoc lists of endpoints it covers.
9. Define TypeScript Interfaces
- Add `src/types/api.ts` with request/response interfaces for all 7 services.
- Acceptance: All 7 services import and use their types.
10. Implement HTTP + MSW Adapters
- Add `src/services/adapters/httpAdapter.ts` and `src/services/adapters/mswAdapter.ts`.
- Each service file must use the adapter via `services/index.ts` seam.
- Acceptance: `tsc --noEmit` clean; `vitest run` can mock the adapter without touching prod code.
11. Implement Auth/Logging Interceptors
- In `httpAdapter.ts`, add request interceptor that:
- Attaches `Authorization` header from a session token (mocked OK).
- Logs request method + URL.
- Normalizes errors to a `ServiceError` class.
- Acceptance: Interceptor exists and is imported only by `httpAdapter.ts` (no component imports).
### 1.2 Zustand Slices
12. Co-locate Slice Tests
- Create `src/store/sessionSlice.test.ts`, `src/store/dicomSlice.test.ts`, etc. (6 files).
- Each test imports the slice factory (`createSessionSlice`) and invokes the interface.
- Acceptance: `vitest run --coverage` shows coverage files for each slice.
13. Create Domain Selectors
- In each slice file, export type-safe selectors via `create`.
- Acceptance: `TopBar` and `BottomBar` consume slice selectors instead of inline memoization.
### 1.3 WebWorkers
14. Create `src/workers/` Files
- `cv.worker.ts`: stub with `onmessage` and `postMessage` for `preprocess` msg type.
- `llm.worker.ts`: stub with `generate` msg type.
- `guardrail.worker.ts`: stub with `safetyCheck` msg type.
- Acceptance: `tsc --noEmit` clean; workers can be imported in a hook.
15. Implement Worker Pool + Hooks
- `src/hooks/useWebWorker.ts`: generic hook accepting worker class + message schema.
- `src/hooks/useCVWorker.ts`, `useLLMWorker.ts`, `useGuardrailWorker.ts`: domain wrappers.
- Acceptance: Hooks can be instantiated without runtime errors in a test render.
16. Transferable Message Protocols
- In each worker stub, explicitly show `Transferable` usage (pass `ImageBitmap` or `ArrayBuffer` in transfer list).
- Acceptance: TypeScript types enforce transferable contracts.
17. Fallback to Main Thread
- Each hook exports `isSupported: boolean`. If `Worker` constructor not available, use a mock implementation on main thread.
- Acceptance: In jsdom test, `isSupported` is false and hook still resolves.
### 1.4 Services Deepening
18. Move `encryption.ts` Out of the Store
- Remove `import { encryptStorage } from '../utils/encryption'` from `src/store/index.ts`.
- Create `src/services/storageAdapter.ts` that exports `createSessionStorage` and `createEncryptedStorage`.
- `store/index.ts` must import `createJSONStorage(() => storageAdapter)` from `services/storageAdapter`.
- Acceptance: `grep encryptStorage src/store/index.ts` returns nothing in src/store/.
### 1.5 Route-Aware Shell
19. Implement `protected.tsx`
- Add `src/routes/protected.tsx` with a `<RequireAuth>` wrapper that redirects to `/login` when `!sessionId`.
- Wrap `/workspace` in `<RequireAuth>`.
- Acceptance: Visit `/workspace` when unauthenticated; browser redirects to `/login`.
20. `WorkspaceShell` Owns 65/35 Split
- `WorkspaceShell.tsx` must render the 65/35 layout directly: `div.flex.h-screen` with `w-[65%]` ZoneA and `w-[35%]` ZoneB.
- `routes/index.tsx` `<DiagnosticWorkspace>` must only render `<WorkspaceShell />` without re-implementing the split.
- Acceptance: `grep "flex-1" src/components/templates/WorkspaceShell.tsx` finds exactly one instance; `grep "ZoneA" src/routes/index.tsx` finds zero (or just an import).
21. Lift `initApp` Out of `App.tsx`
- Add `src/hooks/useAppInit.ts`.
- Move the `initApp` effect into this hook.
- `App.tsx` becomes: `<AppRoutes />` plus `<AppInit />` or composes it.
- Acceptance: `grep "initApp" src/App.tsx` finds nothing.
22. `React.lazy` + `Suspense` in Routes
- Use `React.lazy(() => import('../components/organisms/ZoneA'))` and same for `ZoneB`.
- Wrap `<DiagnosticWorkspace>` children in `<Suspense fallback={<div>Loading...</div>}>`.
- Acceptance: `grep React.lazy src/routes/index.tsx` finds 2+ matches.
### 1.6 Layout Responsiveness
23. Responsive Breakpoints
- `WorkspaceShell.tsx` must use `@media` or Tailwind responsive classes so that on `< md` the 65/35 stack becomes a tabbed/toggle layout (Zone A | Zone B).
- Acceptance: Resize viewport to 375px; only one Zone visible at a time.
24. 60fps Rendering Hints
- Add `will-change: transform` to animated containers.
- Add `contain: layout style paint` to the workspace root.
- Acceptance: `grep -E "will-change|contain:" src/components/templates/WorkspaceShell.tsx` finds both.
## Strict Consequences
- If you cannot complete a specific item within this prompt, do not leave it half-done. Leave the plan checkbox `[ ]` and add a comment in code explaining what is blocking you.
- All code must keep `tsc --noEmit` clean after your changes.
- Do NOT touch backend files or any file outside `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend` unless an item explicitly requires it.
## Final Verification (YOU MUST RUN THESE)
```bash
cd /Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend
npx tsc --noEmit && npm run build
npm run lint
npx vitest run
npx playwright test --list
npm run storybook -- --headless 2>&1 | tail -20
```
Produce a summary of which commands passed/failed and why.

View File

@@ -0,0 +1,189 @@
# SESSION MEMORY — 26 Jun 26
## APP PURPOSE
VKIST MSK Platform — air-gapped musculoskeletal ultrasound analysis. FR-25 Synovitis Grading (knee). Vietnamese NLP. RAG citations. HITL finalization. ≤150 MB bundle. ≤1.5s inference.
## ARCHITECTURE (C4)
### Context
- **UP5** Radiologist: primary user — load scan, review grade, finalize, sign, view GradCAM, engage circuit breaker, review RAG evidence
- **UP1** Senior Expert: clinical protocol validation, MOH guideline approval, threshold sign-off
- **UP4** Support: registration, case queue
- External: PACS (DICOM), EMR/HIS (HL7/FHIR), Triton (GPU inference), ladybugDB (ontology), pgvector (MOH guidelines HNSW), GemmaE2B/MedGemma (LLM), EmbeddingGemma (RAG embeddings)
### Containers
- **PWA** React 18 + TS + Zustand + LiteRT + MediaPipe + Dexie.js. DICOM canvas, GradCAM overlay, edge guardrail (Transformers.js BERT + OpenRedaction + pii-filter) in WebWorkers.
- **NGINX + Keepalived** VIP failover ≤2s, SSL termination
- **FastAPI** edge-inference-svc: DICOM ingest, ML pipeline orchestration, Grad-CAM, report assembly, SSE streaming
- **FastAPI** rag-svc: pgvector top-k retrieval, ladybugDB ontology, LLM consult route, RAG-Referee
- **Postgres + pgvector** HNSW index for MOH guidelines
- **Redis** JWT sessions, rate-limit, consult_mode state
- **MinIO** S3: DICOM, overlays, GradCAM heatmaps, reports
- **Triton** gRPC :8001: 3-step ML pipeline + embeddings
### 3-STEP ML PIPELINE (Triton)
1. **Angle Classification** — ConvNeXt/DenseNet/ResNet/EfficientNet/Swin-V2. Classes: med-lat, post-trans, sup-trans-flex, sup-up-long
2. **Inflammation Detection** — EfficientNet-B0 binary (post-trans, sup-up-long only)
3. **Segmentation** — DeepLabV3-ResNet101 (post-trans) OR UNet3Plus-Attention/DeepLabV3 (sup-up-long). Classes: background, effusion, fat, femur, synovium, tendon
4. **Measurement** — ROI middle 1/3 of bounding box. PIXEL_TO_MM = 45/655
5. **Severity** — Combined: effusion 60% + synovium 40%. Grades 0-3: Rất nhẹ/Nhẹ/Trung bình/Nặng
### GRADCAM
- Zero extra clicks (auto on upload)
- Base64 PNG from FastAPI
- Canvas overlay, adjust opacity
### WEB WORKER TOPOLOGY
- `cv.worker.ts` — LiteRT WASM: CV inference (angle pre-classifier). Isolated WASM.
- `llm.worker.ts` — WebLLM WASM: GemmaE2B local generation. NFR-4 1.5GB heap.
- `guardrail.worker.ts` — Transformers.js WASM/WebGPU: BERT hallucination/mal-intention/scope-breach scoring.
**Isolation:** No SharedArrayBuffer/Atomics. postMessage with structured clone only.
## FRONTEND DESIGN (WS-25)
### Layout: 65/35 split (Zone A: DICOM | Zone B: Results + Chat)
```
┌─ Top Bar (patient ID, back, sign) ──────────────────────┐
├─ Zone A (65%) ────────┬─ Zone B (35%) ────────────────────┤
│ │ Phase Indicator (sticky) │
│ 5 Canvas Layers: │ Classification Card (grade,conf) │
│ 1. Background │ Explanation Panel (GradCAM thumb) │
│ 2. GradCAM │ Chat Panel (Socratic dialogue) │
│ 3. Segmentation │ Override Card (Phase 4) │
│ 4. Annotations (SVG) │ Quick Actions │
│ 5. Viewport HUD │ │
│ │ │
│ Toolbar: pan/zoom/ │ │
│ brush/eraser/caliper │ │
├───────────────────────┴───────────────────────────────────┤
└─ Bottom Bar (NFR status, EMR sync, offline indicator) ─────┘
```
### Canvas Layers (z-index)
- 0: Background DICOM (render once, never redraw on pan/zoom)
- 10: GradCAM (CSS transform, no redraw)
- 50: Segmentation overlay
- 100: SVG Annotations (brush/caliper)
- 200: Viewport HUD (text, mousemove redraw)
- 900-999: System chrome (top/bottom bar)
### Zustand Store Keys
- `sessionId`, `patientId`, `currentPhase`
- `dicomFrame: ImageBitmap | null` (GPU-friendly)
- `dicomMetadata`, `mlResults` (single atomic object)
- `clinicianGrade`, `isOverriding`, `overrideJustification`
- `maskedRegions: MaskedRegion[]` (SVG path data, not raster)
- `chatMessages` (pruned to 20, older → IndexedDB)
- `driftScore`, `refereeIntervened`
- `zoomLevel`, `panOffset` (not persisted, RAF updates)
- `activeTool: pan|zoom|brush|eraser|caliper|pin`
- `showGradCAM`, `showSegmentation`
### 6-PHASE WORKFLOW
1. **Classification** — ML grade, confidence bar
2. **Explanation** — GradCAM visible, RAG evidence, LLM rationale
3. **Negotiation** — Socratic dialogue, circuit breaker, BERT drift
4. **Reconfiguration** — Override card, pixel activation inspector
5. **Final Decision** — Summary, HITL signature
6. **EMR Sync** — HL7/FHIR push, audit hash
## KEY API CONTRACTS
### Core Endpoints
- `POST /api/v1/sessions` → create session
- `POST /api/v1/sessions/{id}/frames` → upload DICOM (multipart)
- `POST /api/v1/analysis-jobs` → trigger ML pipeline
- `GET /api/v1/analysis-jobs/{id}` → get result (angle/inflammation/segmentation/measurement/severity)
- `GET /api/v1/analysis-jobs/{id}/steps` → step-level status
- `PATCH /api/v1/sessions/{id}/review` → clinician review (approve/correct/reject)
- `POST /api/v1/reports` → generate report
- `POST /api/v1/reports/{id}/sign` → HITL digital signature
- `POST /api/v1/reports/{id}/emr-sync` → EMR push (outbox if offline)
### Safety Endpoints
- `POST /api/v1/sessions/{id}/explanations/gradcam` → GradCAM heatmap
- `POST /api/v1/sessions/{id}/safety/circuit-breaker` → trigger circuit breaker
- `POST /api/v1/sessions/{id}/chat/socratic` → Socratic dialogue (SSE streaming)
- `POST /api/v1/sessions/{id}/drift/check` → BERT semantic drift
- `POST /api/v1/sessions/{id}/rag/evidence` → RAG guideline arbitration
- `POST /api/v1/feedback` → ground-truth override telemetry
### ML Response Shape
```json
{
"result": {
"angle": { "class": "sup-up-long", "confidence": 0.9845 },
"inflammation": { "detected": true, "confidence": 0.942 },
"segmentation": { "mask_ref": "s3://...", "overlay_ref": "s3://...", "color_legend": {...} },
"measurement": { "thickness_mm": 6.87, "pixel_to_mm": 0.0687 },
"severity": { "level": 3, "label": "severe", "combined_score": 18.4 }
}
}
```
## NFR SUMMARY
| NFR | Target | Implementation |
|-----|--------|----------------|
| NFR-4 | ≤150 MB bundle | React PWA, tree-shake, split vendor chunk, Dexie.js |
| NFR-5 | ≤1.5s inference | Triton ONNX/TensorRT, client memory ≤150MB |
| NFR-7 | ≤200ms TTFT | SSE streaming, token chunking |
| NFR-8 | Fault tolerance | IndexedDB cache, SW offline mode, session recovery |
| NFR-13 | Grad-CAM zero-click | Auto on upload, no extra UI steps |
| NFR-14 | No client GPU | LiteRT WASM, CPU fallback |
| NFR-16 | Air-gapped primary | All inference + storage on-prem via K3s |
| NFR-17 | Immutable audit | PostgreSQL WAL, no UPDATE/DELETE triggers |
| NFR-18 | RAG citations | 100% LLM text cites MOH via pgvector HNSW |
| NFR-19 | HITL signature | `signer_id != NULL` constraint before FINALIZED |
## DECREE 13 / CIRCULAR 46
- Client: Decree 13 regex scrubber (OpenRedaction + pii-filter + js-data-anonymizer) before network
- Server: FastAPI `phi_scrub` middleware (Microsoft Presidio) as ground-check
- PDF: Circular 46/2018 format, bilingual VI/EN, audit hash, signer block
## SPRINT 1-2 DELIVERABLES
### Sprint 1 (Jun 1526) — "Fast PoC Baseline"
- [ ] PWA shell: React + Zustand + PWA manifest + Service Worker + Dexie.js
- [ ] DICOM canvas: Cornerstone.js + 5-layer canvas stack
- [ ] FastAPI `/api/analyze` endpoint (sync 3-step pipeline)
- [ ] Triton 3-step model ensemble + gRPC
- [ ] NGINX + Keepalived VIP
- [ ] Grad-CAM overlay on primary viewport
- [ ] Circuit-breaker skeleton
### Sprint 2 (Jun 29Jul 10) — "Multi-Modal & NLP Integration"
- [ ] GemmaE2B browser WebLLM + Cloud MedGemma Vertex AI
- [ ] BERT drift monitor + RAG-Referee
- [ ] Socratic dialogue (SSE streaming)
- [ ] ladybugDB ontology traversal
- [ ] Decree 13 scrubber (client + server)
- [ ] Circular 46 PDF report generator
- [ ] HITL signature gate
- [ ] Immutable audit log
- [ ] EMR/HIS HL7/FHIR push
## KEY FILES
- `/workspace/sprint_1_2/Design_Material/SPRINT_1_2_ARCHITECTURE_SPEC.md` — sprint-scoped architecture
- `/PROJ_LEVEL_READING/ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md` — full software architecture
- `/workspace/sprint_1_2/Design_Material/API_docs/API_CONTRACT_DRAFT.md` — API contract v0.2.1
- `/workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/UI_Screen_Spec/Screen_Analysis_WS-25.md` — WS-25 screen analysis
- `/workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/FR_25_UC_SPEC.md` — FR-25 use case spec
- `/workspace/sprint_1_2/Design_Material/FR_25_UC_DESIGN/UC_CONTEXT.md` — use case context
## PHASE SEQUENCE (ML Pipeline)
```
DICOM upload → CLAHE → Angle Classification → (if post-trans) Inflammation → Segmentation
→ (if sup-up-long) Inflammation → Segmentation → Measurement → Severity (0-3)
→ GradCAM overlay → LLM rationale (GemmaE2B/MedGemma + RAG) → Clinician review
→ HITL sign → EMR push
```
## NEXT ACTIONS
- Implement PWA shell (T1-E)
- Set up FastAPI `/api/analyze` + `/api/consult/stream` (T2-B)
- Configure Triton ensemble (T2-A)
- Build WS-25 UI components (Zone A 5-layer canvas + Zone B scrollable panel)
- Wire up edge guardrail WebWorkers (cv.worker.ts, llm.worker.ts, guardrail.worker.ts)

View File

@@ -0,0 +1,43 @@
# SESSION MEMORY — 27 Jun 26
## BACKEND ARCHITECTURE & SPECIFICATION
### 1. Project Context (from Proj_context_26_Jun_26.md)
- **Purpose**: VKIST MSK Platform for air-gapped musculoskeletal ultrasound analysis (FR-25 Synovitis Grading).
- **Core Tech Stack**: FastAPI, PostgreSQL + pgvector, Redis, MinIO (S3), Triton (GPU inference), Local LLM/BERT.
- **ML Pipeline**: 3-step process: Angle Classification $\rightarrow$ Inflammation Detection $\rightarrow$ Segmentation/Measurement $\rightarrow$ Severity Grading (0-3).
- **Safety Stack**: GradCAM, Socratic Chat, BERT drift monitor, RAG-Referee.
- **Compliance**: Decree 13 (PII scrubbing), Circular 46 (PDF reports).
### 2. Backend Structural Specification (from backend-spec.md)
The backend follows a modular "Seams" architecture:
- **API Layer**: `backend/api/` contains FastAPI routers.
- **Implementation Layer**: `backend/implementation/` contains deep modules.
- **Adapters**: `backend/implementation/adapters/` handles low-level external service communication (S3, Triton, LLM, BERT).
**Core Modules:**
- **Auth**: JWT, session management.
- **Patient**: Patient CRUD, ingestion history.
- **Session**: Frame handling, S3 storage, review patching.
- **Analysis Jobs**: Async orchestration of Triton inference via KServe v2 HTTP.
- **Safety**: GradCAM, RAG evidence, Socratic chat, circuit breaker, drift check.
- **Notification/Settings/Telemetry**: Support modules for user preferences and system monitoring.
### 3. Interface Contract (from interface-contract.md)
**Key API Categories:**
- **Auth**: `/api/v1/auth/login`, `/api/v1/users/me`.
- **Patient**: `/api/v1/patients`, `/api/v1/patients/{id}/sessions`.
- **Clinical Workflow**:
- Sessions: `/api/v1/sessions`, `/api/v1/sessions/{id}/frames`, `/api/v1/sessions/{id}/review`.
- Analysis: `/api/v1/analysis-jobs` (async), `/api/v1/analysis` (sync).
- Reports: `/api/v1/reports`, `/api/v1/reports/{id}/sign`, `/api/v1/reports/{id}/emr-sync`.
- **Safety & Explanations**:
- GradCAM: `/api/v1/sessions/{id}/explanations/gradcam`.
- Rationale/Chat: `/api/v1/sessions/{id}/explanations/rationale`, `/api/v1/sessions/{id}/chat/socratic`.
- Guardrails: `/api/v1/sessions/{id}/drift/check`, `/api/v1/sessions/{id}/rag/evidence`.
### 4. Dependency Graph
- **Auth/Patient/Session/Safety/Telemetry** $\rightarrow$ PostgreSQL.
- **Session/AnalysisJobs/Safety** $\rightarrow$ S3.
- **AnalysisJobs** $\rightarrow$ Triton (Modal Serverless).
- **Safety** $\rightarrow$ Local LLM, BERT, RAG Knowledge Base.

View File

@@ -0,0 +1,37 @@
# Session Memory: 2026-06-27 Backend NLP/LLM Consult Implementation
## Task
Build Vietnamese Clinical LLM Consult endpoints (Rationale, Socratic Chat, Guardrail Check, SSE Stream). Follow NFR-16a governance.
## Implementation Details
### 1. LLM Adapter (`backend/implementation/adapters/llm_adapter.py`)
- Build `VertexAILangchainAdapter` via `langchain-google-vertexai`.
- Add `AuditCallbackHandler` for mandatory audit log commits before LLM call (NFR-16a).
- Use `run_in_executor` for `generate` to stop FastAPI block.
### 2. BERT Adapter (`backend/implementation/adapters/bert_adapter.py`)
- Build `BERTAdapter` stubs:
- `drift_check`: Find clinical drift.
- `referee_check`: Check RAG grounding.
- `guardrail_check`: Safety check tokens/chunks.
### 3. Safety Service (`backend/implementation/safety/service.py`)
- Link LLM + BERT adapters.
- Build `_verify_pre_egress` check:
- **Consent**: Check `consent:{session_id}` in Redis.
- **Redaction**: Check redaction manifest hash.
- Update services:
- `rationale`: Context explanation.
- `socratic_chat`: History chat + BERT referee grounding.
- `guardrail_check`: Link BERT guardrails.
- `chat_stream`: SSE generator + per-chunk filter.
- Add post-egress: set `consult_mode:{session_id}` $\rightarrow$ `cloud_vertex` in Redis.
### 4. Infrastructure & API
- **Redis Adapter**: Build `backend/implementation/adapters/redis_adapter.py` for state + consent.
- **Config**: Update `backend/implementation/config.py` with Vertex AI + Redis env vars.
- **API Router**: Update `backend/api/safety_api.py` for `redaction_hash` + SSE errors.
## Governance Flow (NFR-16a)
`Request` $\rightarrow$ `Consent Check (Redis)` $\rightarrow$ `Redaction Verify` $\rightarrow$ `Audit Commit (Langchain Callback)` $\rightarrow$ `LLM Call` $\rightarrow$ `Post-Egress Log` $\rightarrow$ `Response`

View File

@@ -0,0 +1,37 @@
# Session Memory: 2026-06-27 Backend NLP/LLM Consult Implementation
## Task
Implement Vietnamese Clinical LLM Consult endpoints (Rationale, Socratic Chat, Guardrail Check, SSE Stream) with strict NFR-16a governance.
## Implementation Details
### 1. LLM Adapter (`backend/implementation/adapters/llm_adapter.py`)
- Implemented `VertexAILangchainAdapter` using `langchain-google-vertexai`.
- Added `AuditCallbackHandler` to enforce mandatory audit log commits *before* LLM calls (NFR-16a).
- Wrapped synchronous `generate` calls in `run_in_executor` to avoid blocking FastAPI event loop.
### 2. BERT Adapter (`backend/implementation/adapters/bert_adapter.py`)
- Implemented `BERTAdapter` with stubs for:
- `drift_check`: Detects clinical domain drift.
- `referee_check`: Validates RAG grounding.
- `guardrail_check`: Token/chunk level safety verification.
### 3. Safety Service (`backend/implementation/safety/service.py`)
- Integrated LLM and BERT adapters.
- Implemented `_verify_pre_egress` checklist:
- **Consent**: Validates `consent:{session_id}` exists in Redis.
- **Redaction**: Verifies redaction manifest hash.
- Updated services:
- `rationale`: Context-aware explanation generation.
- `socratic_chat`: History-based conversation with BERT referee grounding.
- `guardrail_check`: Direct integration with BERT guardrails.
- `chat_stream`: SSE generator with per-chunk guardrail filtering.
- Added post-egress state update: sets `consult_mode:{session_id}` to `cloud_vertex` in Redis.
### 4. Infrastructure & API
- **Redis Adapter**: Created `backend/implementation/adapters/redis_adapter.py` for session state and consent management.
- **Config**: Updated `backend/implementation/config.py` with Vertex AI and Redis environment variables.
- **API Router**: Updated `backend/api/safety_api.py` to pass `redaction_hash` to services and handle SSE error events.
## Governance Flow (NFR-16a)
`Request` $\rightarrow$ `Consent Check (Redis)` $\rightarrow$ `Redaction Verify` $\rightarrow$ `Audit Commit (Langchain Callback)` $\rightarrow$ `LLM Call` $\rightarrow$ `Post-Egress Log` $\rightarrow$ `Response`

View File

@@ -0,0 +1,301 @@
# Frontend Development Progress - 27 Jun 26
## Session Summary
Date: 2026-06-27
Phase: 2 - Component Architecture & Deepening (35 Figma Designs Implementation Plan)
Mode: Caveman Lite
---
## 1. TopBar Refactor (Phase 2.0)
### Problem
TopBar reached into Zustand store for 4 fields (`patientId`, `clinicianGrade`, `isOverriding`, `sessionId`, `currentPhase`) and inlined sign button. Violated molecule abstraction layer.
### Solution
Refactored `TopBar` to consume `PatientHeader` and `ActionCluster` molecules via props instead of store selectors.
### Files Changed
- `src/components/organisms/TopBar.tsx`
- Removed direct `useDiagnosticWorkspaceStore` import
- Added props interface: `patientId`, `patientName`, `role`, `status`, `sessionId`, `currentPhase`, `canSign`, `isCircuitBroken`, `onSign`, `onCircuitBreak`, `onBack`, `onMore`
- Replaced inline patient info + sign button with `<PatientHeader />` and `<ActionCluster />` molecules
- Added `canSign` prop to `ActionCluster` for HITL gating
- `src/components/molecules/ActionCluster.tsx`
- Added `canSign?: boolean` prop
- Added `signDisabled` computed value: `isCircuitBroken || !canSign`
- Sign button now uses `disabled={signDisabled}` instead of just `isCircuitBroken`
- `src/components/templates/WorkspaceShell.tsx`
- Wired all TopBar props from store using selectors
- Passes `patientId`, `sessionId`, `currentPhase`, `canSign`, `isCircuitBroken`, `onSign`, `onCircuitBreak`
- `onCircuitBreak` now accepts `(active: boolean) => void` and triggers `window.confirm` modal for break/recovery
---
## 2. Canvas & Visualization Deepening (Phase 2.1)
### DiagnosticCanvas Enhancements
- Tool State: reads `activeTool`, `setActiveTool`, `zoomLevel`, `panOffset`, `showGradCAM`, `showSegmentation` from Zustand `uiSlice`
- 5-Layer Canvas Stack: 5 layers with dynamic rendering
- Layer 0: Cornerstone.js DICOM viewport
- Layer 10: GradCAM overlay (conditional via `showGradCAM`)
- Layer 50: Segmentation mask (conditional via `showSegmentation`)
- Layer 100: SVG Annotations + OffscreenCanvas
- Layer 200: Viewport HUD + live coords/zoom
- Performance:
- `requestAnimationFrame` dirty-flag loop for annotation canvas
- `ResizeObserver` syncs annotation canvas resolution
- Cornerstone lifecycle: init + viewport update (no reload on pan/zoom)
- `useCallback` for event handlers
- Touch:
- `onTouchStart`, `onTouchMove` for mobile/tablet coords
- Touch events mapped to same coordinate system
- Tool Palette:
- Extracted constant `TOOLS` array with icon mapping (`pin``map-pin`)
- Active tool state store-driven
- Live HUD:
- Replaced static HUD with live `mousePos` state
- Zoom level from store
### Canvas Performance
- Minimized Cornerstone redraws: separate image load from viewport updates
- `useRef` for OffscreenCanvas, RAF loop, dirty flag, image tracking
- Added `mousePos` for live coords
---
## 3. Reasoning & Chat Panel Deepening (Phase 2.2)
### ZoneB Composition
- Replaced placeholder with `PhaseIndicator` + `ReasoningPanel` + `ChatPanel`
- Layout: Sticky PhaseIndicator top, scrollable content below
- `ZoneBProps` interface covers all data flows
### ReasoningPanel Enhancements
- Added optional `PhaseIndicator` via `showPhaseIndicator` prop
- Classification Card:
- Color-coded grade badge (0-3) with tooltip
- Confidence score with `ConfidenceBar`
- Explanation Panel:
- GradCAM thumbnail with hover overlay
- Thickness measurement extraction/display
- Show Full GradCAM toggle via `showFullGradCam` / `onToggleFullGradCam` props
- Override Card:
- Rendered when `onSubmitOverride` provided
- Composes existing `OverrideCard` molecule
- Quick Actions Bar:
- `CircuitBreaker` molecule
- `NFRIndicator` molecule
- Report button
- EMR sync button
- Fullscreen toggle
### ChatPanel Enhancements
- Input:
- Replaced `<input>` with auto-resizing `<textarea>`
- `Enter` sends
- `Shift+Enter` inserts newline
- Auto-resize to 120px
- Streaming:
- Added `isStreaming` prop
- Streaming indicator in header
- Inline pulsing cursor on last assistant message when streaming
- Message Actions:
- Hover-revealed toolbar on non-system messages
- Copy button (`navigator.clipboard.writeText`)
- Regenerate button (assistant only)
- Feedback buttons: helpful (heart) / not helpful (alert-circle)
- All optional via callback props
- Virtualization:
- Placeholder: `react-window` `FixedSizeList` integration point
- Current: standard overflow-y-auto
---
## 4. Tool State & Annotations (Phase 2.3)
### uiSlice Additions
- `brushSize: number` (default 5, range 1-50)
- `brushColor: string` (default '#ff0000')
- `eraserSize: number` (default 10, range 1-100)
- `caliperUnit: 'mm' | 'px'` (default 'mm')
- `pressureSensitivity: boolean` (default false)
- Setters with validation
### annotationSlice Additions
- `selectedAnnotationId: string | null`
- `lockedLayerIds: string[]`
- Actions:
- `addAnnotation`
- `selectAnnotation`
- `deleteAnnotation`
- `updateAnnotation`
- `lockLayer`
- Derived selectors:
- `selectSelectedAnnotation`
- `selectLockedLayers`
- Undo/redo for all annotation mutations
### Store Updates
- `store/index.ts`:
- Exported new selectors
- Updated `resetSession` to clear new state fields
---
## 5. Safety & Compliance Wiring (Phase 2.4)
### Circuit Breaker
- `ActionCluster` passes `isCircuitBroken` state
- Sign button disabled when circuit broken
- `WorkspaceShell` applies `ring-2 ring-red-500 animate-pulse` when `isOverriding=true`
- Confirmation via `window.confirm` before break/recovery
### Defer 13 Scrubber (PIIScrubber)
- Fixed broken destructuring in `src/components/molecules/PIIScrubber.tsx`
- Added missing props: `text`, `onChange`, `className`
- Compiles and binds props correctly
### HITL Signature Gate
- Fixed missing `Input` import in `src/components/molecules/HITLGate.tsx`
- Added `import { Input } from '../atoms/Input';`
- Ready for credential entry modal
### NFR Status Indicator (BottomBar)
- Added `WorkerHealthEntry` interface
- Extended `BottomBarProps` with `workerHealth` and `isOffline`
- Integrated `NFRIndicator` for live status
- Removed hardcoded 'offline' when real values present
---
## 6. TypeScript & Lint Verification
### TypeScript
- `tsc --noEmit` passes clean
- All modified files compile clean
### ESLint
- All modified files pass lint
- Pre-existing warnings: stories/tests (.stories.tsx old Storybook imports, test `.set` destructuring)
- Not introduced this session
---
## 7. Plan File Updates
Updated `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/.kilo/plans/35-figma-designs-implementation-plan.md`:
- Tick 2.0 TopBar deepening (molecule consumption)
- Tick 2.1 Canvas & Visualization
- Tick 2.2 Reasoning & Chat Panel (Override Card + Quick Actions)
- Tick 2.3 Interaction & State Management (tool state, annotations, measurements)
- Tick 2.4 Safety & Compliance (Circuit Breaker, PII Scrubber, HITL Gate, NFR Indicator)
---
## 8. Known Pre-existing Issues (Not Introduced This Session)
- `src/services/safetyService.ts`: Vite build error (unrelated to Phase 2)
- Storybook stories: import `@storybook/react` directly instead of framework package
- Store slices tests: unused `set` destructuring warnings
---
## 9. Key Architectural Improvements
1. **Molecule Layer Strengthened**: TopBar no longer leaks store selectors; uses PatientHeader + ActionCluster props
2. **5-Layer Canvas Stack**: Dynamic store-driven visibility with OffscreenCanvas annotations and RAF rendering
3. **ZoneB Composed**: Clean separation of PhaseIndicator, ReasoningPanel, ChatPanel
4. **Tool State Centralized**: All tool preferences in uiSlice with validation
5. **Annotation System Ready**: Vector storage, undo/redo hooks, layer locking
6. **Safety Molecules Wired**: CircuitBreaker, PIIScrubber, HITLGate, NFRIndicator integrated into organisms
7. **Zero New Type Errors**: All changes type-check clean
---
## 10. Next Steps
- Wire Cornerstone DICOM loading (placeholder)
- Implement react-window virtualization in ChatPanel at message threshold
- Add Worker hooks (Phase 3.1)
- Implement Offline/Dexie sync (Phase 3.2)
- Build safety modals (CircuitBreaker confirm, HITL credential entry)
- Create 5-layer rendering tests
---
## 11. Phase 3: Worker Integration + Offline Sync (Appended 2026-06-28)
### Worker Hooks
- `src/hooks/workers/useCVWorker.ts`: DICOM frame → CLAHE resize → `cv.worker` → angle class + confidence. Real worker + mock adapter.
- `src/hooks/workers/useLLMWorker.ts`: prompt build + RAG context → `llm.worker` stream with token buffering for smooth display.
- `src/hooks/workers/useGuardrailWorker.ts`: pre-scan user input for hallucination/mal-intent → safety score → UI warning if threshold exceeded.
- `src/hooks/workers/useDriftScore.ts`: extract drift score from guardrail module.
- `src/hooks/workers/useSocraticChat.ts`: socratic dialogue via `llm.worker`.
- `src/hooks/workers/mocks.ts`: mock worker adapters for test environment using setTimeout.
- `src/workers/lifecycle.ts`: `WorkerPool` (lazy init, idle GC 5 min, error boundary).
- Barrel: `src/workers/index.ts` exports workerPool + message helpers.
### Offline Storage + Sync
- `src/services/db.ts` Dexie schema: `sessions`, `chatMessages`, `pendingReports`, `auditLog`.
- `src/services/sync.ts` SyncEngine: online/offline detect, queue when offline, exponential backoff retry (1s base, 30s cap, 5 max), LWW conflict resolution.
- `src/services/index.ts` exports `db`, `syncEngine`.
### PWA / Service Worker
- `vite.config.ts`: `vite-plugin-pwa` `generateSW` (Workbox). CacheFirst static assets, NetworkFirst API runtime, precache manifest.
- `npm i -D vite-plugin-pwa` added.
### Build Verification
- Pre-existing TS errors unchanged in `src/services/adapters/`, `src/*Service.ts`, store tests, stories. None introduced in new files.
---
## 12. Phase 4: Performance, NFR, a11y, i18n (Appended 2026-06-28)
### Bundle + Rendering
- `vite.config.ts`: `rollup-plugin-visualizer` (gzipSize, `dist/bundle-stats.html`). `manualChunks`:
- `vendor-media`: `cornerstone-*`, dicom parsers
- `vendor-store`: dexie, zustand
- `worker`: `src/workers/*`
- `app-core`: react, react-dom, i18next, react-router-dom, date-fns
- `DiagnosticCanvas.tsx`: `useMemo` on viewport transform.
- `ChatPanel.tsx`: `React.memo` `MessageBubble` with comparator `id`, `content`, `isStreaming`.
- `src/services/db.ts`: `saveChatMessagesWithEviction(messages, maxPerSession=20)` caps chat history via FIFO bulkPut + bulkDelete.
### i18n
- `src/i18n.ts`: i18next + `initReactI18next` + `i18next-http-backend`. Lng = `navigator.language` if starts with `en`, else `en`. `/locales/{{lng}}.json`.
- `src/locales/en.json`, `src/locales/vi.json`: 37 UI keys across TopBar, PatientHeader, BottomBar, ChatPanel, ReasoningPanel, WorkspaceShell.
- `src/App.tsx`: root wrapped in `<I18nextProvider i18n={i18n}>`.
- ReasoningPanel/BottomBar/TopBar/ChatPanel: hardcoded strings replaced with `t('Component.key')`.
### Accessibility
- `src/components/templates/WorkspaceShell.tsx`: root wrapper gets `role="main"`, `tabIndex={-1}`.
- `src/components/organisms/BottomBar.tsx`: NFR status container `aria-live="polite"`.
- `src/index.css`: global `:focus-visible` outline 2px solid var(--color-primary-500), offset 2px.
- Grad-CAM `<img alt="Grad-CAM">` retained; decorative icons already `aria-hidden="true"`.
### NFR / Lockdown
- `src/nfr/LockdownValidator.ts`: runtime DOM + fetch + XHR patch. Returns external URLs not matching `window.location.origin`.
- `src/services/adapters/httpAdapter.ts`: `VITE_API_BASE_URL` must start with `/` (relative) else throw. Same-origin only.
- `vite.config.ts`: `server.host='127.0.0.1'`, `server.https=false`. `preview.headers` CSP:
- `script-src 'self' 'wasm-unsafe-eval'`
- `style-src 'self' 'unsafe-inline'`
- `img-src 'self' data: https://fonts.gstatic.com`
- `font-src 'self' https://fonts.gstatic.com`
- `connect-src 'self' https://fonts.googleapis.com`
- Repo grep found zero external URL strings in `src/` and `public/`. Workbox retained Google Fonts runtime caching as sole allowed external origin.
### Bugfixes
- `ReasoningPanel.tsx`: missing `function ReasoningPanel(props)` caused syntax error; fixed + props destructured. `nfrStatus` type inline to avoid broken re-export path.
- `i18n.ts`: `useTranslation` export fixed; `HttpBackend` added as plugin.
- `ZoneB.tsx`: `import { ReasoningPanel }` → default import.
### Plan Updates
- `.kilo/plans/35-figma-designs-implementation-plan.md`:
- Phase 3.1 + 3.2 ticked during worker/offline work.
- Phase 4.1-4.3 items ticked where verification supports.
### Known Compile State
- `npm run build` fails on pre-existing unrelated files (dicomService typings, store tests, stories missing modules). New Phase 3/4 files compile clean.

View File

@@ -0,0 +1,225 @@
# Frontend Development Progress - 27 Jun 26
## Session Summary
Date: 2026-06-27
Phase: 2 - Component Architecture & Deepening (35 Figma Designs Implementation Plan)
Mode: Caveman Lite
---
## 1. TopBar Refactor (Phase 2.0)
### Problem
TopBar was directly reaching into the Zustand store for 4 fields (patientId, clinicianGrade, isOverriding, sessionId, currentPhase) and inlining the sign button. This violated the molecule abstraction layer.
### Solution
Refactored `TopBar` to consume `PatientHeader` and `ActionCluster` molecules via props instead of store selectors.
### Files Changed
- `src/components/organisms/TopBar.tsx`
- Removed direct `useDiagnosticWorkspaceStore` import
- Added props interface: `patientId`, `patientName`, `role`, `status`, `sessionId`, `currentPhase`, `canSign`, `isCircuitBroken`, `onSign`, `onCircuitBreak`, `onBack`, `onMore`
- Replaced inline patient info + sign button with `<PatientHeader />` and `<ActionCluster />` molecules
- Added `canSign` prop to `ActionCluster` for HITL gating
- `src/components/molecules/ActionCluster.tsx`
- Added `canSign?: boolean` prop
- Added `signDisabled` computed value: `isCircuitBroken || !canSign`
- Sign button now uses `disabled={signDisabled}` instead of just `isCircuitBroken`
- `src/components/templates/WorkspaceShell.tsx`
- Wired all TopBar props from store using selectors
- Passes `patientId`, `sessionId`, `currentPhase`, `canSign`, `isCircuitBroken`, `onSign`, `onCircuitBreak`
- `onCircuitBreak` now accepts `(active: boolean) => void` and triggers `window.confirm` modal for break/recovery
---
## 2. Canvas & Visualization Deepening (Phase 2.1)
### DiagnosticCanvas Enhancements
- Tool State Management: Component now reads `activeTool`, `setActiveTool`, `zoomLevel`, `panOffset`, `showGradCAM`, `showSegmentation` from Zustand `uiSlice`
- 5-Layer Canvas Stack: All 5 layers implemented with proper dynamic rendering
- Layer 0: Cornerstone.js DICOM viewport
- Layer 10: GradCAM overlay (conditionally rendered via `showGradCAM` store toggle)
- Layer 50: Segmentation mask (conditionally rendered via `showSegmentation` store toggle)
- Layer 100: SVG Annotations layer with OffscreenCanvas support
- Layer 200: Viewport HUD with live mouse coordinates and zoom level
- Performance Optimizations:
- `requestAnimationFrame` dirty-flag render loop for annotation canvas
- `ResizeObserver` to keep annotation canvas resolution in sync
- Separated Cornerstone lifecycle into initialization + viewport update effects (no reload on pan/zoom)
- `useCallback` for event handlers
- Touch Support:
- `onTouchStart`, `onTouchMove` handlers for mobile/tablet coordinate tracking
- Touch events mapped to same coordinate system as mouse
- Tool Palette:
- Extracted to constant `TOOLS` array with proper icon mapping (`pin` corrected to `map-pin`)
- Active tool state now store-driven, not prop-driven
- Live HUD:
- Replaced static "X: 124.5mm | Y: 88.2mm" with live `mousePos` state
- Zoom level pulled from store
### Canvas Performance
- Minimized Cornerstone redraws by separating image load from viewport updates
- Used `useRef` for OffscreenCanvas, RAF loop, dirty flag, image tracking
- Added `mousePos` state for live coordinate display
---
## 3. Reasoning & Chat Panel Deepening (Phase 2.2)
### ZoneB Composition
- Replaced placeholder with full composition: `PhaseIndicator` + `ReasoningPanel` + `ChatPanel`
- Layout: Sticky PhaseIndicator at top, scrollable content area below
- Comprehensive `ZoneBProps` interface covering all data flows
### ReasoningPanel Enhancements
- Added optional `PhaseIndicator` (controlled by `showPhaseIndicator` prop)
- Classification Card:
- Color-coded grade badge (0-3) with tooltip wrapper
- Confidence score with `ConfidenceBar`
- Explanation Panel:
- GradCAM thumbnail with hover overlay ("Expand View")
- Thickness measurement extraction and display
- "Show Full GradCAM" toggle via `showFullGradCam` / `onToggleFullGradCam` props
- Override Card:
- Conditionally rendered when `onSubmitOverride` callback provided
- Composes existing `OverrideCard` molecule
- Quick Actions Bar:
- `CircuitBreaker` molecule (toggle)
- `NFRIndicator` molecule
- Report generation button
- EMR sync button
- Fullscreen toggle
### ChatPanel Enhancements
- Input Handling:
- Replaced `<input>` with auto-resizing `<textarea>`
- `Enter` sends message
- `Shift+Enter` inserts newline
- Auto-resize up to 120px height
- Streaming Support:
- Added `isStreaming` prop
- Streaming indicator in header ("Streaming...")
- Inline pulsing cursor on last assistant message when streaming
- Message Actions:
- Hover-revealed action toolbar on non-system messages
- Copy button (uses `navigator.clipboard.writeText`)
- Regenerate button (assistant messages only)
- Feedback buttons: helpful (heart) / not helpful (alert-circle)
- All actions optional via callback props
- Virtualization:
- Placeholder comment block showing `react-window` `FixedSizeList` integration point
- Current implementation uses standard overflow-y-auto for simplicity
---
## 4. Tool State & Annotations (Phase 2.3)
### uiSlice Additions
- `brushSize: number` (default 5, range 1-50)
- `brushColor: string` (default '#ff0000')
- `eraserSize: number` (default 10, range 1-100)
- `caliperUnit: 'mm' | 'px'` (default 'mm')
- `pressureSensitivity: boolean` (default false)
- Setters for all above with validation
### annotationSlice Additions
- `selectedAnnotationId: string | null`
- `lockedLayerIds: string[]`
- Actions:
- `addAnnotation`
- `selectAnnotation`
- `deleteAnnotation`
- `updateAnnotation`
- `lockLayer`
- Derived selectors:
- `selectSelectedAnnotation`
- `selectLockedLayers`
- Undo/redo integration for all annotation mutations
### Store Updates
- `store/index.ts`:
- Exported new selectors
- Updated `resetSession` to clear new state fields
---
## 5. Safety & Compliance Wiring (Phase 2.4)
### Circuit Breaker
- `ActionCluster` updated to pass `isCircuitBroken` state
- Sign button disabled when circuit broken
- `WorkspaceShell` applies `ring-2 ring-red-500 animate-pulse` when `isOverriding=true`
- Confirmation modal via `window.confirm` before break/recovery
### Defer 13 Scrubber (PIIScrubber)
- Fixed broken destructuring in `src/components/molecules/PIIScrubber.tsx`
- Added missing props: `text`, `onChange`, `className`
- Component now compiles and correctly binds props
### HITL Signature Gate
- Fixed missing `Input` import in `src/components/molecules/HITLGate.tsx`
- Added `import { Input } from '../atoms/Input';`
- Component now ready for credential entry modal
### NFR Status Indicator (BottomBar)
- Added `WorkerHealthEntry` interface
- Extended `BottomBarProps` with `workerHealth` and `isOffline`
- Integrated `NFRIndicator` molecule for live status display
- Removed hardcoded 'offline' status when real values present
---
## 6. TypeScript & Lint Verification
### TypeScript
- `tsc --noEmit` passes clean
- All modified files compile without errors
### ESLint
- All modified files pass lint
- Pre-existing warnings in stories/tests (.stories.tsx using old Storybook imports, test `.set` destructuring)
- These are not introduced by this session's changes
---
## 7. Plan File Updates
Updated `/Users/davestran/Downloads/vkist_internship/PILOT_PROJECT/.kilo/plans/35-figma-designs-implementation-plan.md`:
- Tick off 2.0 TopBar deepening (molecule consumption)
- Tick off 2.1 Canvas & Visualization (all sub-items)
- Tick off 2.2 Reasoning & Chat Panel (all sub-items including Override Card + Quick Actions)
- Tick off 2.3 Interaction & State Management (tool state, annotations, measurements)
- Tick off 2.4 Safety & Compliance Features (Circuit Breaker, PII Scrubber, HITL Gate, NFR Indicator)
---
## 8. Known Pre-existing Issues (Not Introduced This Session)
- `src/services/safetyService.ts`: Vite build error (unrelated to Phase 2 work)
- Storybook stories: import `@storybook/react` directly instead of framework package
- Store slices tests: unused `set` parameter destructuring warnings
---
## 9. Key Architectural Improvements
1. **Molecule Layer Strengthened**: TopBar no longer leaks store selectors; consumed via PatientHeader + ActionCluster props
2. **5-Layer Canvas Stack**: Dynamic, store-driven layer visibility with OffscreenCanvas annotation support and RAF-based rendering
3. **ZoneB Composed**: Clean separation of PhaseIndicator, ReasoningPanel, ChatPanel
4. **Tool State Centralized**: All tool preferences in uiSlice with validation
5. **Annotation System Ready**: Vector-based storage, undo/redo hooks, layer locking in place
6. **Safety Molecules Wired**: CircuitBreaker, PIIScrubber, HITLGate, NFRIndicator integrated into organisms
7. **Zero New Type Errors**: All changes type-check clean
---
## 10. Next Steps
- Wire actual Cornerstone DICOM loading (currently placeholder)
- Implement react-window virtualization in ChatPanel when message count exceeds threshold
- Add Worker hooks (Phase 3.1)
- Implement Offline/Dexie sync (Phase 3.2)
- Build safety modals (CircuitBreaker confirm, HITL credential entry)
- Create actual 5-layer rendering tests

159
session_memory/6_Jul_26.md Normal file
View File

@@ -0,0 +1,159 @@
---
title: Session Memory 6 Jul 26
date: 2026-07-06
status: active
---
## Summary
Sprint_1_2 clinical chat saw major LLM UX, streaming, and inference work (Jul 56). The highest-risk open issue is **frontend markdown rendering of generated assistant text** — it has caused tab freezes and catastrophic crashes during reasoning streams. Reasoning paths now use **plain text only** (`StreamingPlainText`) as a mitigation. **Beta functionality** (Planning mode + 🔥 high reasoning level) must be completed **by end of this week** (target: **Friday 10 Jul 2026**).
---
## Change log (what was updated)
### Clinical chat UI & model lifecycle
| Area | Change | Key files |
|------|--------|-----------|
| LLM loading bubble | Install vs load phases with distinct copy, progress bar, disabled composer | `ClinicalChatPanel.tsx`, `useClinicalChat.ts`, `modelLoadProgress.ts` |
| Install vs load semantics | First OPFS download (~1.9 GB) vs cached checkpoint init into worker/GPU | `useClinicalChat.ts` (`ModelLoadPhase: 'installing' \| 'loading'`) |
| Sidebar card switch | Both diagnosis + review layers stay mounted (CSS hide/show) so Gemma is not torn down on carousel switch | `SidebarLayerCarousel.tsx` |
| OPFS persistence | Completed install survives reload; interrupted download not resumable (manifest only written on success) | `opfsModelStore.ts` |
| Worker init progress | `init_progress` events wired through `LlmWorkerClient.init(onProgress)` | `llmWorkerClient.ts`, `llm.worker.ts` |
### Inference modes, reasoning levels & backends
| Area | Change | Key files |
|------|--------|-----------|
| Unified chat modes | `ask` merged into `chat`; inference modes: **Chat**, **Planning** (beta), **Agent** | `clinicalChatModes.ts`, `analyzePromptComplexity.ts` |
| Reasoning levels | 🧘 chill / 🤔 moderate / 🔥 high (beta); bar visibility persisted | `chatReasoningLevel.ts`, `ClinicalChatPanel.tsx` |
| Edge vs server toggle | 🤔 moderate: **Máy** (Gemma 4 E2B edge) vs **Server** (Gemma 4 E4B Modal Ollama `think:true`) | `reasoningModelBackend.ts`, `ollamaLlmClient.ts`, `inferenceBackend.ts` |
| Ollama dev proxy | `VITE_OLLAMA_CHAT_URL=/api/ollama-chat/api/chat`, model `gemma4:e4b` | `.env.development`, `vite.config.ts` |
| Agent mode | Uses Modal Ollama E4B when configured; tool loop unchanged | `clinicalChatModes.ts`, `runClinicalChatTurn.ts` |
| OOM mitigation | Bootstrap at 2048 tokens; `releaseInference()` before reload; reuse engine when `configured.maxTokens >= required` | `clinicalChatConfig.ts`, `llm.worker.ts`, `llmModelBootstrap.ts` |
| Qwen3 experiment | Dual-model (Qwen `.litertlm` for moderate) attempted then **disabled**`usesQwenReasoningLevel()` returns `false`; moderate uses Gemma CoT again | `qwenOpfsModelStore.ts`, `reasoningLlmClient.ts`, `chatReasoningLevel.ts` |
### Streaming, CoT split & markdown
| Area | Change | Key files |
|------|--------|-----------|
| CoT split | `splitGemmaThoughtOutput`, `isThoughtChannelComplete`; thought vs answer channels in message state | `prompts.ts`, `clinicalChat.ts`, `useClinicalChat.ts` |
| Collapsible reasoning | `ClinicalChatThought` — expand while streaming, auto-collapse when thought completes | `ClinicalChatThought.tsx` |
| Stream throttle | RAF-coalesced updates (~60/s) to reduce main-thread pressure | `streamUpdateThrottle.ts` |
| Token-by-token Ollama | Imperative DOM via `clinicalChatStreamRegistry` + `StreamingPlainText` (bypasses React 18 batching) | `clinicalChatStreamRegistry.ts`, `StreamingPlainText.tsx`, `ollamaLlmClient.ts` |
| Markdown renderer | Custom `ChatMarkdown` (bold, italic, code, lists, headings) — **deferred until stream ends** via `requestIdleCallback` + `startTransition` | `ChatMarkdown.tsx`, `ClinicalChatMessageBubble.tsx` |
| Reasoning = plain text | When `tracksThought`, thought + answer use `StreamingPlainText` only — **no `ChatMarkdown`** | `ClinicalChatThought.tsx`, `ClinicalChatMessageBubble.tsx` |
### Agent tools & Modal testing
| Area | Change | Key files |
|------|--------|-----------|
| Tool catalog doc | Walkthrough of Edge-LLM agent tools: `exa_search`, `supabase_query`, `escalate_medgemma` | session + `agent_tools_contract.md` |
| 3-layer smoke harness | Layer 0 Modal Ollama, Layer 1 BFF routes, Layer 2 browser `ToolExecutor` | `ml/tests/agent_tools/` |
| Python reference tests | Modal `/api/chat` streaming + thinking cases | `PILOT_PROJECT/tmp/test_endpoint.py`, `test_endpoint_img.py` |
| Gemma4 E4B deploy script | Modal Ollama serverless for Gemma 4 E4B | `PILOT_PROJECT/tmp/GemmaE4B_ollama_deploy.py` |
| gemma4_e2b lab | Tool smoke panel (no Gemma) for isolated tool calls | `ml/tests/gemma4_e2b/src/lib/toolSmoke.ts` |
---
## Critical issue: markdown rendering → catastrophic crash
### Symptom
Rendering **LLM-generated markdown** in the clinical chat UI (especially during or immediately after **reasoning / CoT streams**) has caused:
- Main-thread freezes (composer unresponsive while tokens still arrive)
- Tab crashes / OOM under combined **WebGPU model memory + large text buffers + markdown parse**
### Root cause (confirmed in debugging)
1. **Per-token full re-parse** — early implementation ran `ChatMarkdown` / `renderBlocks()` on the entire growing thought string every token → hundreds of parses per second.
2. **React re-render storm**`setMessages` on every token re-rendered the full message list.
3. **Post-stream markdown** — even with “parse after stream ends”, heavy `renderBlocks()` on long CoT + answer text still spikes CPU/memory.
4. **Dual large strings**`rawAccumulator`, `thoughtContent`, and `content` can all hold 2048-token traces simultaneously at 🤔 moderate.
### Mitigations applied (6 Jul)
- Stream updates throttled (`createStreamUpdateThrottle`)
- Plain text while `streaming === true`
- `React.memo` on `ClinicalChatMessageBubble`
- Reasoning paths (`tracksThought`) → **`StreamingPlainText` only, markdown disabled**
- Ollama path → imperative DOM updates per token
- `ChatMarkdown` uses idle callback + `startTransition` for post-stream formatting (chill / non-reasoning answers only)
### Current policy
> **Be wary of re-enabling markdown in reasoning mode until a safe renderer is proven.**
`ClinicalChatThought.tsx` comment: *"Reasoning panel — plain text only (markdown disabled while isolating stream crashes)."*
### Follow-up for later sprints
- [ ] Reproduce crash with a minimal `ChatMarkdown` + long fixture (no LLM) — isolate `renderBlocks` vs React
- [ ] Consider `react-markdown` with strict plugins OR server-side pre-render for final answer only
- [ ] Cap thought trace length in UI (truncate + “show more”)
- [ ] Virtualize message list for long sessions
- [ ] Re-test 🔥 high mode and Planning mode with any new markdown path before removing plain-text guard
- [ ] Audit whether chill-mode `ChatMarkdown` after stream is safe on 8 GB RAM devices
---
## Beta functionality — deadline end of week
**Target: complete by Friday 10 Jul 2026** (end of sprint week).
Features still marked `beta: true` (UI tab/button disabled until ready):
| Feature | ID | Location | What “done” means |
|---------|-----|----------|-------------------|
| **Planning mode** | `planning` | `clinicalChatModes.ts` | Checklist output stable; remove `beta` flag; selectable in mode tabs |
| **High reasoning** | `high` 🔥 | `chatReasoningLevel.ts` | 2048-token multi-turn works without crash; remove `beta` flag |
Related work not yet beta-flagged but in scope:
- Agent tool integration (Layer 3 after smoke tests pass)
- Live BFF tools (`VITE_CLINICAL_CHAT_MOCK_TOOLS=false` + credentials)
- Edge/server reasoning toggle polish (default server E4B when Modal up)
---
## Architecture snapshot (clinical LLM, 6 Jul EOD)
```
User message
├─ Mode: chat ─┬─ 🧘 chill → Gemma E2B edge (no CoT)
│ ├─ 🤔 moderate → Máy: Gemma E2B + CoT (plain text)
│ │ Server: Gemma E4B Ollama think:true (plain text)
│ └─ 🔥 high (BETA) → disabled in UI
├─ Mode: planning (BETA) → disabled in UI
└─ Mode: agent → Gemma E4B Ollama + tools (exa, supabase, medgemma)
Display:
tracksThought → StreamingPlainText (thought panel + answer)
else → StreamingPlainText while streaming → ChatMarkdown when done
```
---
## Key env / endpoints (dev)
```env
VITE_CLINICAL_CHAT_USE_LLM=true
VITE_OLLAMA_CHAT_URL=/api/ollama-chat/api/chat
VITE_OLLAMA_MODEL=gemma4:e4b
VITE_CLINICAL_CHAT_MOCK_TOOLS=true # flip false for live agent tools
```
Modal Gemma E4B (reference): `PILOT_PROJECT/tmp/test_endpoint.py``dtj-tran--ollama-gemma4-e4b-ollamaserver-web.modal.run`
---
## Session references
- Prior frontend memory: `session_memory/27_jun_26/27_jun_26_frontend.md`
- Agent tools smoke README: `CODEBASE/ml/tests/agent_tools/README.md`
- Today's active dev: `npm run dev` on frontend implementation

View File

@@ -0,0 +1,554 @@
# VKIST MSK Pilot — Intern Onboarding Index
**Date:** 7 Jul 2026
**Audience:** Phuc (ML / agentic), Quan (frontend / PWA), Dave Tran - Dat (lead)
**Repo:** `pilot_msk_ultrasound_stack` — clone already shared via GitHub
Read this file first, then open your role-specific guide:
| Track | Document |
| ---------------------------------------------- | ------------------------------------------------------------------ |
| **Phuc** — generative logic, agent, guardrails | [ONBOARDING_PHUC_ML_ENGINEER.md](./ONBOARDING_PHUC_ML_ENGINEER.md) |
| **Quan** — mobile UI, Figma, Stitch, PWA | [ONBOARDING_QUAN_FRONTEND.md](./ONBOARDING_QUAN_FRONTEND.md) |
> **Abbreviations:** See [§12 Glossary](./ONBOARDING_INDEX.md#12-glossary-of-abbreviations) — FR, NFR, RAG, CoT, PHI, PWA, etc.
---
## 1. What this project is
**VKIST MSK Pilot** moves academic musculoskeletal ultrasound ML (angle → inflammation → segmentation → synovitis grade 03) into a clinical web platform for Vietnamese public hospitals (Bach Mai, Viet Duc class facilities).
| Dimension | Summary |
| ---------- | ------------------------------------------------------------------------------------------------------- |
| **Cycle** | Jun 2 Sep 2, 2026 (strict 3-month window) |
| **Method** | Hybrid: waterfall phases + agile sprints |
| **Stack** | React PWA + FastAPI + Triton CV + 3-tier LLM (Edge Gemma → Gemini → MedGemma) + RAG over MOH guidelines |
| **Users** | Radiologists, surgeons, physiotherapists, patients/caregivers — each with different UI needs |
**Hard constraints (non-negotiable):**
- **Decree 13/2023** — no raw PHI leaves device without scrubbing
- **No open-ended patient chat** — structured, gated summaries only
- **Air-gapped hospital LAN** — cloud LLM only under NFR-16a (consent + audit + redaction)
- **Legacy smartphones** — zero-GPU fallback rendering for patient fleet
Full vision: `[PILOT_PROJECT/PROJ_LEVEL_READING/PLAN/CONTEXT_VISION_SCOPE.md](../../PILOT_PROJECT/PROJ_LEVEL_READING/PLAN/CONTEXT_VISION_SCOPE.md)`
**Tóm tắt (VI):** Dự án AI siêu âm khớp gối cho bệnh viện VN; 3 tháng; bảo mật Decree 13; các tinh năng hiện giờ đang optimize cho người dùng chính là bác sĩ cơ xương khớp - trong tương lai có thể cân nhắc `bệnh nhân` là đối tượng phát triển; làm tiếp phần Dave đã xây, không làm lại từ đầu.
---
## 2. How we work — continuation engineering
You are **not** here to read docs for three weeks. You inherit Dave's foundation and ship **incremental PRs** that close spec gaps toward sprint checkpoints.
```mermaid
flowchart LR
foundation[Dave_Foundation]
specs[Specs_and_UCs]
backlog[Open_Backlog]
checkpoint[Sprint_Checkpoint]
internPR[Intern_Incremental_PR]
foundation --> internPR
specs --> backlog
backlog --> internPR
internPR --> checkpoint
```
1. Dave assigns **one backlog ID** (P-n or Q-n).
2. You read **only** the spec + code files for that ticket.
3. Ship a small PR (≤300 lines preferred); demo something runnable weekly.
4. Dave assigns the next ticket.
**Tóm tắt (VI):** Dave giao ticket → đọc đúng spec → PR nhỏ → demo hàng tuần → ticket tiếp theo.
---
## 3. Foundation already built (do not rebuild)
Dave's latest handoff: `[session_memory/6_Jul_26.md](../6_Jul_26.md)`
| Area | Status | Key locations |
| --------------------- | ------------------------ | ------------------------------------------------------------------------- |
| FR-25 workspace shell | **Working PoC** | `ClinicalWorkspacePage.tsx`, `WorkspaceShell.tsx`, `DiagnosticCanvas.tsx` |
| CV inference | **Live** (FastAPI proxy) | `POST /api/test/analyze/batch`, `mlInferenceCacheStore.ts` |
| Clinical chat UI | **Working** | `ClinicalChatPanel.tsx`, `useClinicalChat.ts` |
| Edge Gemma 4 E2B | **Working** | `llm.worker.ts`, `clinicalLlmRuntime.ts`, OPFS |
| CoT streaming | **Working** (plain text) | `clinicalChatStreamRegistry.ts`, `StreamingPlainText.tsx` |
| Agent runtime | **Working** (mock tools) | `agent_runtime/`, `agentLoop.ts` |
| Cloud BFF | **Wired** | `cloud_orchestrate.py`, `cloud_consult.py`, `agent_tools.py` |
| 35 Figma routes | **Stub pages** | `figma-component-specs.md` |
| Safety / guardrails | **Spec + stubs** | `bert_adapter.py`; `guardrail.worker.ts` missing |
| Patient PWA | **Not started** | Sprint 4 |
**Tóm tắt (VI):** Workspace, chat, agent, inference đã chạy; guardrail và UI mobile/patient còn thiếu — đó là việc của bạn.
---
## 4. Sprint checkpoints (what your PRs advance)
| ID | Checkpoint | Sprint | Window | Owner |
| ---- | ----------------------------------------------- | ------ | -------------- | --------------------- |
| CP-1 | Beta clinical modes (Planning + high reasoning) | 2 | Jul 10 | Dave → Phuc polish |
| CP-2 | Live agent tools (mock → BFF) | 23 | Jul 1424 | **Phuc** |
| CP-3 | Guardrail slice v1 | 23 | Jul 21 Aug 7 | **Phuc** + Dave |
| CP-4 | Collaborative annotation UX | 3 | Jul 1324 | Dave; Quan UI support |
| CP-5 | Patient list/cards → Figma mobile | 34 | Jul 28 Aug 7 | **Quan** |
| CP-6 | Patient PWA (large text, 375px) | 4 | Jul 27 Aug 7 | **Quan** |
| CP-7 | Accessibility on 2 screens | 45 | Aug 421 | **Quan** |
| CP-8 | RAG retrieval gate / referee stub | 35 | Aug 121 | **Phuc** |
**Timeboxes:** Phuc until ~Aug 31; Quan ~2 months (ends ~early Aug).
**Tóm tắt (VI):** Mỗi PR phải đẩy một checkpoint CP-n, không chỉ viết tài liệu.
---
## 5. Open backlog
Dave maintains priority. Do not self-assign large items.
### Phuc — generative / agentic
| ID | Gap | Starting code | Done when |
| --- | -------------------------------- | ----------------------------------------------- | ----------------------------------------------------- |
| P-1 | Agent tools mock | `tools/registry.ts`, `agent_tools.py` | Live BFF; `VITE_CLINICAL_CHAT_MOCK_TOOLS=false` works |
| P-2 | `bert_adapter.py` stubs | `bert_adapter.py`, `safety/service.py` | One real PASS/FAIL check on outbound text |
| P-3 | `guardrail.worker.ts` missing | Create near `llm.worker.ts` | Worker hooks into chat before display |
| P-4 | RAG not enforced in chat mode | `runClinicalChatTurn.ts` | Clinical answers require retrieval context |
| P-5 | Planning + high reasoning `beta` | `clinicalChatModes.ts`, `chatReasoningLevel.ts` | Stable runs; beta flags removed |
| P-6 | `escalate_medgemma` E2E | `cloud_llm_gateway.py`, `tools/registry.ts` | Demo with audit log entry |
| P-7 | Conformal decoding eval | `ml/spec/conformal_decoding_*.md` | 3 VI clinical test cases (research) |
### Quan — frontend / PWA
| ID | Gap | Starting code | Done when |
| --- | ----------------------------- | ----------------------------- | ----------------------------------------------------------------- |
| Q-1 | 35 stub routes | `src/pages/*.tsx` | 2 priority screens match Figma |
| Q-2 | Safety overlay not routed | `SafetyEscalationOverlay.tsx` | Reachable from workspace |
| Q-3 | No mobile breakpoint | Patient pages | 375px pass, touch ≥44px |
| Q-4 | Accessibility open | Patient components | 1 view WCAG-oriented pass |
| Q-5 | No error boundaries | `App.tsx` | Friendly fallback on workspace route |
| Q-6 | Patient dashboard | `patient-card-*.tsx` | Jargon-free, large type, VI copy |
| Q-7 | Zero-GPU fallback UI | TBD banner | UI state when WebGL weak (Dave provides hook) |
| Q-8 | Stitch sync after ship (lite) | Stitch board | Optional: update frame when PR merges; no standalone proposal doc |
**Tóm tắt (VI):** Bảng backlog trên — chờ Dave giao ID cụ thể trước khi code.
---
## 6. Codebase map
**Active code:** `PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/`
**Do not develop in:** `PILOT_PROJECT/workspace/LEGACY/` (read-only reference)
> **Note:** Root `PILOT_PROJECT/README.md` references stale paths (`Reading_docs/`, `CONTEXT.md`). Use paths in this doc instead.
| Module | Path | Phuc | Quan |
| -------------- | --------------------- | ----------------------------------------- | ------------------------------------------------ |
| Frontend | `CODEBASE/frontend/` | `lib/llm/`, `workers/`, `useClinicalChat` | **Primary**`pages/`, `components/`, `styles/` |
| Backend | `CODEBASE/backend/` | **Primary** — routers, safety, adapters | Read-only |
| ML / NLP | `CODEBASE/ml/` | **Primary**`agent_runtime/`, tests | Do not touch |
| Knowledge | `CODEBASE/knowledge/` | When P-4/P-8 | Do not touch |
| Design | `Design_Material/` | Safety, API | UCs, screen specs |
| Session memory | `session_memory/` | `6_Jul_26.md` | `26_Jun_26/Proj_context` for layout |
### Frontend tree (Quan primary)
```
frontend/
├── spec/ ← uix_prototype_coverage, figma-component-specs, DESIGN
└── implementation/src/
├── pages/ ← Quan: screens
├── components/atoms|molecules|shells/
├── styles/design-tokens.ts
├── hooks/ ← Phuc: useClinicalChat only
├── lib/llm/ ← Phuc ONLY
└── workers/ ← Phuc ONLY
```
### ML + backend tree (Phuc primary)
```
ml/implementation/nlp/agent_runtime/src/
orchestrator/agentLoop.ts
tools/registry.ts
prompt/toolSystemPrompt.ts, toolOutputParser.ts
frontend/.../src/lib/llm/ ← edge inference, chat routing
backend/routers/ ← cloud_*, agent_tools
backend/services/ ← cloud_llm_gateway, agent_tools_service
backend/implementation/safety/, adapters/bert_adapter.py
knowledge/spec/ ← RAG when P-4/P-8
```
**Tóm tắt (VI):** Phuc = ml + backend + lib/llm; Quan = frontend pages/components; LEGACY không đụng.
---
## 7. Reading guide (summary)
Each role doc has three tiers. Quick reference:
| Tier | When |
| ---------------- | ---------------------------------- |
| **Must read** | Day 13, before first PR |
| **Should read** | When your ticket touches that area |
| **Learn deeper** | Keywords — grow over internship |
See [Phuc doc](./ONBOARDING_PHUC_ML_ENGINEER.md#3-what-to-read-what-to-learn) and [Quan doc](./ONBOARDING_QUAN_FRONTEND.md#3-what-to-read-what-to-learn) for full lists.
**Tóm tắt (VI):** Đọc bắt buộc trước PR đầu; đọc thêm theo ticket; học sâu theo từ khóa khi cần.
---
## 8. Dev toolkit
### Shared (both)
| Tool | Purpose |
| ----------------- | ----------------------------------------------------------- |
| **Cursor / Kilo** | Agentic coding — load `PILOT_PROJECT/AGENT_SKILL/` first |
| **Git + GitHub** | Branch per ticket; PR ≤300 lines |
| `secrets/` | Local only — see `PILOT_PROJECT/secrets_template/README.md` |
| `session_memory/` | Dave handoff — start with `6_Jul_26.md` |
### Phuc additions
```bash
# Frontend + clinical chat
cd PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation
npm install && npm run dev # :5173
# Gemma lab
cd ../../ml/tests/gemma4_e2b && npm install && npm run dev # :5174
# Agent smoke
cd ../agent_tools && npm install && npm run smoke
# Backend
cd ../../.. && conda activate vkist_ultra
PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py
```
Key env: `VITE_CLINICAL_CHAT_USE_LLM`, `VITE_CLINICAL_CHAT_MOCK_TOOLS`, `VITE_API_BASE_URL`
### Quan additions
```bash
cd PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation
npm install && npm run dev # :5173
```
| Tool | Link |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| **Figma** (authority) | [MSK_Workspace_UIX-Design](https://www.figma.com/design/uakk3T4efS1AIckfTEFSZO/MSK_Workspace_UIX-Design?node-id=0-1) |
| **Stitch** (acceleration) | [Whiteboard 8188910238055386921](https://stitch.withgoogle.com/projects/8188910238055386921) |
| **Responsive test** | Chrome DevTools → iPhone SE 375px |
Stitch board has **web mocks** today — Quan creates **mobile variants** and exports references for the coding agent (see Quan doc).
**Tóm tắt (VI):** Cursor/Kilo + Git; Phuc thêm conda/smoke tests; Quan thêm Figma + Stitch + DevTools mobile.
---
## 9. Bonus — Feynman Technique with agents
When a concept or code path feels **hard**, don't just ask the agent to fix it — ask it to **teach it back to you** using the [Feynman Technique](https://en.wikipedia.org/wiki/Feynman_technique): explain simply, use an analogy, expose gaps, then verify your understanding.
### When to use
| Situation | Use Feynman prompt |
| ---------------------------------------------------- | -------------------------------- |
| You read code but can't explain it in your own words | ✅ |
| Agent generated a diff you don't fully understand | ✅ Before commit |
| Spec uses jargon (RAG, CoT, BFF, …) | ✅ |
| You're stuck >30 min on the same concept | ✅ Then ask Dave if still blocked |
### Universal template (copy into Cursor / Kilo)
```
Feynman explain — [TOPIC or FILE PATH]:
1. Explain like I'm a new intern with CS basics but new to this codebase.
2. Use one everyday analogy (non-medical if possible).
3. Walk the data/control flow in ≤8 numbered steps.
4. Define every abbreviation you use (or link to our glossary).
5. List 3 "common mistakes" a beginner would make here.
6. Ask me 2 short quiz questions — wait for my answers before continuing.
7. If I get them wrong, correct me simply.
Context: VKIST MSK Pilot, ticket [P-n or Q-n].
Do NOT write code yet — teach only.
```
### After the agent teaches you
1. **Rewrite** the idea in 35 bullets in your own words (PR description, standup note, or personal log).
2. If you still can't rewrite without copying — **you're not ready to commit** that code; ask one follow-up or ping Dave.
3. Optional: paste your summary to Dave at weekly demo — proves understanding, not just output.
Role-specific Feynman prompts: [Phuc §12](./ONBOARDING_PHUC_ML_ENGINEER.md#12-bonus--feynman-technique-prompts) · [Quan §14](./ONBOARDING_QUAN_FRONTEND.md#14-bonus--feynman-technique-prompts)
**Tóm tắt (VI):** Khi khó hiểu — bảo agent giải thích đơn giản (Feynman), trả lời quiz, tự viết lại bằng lời mình; chưa hiểu thì chưa commit.
---
## 10. Collaboration contract
| Rule | Detail |
| ------------- | ----------------------------------------------------------- |
| Assignment | Dave picks backlog ID — you don't self-select large scope |
| PR size | ≤300 lines; one backlog ID per PR |
| Weekly demo | Runnable change required |
| Agentic tools | OK after reading ticket spec; pass `AGENT_SKILL` checklists |
| Research | ≤40% time; only when tied to P-7 or approved milestone |
| Blocked >4h | Ping Dave with exact error + file path |
**Tóm tắt (VI):** PR nhỏ, demo mỗi tuần, hỏi Dave khi kẹt quá 4 giờ.
---
## 11. Current assignments (Dave fills in)
| Intern | Active ticket | Checkpoint | PR link | Status |
| ------ | ------------------------- | ---------- | ------- | ------ |
| Phuc | P-___ | CP-___ | | |
| Quan | Q-1 (`/` worklist mobile) | CP-5 | | |
---
## 12. Glossary of abbreviations
Terms you will see daily in specs, code comments, and standups. Bookmark this section.
### Project & process
| Abbr. | Full form | Meaning in this project |
| --------------- | --------------------------------- | -------------------------------------------------------------- |
| **VKIST** | Viện Khoa học Công nghệ (context) | Research institute partner; MSK ultrasound AI origin |
| **MSK** | Musculoskeletal | Joint/tendon/ligament domain — knee ultrasound focus |
| **PoC** | Proof of Concept | Current sprint build — not production-certified yet |
| **CP-n** | Checkpoint n | Sprint deliverable milestone (see §4) |
| **P-n / Q-n** | Phuc-ticket / Quan-ticket | Backlog IDs in §5 |
| **UC-n** | Use Case n | Clinical workflow spec (e.g. `UC-48376` load scan session) |
| **FR-n** | Functional Requirement n | Feature requirement (e.g. **FR-25** synovitis grading) |
| **NFR-n** | Non-Functional Requirement n | Quality constraint (latency, safety, privacy) |
| **WS-25** | Workspace 25 | Clinician diagnostic workspace UI (65/35 layout) |
| **HITL** | Human-in-the-Loop | Clinician must review/confirm before AI output is final |
| **UP5, UP1, …** | User Persona n | Stakeholder archetype (UP5 = radiologist, UP1 = senior expert) |
### Clinical & domain (Vietnamese context)
| Abbr. | Full form | Meaning |
| --------------- | ----------------------------------------------------------- | --------------------------------------------------------- |
| **MOH** | Ministry of Health (Vietnam) | Source of clinical guidelines for RAG corpus |
| **DICOM** | Digital Imaging and Communications in Medicine | Standard medical image format (ultrasound slices) |
| **PACS** | Picture Archiving and Communication System | Hospital image archive — DICOM source |
| **EMR / HIS** | Electronic Medical Record / Hospital Information System | Hospital patient record system |
| **HL7 / FHIR** | Health Level 7 / Fast Healthcare Interoperability Resources | Standards for EMR data exchange |
| **Synovitis** | — | Joint lining inflammation — graded **03** by this system |
| **GradCAM** | Gradient-weighted Class Activation Mapping | Heatmap showing which image regions drove AI grade |
| **ULTS** | Ultrasound | Modality used in FR-25 |
| **Circular 46** | Thông tư 46/2018/TT-BYT | Vietnam clinical report format regulation |
| **Decree 13** | Nghị định 13/2023/NĐ-CP | Vietnam personal data protection law (PHI rules) |
### Security & privacy
| Abbr. | Full form | Meaning |
| --------------- | ----------------------------------- | ----------------------------------------------------------------- |
| **PHI** | Protected Health Information | Patient-identifiable health data — never commit to git |
| **PII** | Personally Identifiable Information | Names, IDs, phone — scrubbed before cloud egress |
| **RBAC** | Role-Based Access Control | Permissions by clinician role (radiologist, therapist, …) |
| **JWT** | JSON Web Token | Session auth token (stored in Redis server-side) |
| **NFR-16a** | NFR exception 16a | PoC-only rule allowing cloud LLM with redaction + consent + audit |
| **AES-256-GCM** | Advanced Encryption Standard | Client-side encryption for local storage (target arch) |
### Architecture & infrastructure
| Abbr. | Full form | Meaning |
| ------------ | ----------------------------------- | -------------------------------------------------------------- |
| **PWA** | Progressive Web App | Installable web app — Sprint 4 patient target |
| **BFF** | Backend-for-Frontend | FastAPI layer that proxies agent tools + cloud LLM for browser |
| **API** | Application Programming Interface | HTTP endpoints (`/api/v1/...`) |
| **K3s** | Lightweight Kubernetes | On-prem hospital cluster (target deployment) |
| **Triton** | NVIDIA Triton Inference Server | GPU server for CV models + embeddings (`:8001` gRPC) |
| **FastAPI** | — | Python web framework for backend (`:8000`) |
| **NGINX** | — | Reverse proxy / SSL termination in front of FastAPI |
| **MinIO** | — | S3-compatible object store for DICOM blobs, reports |
| **pgvector** | PostgreSQL vector extension | Vector DB for MOH guideline semantic search (HNSW index) |
| **HNSW** | Hierarchical Navigable Small World | Approximate nearest-neighbor index for RAG retrieval |
| **GCP** | Google Cloud Platform | Hosts Gemini (Vertex AI) |
| **Modal** | Modal.com | Serverless GPU platform hosting MedGemma |
| **LAN** | Local Area Network | Hospital internal network — air-gapped primary mode |
| **CI/CD** | Continuous Integration / Deployment | Jenkins + GitLab pipeline |
| **SSE** | Server-Sent Events | One-way streaming from server to browser |
### ML, NLP & generative AI
| Abbr. | Full form | Meaning |
| --------------------- | ------------------------------------------------------- | ------------------------------------------------------------------- |
| **CV** | Computer Vision | Image ML pipeline (angle → inflammation → segmentation) |
| **NLP** | Natural Language Processing | Clinical text cleanup, explanations, chat |
| **LLM** | Large Language Model | Generative text models (Gemma, Gemini, MedGemma) |
| **RAG** | Retrieval-Augmented Generation | Fetch MOH guideline chunks before generating answer |
| **CoT** | Chain-of-Thought | Model "thinking" trace before final answer (thought channel) |
| **TTFT** | Time to First Token | Latency until first streamed word appears (NFR-7: ≤200 ms target) |
| **E2B / E4B** | Gemma 4 2B / 4B params | Edge (browser) vs server Ollama model sizes |
| **WebLLM** | Web Large Language Model | In-browser inference via WebGPU + WASM |
| **OPFS** | Origin Private File System | Browser storage for ~1.9 GB Gemma checkpoint |
| **WASM** | WebAssembly | Binary format for in-browser ML (MediaPipe, LiteRT) |
| **WebGPU** | — | Browser GPU API — required for edge Gemma |
| **WebGL** | — | Older browser 3D API — fallback path for weak devices |
| **BERT** | Bidirectional Encoder Representations from Transformers | Classifier for drift/guardrail/referee (partially stubbed) |
| **EmbeddingGemma** | — | 768-dim embedding model for RAG vector search |
| **MedGemma** | — | Tier-3 clinical deep-reasoning LLM (Modal, NFR-16a) |
| **Gemini** | — | Tier-2 orchestration/translation LLM (GCP Vertex AI) |
| **Gemma** | — | Tier-1 edge conversation LLM (browser MediaPipe) |
| **Agent** | — | Multi-step tool-calling loop (`<tool_call>` / `<final>` protocol) |
| **Guardrail** | — | Safety filter on generated text (hallucination, scope breach) |
| **RAG-Referee** | — | 3-axis citation validator for MedGemma output |
| **Circuit breaker** | — | Clinician challenges AI grade via Socratic dialogue before sign-off |
| **Socratic dialogue** | — | Structured Q&A between clinician and AI about a grade |
| **DeepLab / UNet** | — | Segmentation model architectures in CV pipeline |
| **ConvNeXt / MedViT** | — | Angle classification model families (legacy stack) |
| **LiteRT** | — | On-device inference runtime (TensorFlow Lite successor) |
| **TFLite / TFJS** | TensorFlow Lite / TensorFlow.js | Client-side ML runtimes |
| **ONNX** | Open Neural Network Exchange | Model interchange format for Triton deployment |
| **VRAM** | Video RAM | GPU memory — relevant for Triton/Modal sizing |
### Frontend & UI
| Abbr. | Full form | Meaning |
| ------------- | ------------------------------------------ | ------------------------------------------------------------- |
| **UI / UIX** | User Interface / User Interface eXperience | Screens, layout, interaction design |
| **SPA** | Single Page Application | React app — one HTML shell, client-side routing |
| **TS / TSX** | TypeScript / TypeScript + JSX | Frontend language + React syntax |
| **Zustand** | — | Lightweight React state store (target arch; PoC uses context) |
| **Dexie.js** | — | IndexedDB wrapper for offline cache |
| **IndexedDB** | — | Browser local database for inference cache, session replay |
| **MRN** | Medical Record Number | Patient hospital ID shown in TopBar |
| **FR-26** | Functional Requirement 26 | Future patient-facing dashboard scope |
| **a11y** | Accessibility | WCAG-oriented design (contrast, font scale, touch targets) |
### Tools & workflow
| Abbr. | Full form | Meaning |
| ------------ | ---------------------------------- | ----------------------------------------- |
| **PR** | Pull Request | Code review unit — keep ≤300 lines |
| **IDE** | Integrated Development Environment | VS Code / Cursor |
| **Dev Mode** | Figma Dev Mode | Inspect panel for spacing, tokens, CSS |
| **SDK** | Software Development Kit | e.g. `@google/stitch-sdk` for asset fetch |
**Tóm tắt (VI):** Bảng viết tắt trên dùng hàng ngày — FR/NFR = yêu cầu; RAG/LLM/CoT = AI sinh ngôn ngữ; PHI/Decree 13 = bảo mật; PWA/DICOM/PACS = lâm sàng & UI; hỏi Dave nếu gặp từ mới.
---
## 13. Quick links
| Resource | Path |
| -------------------- | ------------------------------------------------------------------------------------- |
| Project vision | `PILOT_PROJECT/PROJ_LEVEL_READING/PLAN/CONTEXT_VISION_SCOPE.md` |
| Sprint architecture | `PILOT_PROJECT/workspace/sprint_1_2/Design_Material/SPRINT_1_2_ARCHITECTURE_SPEC.md` |
| UI truth doc | `PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/spec/uix_prototype_coverage.md` |
| Agent skills | `PILOT_PROJECT/AGENT_SKILL/INTRODUCTION.md` |
| Dave's Jul 6 handoff | `session_memory/6_Jul_26.md` |
| Layout reference | `session_memory/26_Jun_26/Proj_context_26_Jun_26.md` |

View File

@@ -0,0 +1,396 @@
# Onboarding — Phuc (ML Engineer / Agentic & Generative Logic)
**Start here:** [ONBOARDING_INDEX.md](./ONBOARDING_INDEX.md) — [glossary](./ONBOARDING_INDEX.md#12-glossary-of-abbreviations) · [Feynman prompts (INDEX §9)](./ONBOARDING_INDEX.md#9-bonus--feynman-technique-with-agents)
**Date:** 7 Jul 2026
**Timebox:** ~Jul 7 Aug 31, 2026
**Lane:** Generative LLM, agent runtime, guardrails, RAG gates — co-designed with Dave
---
## 1. Your mission
You continue the **generative & agentic layer** Dave built. Your deliverable is **merged code** that advances checkpoints **CP-2, CP-3, CP-8** — not architecture slides.
You co-own agentic design and guardrail logic with Dave: you propose, Dave approves scope, you ship one vertical slice per PR.
**You are not expected to:** rewrite the CV/Triton pipeline, own full compliance sign-off, or refactor UI components.
**Tóm tắt (VI):** Tiếp tục xây LLM/agent/guardrail; mỗi tuần có PR chạy được; thiết kế guardrail cùng Dave; không làm lại pipeline những kết quả đã có - nếu có thì phải optimize & hạn chế redundancies overhead trong thiết kế.
---
## 2. Your codebase scope
### In scope — touch freely
| Path | What |
| --------------------------------------------------------------------------- | --------------------------------------- |
| `CODEBASE/ml/implementation/nlp/agent_runtime/` | Agent loop, tools, prompts |
| `CODEBASE/ml/tests/gemma4_e2b/` | Isolated Gemma + agent lab (`:5174`) |
| `CODEBASE/ml/tests/agent_tools/` | BFF / Modal / Exa smoke tests |
| `CODEBASE/ml/spec/conformal_decoding_llm_with_domain_stritcly_constrain.md` | Research spec |
| `CODEBASE/frontend/implementation/src/lib/llm/` | Edge inference, chat routing, prompts |
| `CODEBASE/frontend/implementation/src/hooks/useClinicalChat.ts` | Chat bootstrap + streaming |
| `CODEBASE/frontend/implementation/src/workers/llm.worker.ts` | MediaPipe Gemma worker |
| `CODEBASE/frontend/implementation/src/workers/` | Create `guardrail.worker.ts` here (P-3) |
| `CODEBASE/backend/routers/cloud_*.py`, `agent_tools.py` | Cloud + tool BFF |
| `CODEBASE/backend/services/cloud_llm_gateway.py`, `agent_tools_service.py` | Gateway + tools |
| `CODEBASE/backend/implementation/safety/` | Safety service |
| `CODEBASE/backend/implementation/adapters/bert_adapter.py` | Guardrail stubs |
| `CODEBASE/knowledge/spec/` | When working P-4 / P-8 |
### Read-only — understand, don't refactor
- `CODEBASE/backend/implementation/pipeline/` — CV inference (Dave)
- `CODEBASE/infra/` — unless P-6 Modal debugging
- `CODEBASE/frontend/implementation/src/components/` — Quan's lane
- `workspace/LEGACY/` — reference only
### Architecture mental model
```
User message
→ useClinicalChat / runClinicalChatTurn
→ [chat|planning] → Edge Gemma (llm.worker) OR Ollama E4B
→ [agent] → agentLoop → tools (exa, supabase, medgemma)
→ [YOUR GAPS] guardrail check, RAG gate, cloud escalation audit
→ display (StreamingPlainText)
```
Tier 1 = browser Gemma · Tier 2 = Gemini · Tier 3 = MedGemma (NFR-16a governed)
**Tóm tắt (VI):** Phạm vi = ml/agent_runtime, lib/llm, backend safety/routers; không sửa UI hay LEGACY.
---
## 3. What to read, what to learn
### Must read (Day 1 — before first PR)
| # | Document | Path |
| --- | ---------------------- | ---------------------------------------------------------------------------- |
| 1 | Dave's latest handoff | `session_memory/6_Jul_26.md` |
| 2 | Regulatory constraints | `PILOT_PROJECT/PROJ_LEVEL_READING/PLAN/CONTEXT_VISION_SCOPE.md` §4 |
| 3 | Agent protocol | `CODEBASE/ml/implementation/nlp/README.md` |
| 4 | PHI safety | `PILOT_PROJECT/AGENT_SKILL/secrets_and_phi_safety/SKILL.md` |
| 5 | Agent tools rules | `PILOT_PROJECT/AGENT_SKILL/agent_tool_execution/SKILL.md` |
| 6 | LLM + safety NFRs | `Design_Material/SPRINT_1_2_ARCHITECTURE_SPEC.md` — LLM tiers, NFR-10/16a/18 |
### Should read (when ticket assigned)
| Ticket | Read |
| -------- | --------------------------------------------------------------------------------------------------------------- |
| P-1, P-6 | `backend/spec/agent_tools_contract.md`, `agent_runtime/.../tools/registry.ts`, `backend/routers/agent_tools.py` |
| P-2, P-3 | `bert_adapter.py`, `safety/service.py`, `FR_25_UC_DESIGN/Q1_*.md``Q4_*.md` |
| P-4, P-8 | `knowledge/spec/knowledge_spec.md`, `agentLoop.ts`, `runClinicalChatTurn.ts` |
| P-5 | `clinicalChatModes.ts`, `chatReasoningLevel.ts`, `6_Jul_26.md` §beta |
| P-7 | `ml/spec/conformal_decoding_llm_with_domain_stritcly_constrain.md` |
| Any | Relevant `UC-*.md` in `Design_Material/FR_25_UC_DESIGN/Use_Case/` |
**Week 1 code trace (do once):**
`useClinicalChat.ts``runClinicalChatTurn.ts``agentLoop.ts` OR `llm.worker.ts``cloud_llm_gateway.py`
### Learn deeper (keywords)
> Abbreviations: [Glossary INDEX §12](./ONBOARDING_INDEX.md#12-glossary-of-abbreviations)
Study as tickets require — use papers, specs, and agent exploration:
| Keywords | Tickets |
| ------------------------------------------------------------------- | -------- |
| `prompt-constrained tool calling`, `agent orchestration loop` | P-1 |
| `RAG pre-retrieval`, `pgvector`, `semantic chunk`, `MOH corpus` | P-4, P-8 |
| `BERT guardrail`, `hallucination detection`, `scope-breach scoring` | P-2, P-3 |
| `conformal decoding`, `domain-constrained generation` | P-7 |
| `CoT streaming`, `thought/answer channel`, `WebLLM MediaPipe` | P-5 |
| `NFR-16a`, `consent audit`, `cloud egress redaction` | P-6 |
| `RAG-Referee`, `citation validation`, `3-axis contestant` | P-8 |
| `Transformers.js`, `Web Worker`, `OPFS model cache` | P-3 |
**Tóm tắt (VI):** Đọc 6 tài liệu bắt buộc ngày 1; đọc thêm theo ticket P-n; học sâu theo từ khóa khi làm.
---
## 4. Dev toolkit
| Tool | Command |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Clinical chat UI | `cd CODEBASE/frontend/implementation && npm run dev``:5173` |
| Gemma lab | `cd CODEBASE/ml/tests/gemma4_e2b && npm run dev``:5174` |
| Agent smoke (test-the-agentic-functionality) | `cd CODEBASE/ml/tests/agent_tools && npm run smoke` |
| Backend smoke (test-the-agentic-functionality) | `conda activate vkist_ultra`; `cd CODEBASE`; `PYTHONPATH=. python backend/tests/smoke_agent_tools_server.py` |
| Agentic IDE | Cursor / Kilo — load `AGENT_SKILL/agent_tool_execution` |
**Environment variables (frontend** `.env.development`**):**
```env
VITE_CLINICAL_CHAT_USE_LLM=true
VITE_CLINICAL_CHAT_MOCK_TOOLS=true # flip false for P-1 done
VITE_OLLAMA_CHAT_URL=/api/ollama-chat/api/chat
VITE_API_BASE_URL=http://localhost:8001
```
Secrets: ask Dave — `PILOT_PROJECT/secrets_template/README.md`
**Tóm tắt (VI):** Chạy :5173 chat, :5174 lab, smoke tests; đổi MOCK_TOOLS=false khi P-1 xong.
---
## 5. Day 1 playbook (then ship)
**Morning**
1. Run `:5173` → open `/workspace/:patientId` → test chat + agent modes.
2. Run `:5174` gemma4_e2b → complete one tool smoke call.
3. Trace one chat turn and one agent turn (Section 3 code trace).
**Afternoon**
4. Hands-on verify backlog P-1P-7 (✅/❌ in your notes).
5. Sync with Dave → receive first ticket (likely **P-1** or **P-5**).
Do **not** spend Week 1 writing standalone architecture docs.
**Tóm tắt (VI):** Ngày 1: chạy app, trace code, sync Dave lấy ticket — không viết doc dài.
---
## 6. Spec vs implementation (your work surface)
| Component | Built today | Your tickets |
| ----------------------------------- | -------------- | ------------------ |
| Edge Gemma inference | ✅ | P-5 polish |
| Agent loop + `<tool_call>` protocol | ✅ (mock tools) | P-1 |
| Cloud gateway + audit hooks | ⚠️ partial | P-6 |
| Auto-retrieval in agent mode | ✅ | P-4 extend to chat |
| `bert_adapter.py` / safety service | ❌ stub | P-2 |
| `guardrail.worker.ts` | ❌ missing | P-3 |
| RAG-Referee 3-axis | ❌ spec only | P-8 |
| Planning + high reasoning beta | ⚠️ disabled | P-5 |
**Key facts:**
- Production chat = **browser Gemma 4 E2B** (MediaPipe), not server-side generation.
- Agent mode uses **prompt tags** (`<tool_call>`, `<final>`), not native function calling.
- Do **not** re-enable `ChatMarkdown` in reasoning mode — see `6_Jul_26.md` crash policy.
**Tóm tắt (VI):** Guardrail và RAG chat còn stub — đó là trọng tâm của bạn.
---
## 7. Your backlog (detail)
| ID | Gap | Spec | Starting files |
| ------- | ------------------------------ | --------------------------------- | ----------------------------------------------- |
| **P-1** | Mock agent tools | `agent_tools_contract.md` | `tools/registry.ts`, `agent_tools.py` |
| **P-2** | BERT adapter stubs | NFR-10, Q1Q4 | `bert_adapter.py`, `safety/service.py` |
| **P-3** | No edge guardrail worker | Web Workers spec | Create `guardrail.worker.ts` |
| **P-4** | No RAG gate in chat mode | NFR-18, `knowledge_spec.md` | `runClinicalChatTurn.ts` |
| **P-5** | Beta modes disabled | `6_Jul_26.md` | `clinicalChatModes.ts`, `chatReasoningLevel.ts` |
| **P-6** | MedGemma escalation incomplete | NFR-16a | `cloud_llm_gateway.py`, `tools/registry.ts` |
| **P-7** | Conformal decoding research | `ml/spec/conformal_decoding_*.md` | `gemma4_e2b/` lab |
**Tóm tắt (VI):** P-1 đến P-7 — chờ Dave giao ID / chủ động đề xuất giải pháp kỹ thuật & thiết kế; mỗi PR một ticket.
---
## 8. SMART milestones
### M1 — First merge (Jul 7 Jul 18)
| | |
| -------------- | --------------------------------------------- |
| **Ticket** | P-1 or P-5 (Dave assigns) |
| **Specific** | Live agent tools OR remove beta from one mode |
| **Measurable** | PR merged; live demo on `:5173` |
| **Achievable** | One backlog ID only |
| **Relevant** | CP-1 / CP-2 |
| **Time-bound** | Sprint 3 week 1 |
### M2 — Guardrail v1 (Jul 14 Aug 1)
| | |
| -------------- | ------------------------------------------------------- |
| **Ticket** | P-2 or P-3 (co-scoped with Dave) |
| **Specific** | RFC ≤1 page + one real PASS/FAIL check on outbound text |
| **Measurable** | Test/smoke on 3 fixtures from Q1Q4 scenarios |
| **Achievable** | Keyword gate OK for v1; full BERT optional |
| **Relevant** | CP-3, NFR-10 |
| **Time-bound** | Sprint 3 end |
### M3 — Retrieval or escalation (Jul 28 Aug 15)
| | |
| -------------- | -------------------------------------------------- |
| **Ticket** | P-4 or P-6 |
| **Specific** | RAG gate in chat OR MedGemma escalation with audit |
| **Measurable** | Screen recording + log snippet in PR |
| **Relevant** | CP-2 / CP-8, NFR-18 |
| **Time-bound** | Sprint 4 |
### M4 — Research or hardening (Aug 1 Aug 31)
| | |
| -------------- | ---------------------------------------------------- |
| **Ticket** | P-7 or extend P-2/P-3 |
| **Specific** | Conformal eval (3 VI cases) OR second guardrail axis |
| **Optional** | If M2M3 complete |
| **Time-bound** | Internship end |
**Tóm tắt (VI):** 4 mốc — PR đầu tiên tuần 1; guardrail tuần 34; RAG/escalation tháng 8.
---
## 9. Anti-patterns
- ❌ Refactor `agent_runtime` package structure without inform to members on update logic (no change-Log)
- ❌ Swap model families (Qwen experiment is closed)
- ❌ Touch `LEGACY/` or Quan's UI components
- ❌ Enable `ChatMarkdown` during reasoning streams (markdown rendering on LLM's outcome is problem - we shall put it under the rug)
- ❌ Commit secrets or PHI — ever
**Tóm tắt (VI):** Không refactor lớn, không đổi model, không markdown khi reasoning.
---
## 10. Definition of done
- [ ] ≥3 merged PRs across P-1P-6
- [ ] Can demo live agent tool call or guardrail check without Dave driving
- [ ] CP-2 or CP-3 marked advanced in team backlog
- [ ] Read `secrets_and_phi_safety` + `agent_tool_execution` skills
**Tóm tắt (VI):** 3 PR, demo được guardrail/agent, checkpoint CP-2 hoặc CP-3 tiến triển.
---
## 11. Agent prompt examples (optimize / change based on usecase)
**Explore generative flow:**
```
Trace runClinicalChatTurn.ts for agent mode: list each file called,
what tool protocol is used, and where mock tools are injected.
```
**Start P-1:**
```
Help me flip VITE_CLINICAL_CHAT_MOCK_TOOLS to false and fix any
BFF errors. Start from tools/registry.ts and agent_tools.py.
Follow AGENT_SKILL/agent_tool_execution/SKILL.md.
```
**Start P-2:**
```
Implement a minimal scope-breach keyword gate in bert_adapter.py
returning PASS/FAIL. Use Q1_True_Agreement.md as fixture context.
```
---
## 12. Bonus — Feynman Technique prompts
Use when a generative/agent concept feels opaque **before** you commit agent-generated code.
Master template: [INDEX §9](./ONBOARDING_INDEX.md#9-bonus--feynman-technique-with-agents).
### Hard concepts → ready-made prompts
**Agent loop & `<tool_call>` protocol (P-1):**
```
Feynman explain agentLoop.ts and the <tool_call> / <final> protocol in
@vkist/agent-runtime. Why prompt tags instead of OpenAI function calling?
Trace one turn with exa_search mocked. Quiz me after.
```
**Edge vs cloud 3-tier LLM (general):**
```
Feynman explain our 3-tier LLM: Edge Gemma → Gemini → MedGemma.
When does each tier run? What is NFR-16a in one sentence?
Use a "triage desk at a hospital" analogy. Quiz me.
```
**clinicalChatStreamRegistry + streaming (P-5 area):**
```
Feynman explain why clinicalChatStreamRegistry.ts paints tokens to the DOM
directly instead of React setState on every token. What breaks if we use
ChatMarkdown during CoT? ≤8 steps. Quiz me.
```
**RAG pre-retrieval (P-4 / P-8):**
```
Feynman explain mandatory RAG pre-generation in our system: what pgvector
retrieves, when agentLoop auto-calls supabase_query, and why chat mode
doesn't do this yet. MOH corpus in plain language. Quiz me.
```
**Guardrail vs RAG-Referee (P-2 / P-3):**
```
Feynman explain the difference between edge guardrail, bert_adapter
drift check, and RAG-Referee. Which are implemented vs stub today?
Use a "editor vs fact-checker" analogy. Quiz me.
```
**After each Feynman session:** write 3 bullets in your own words in the PR or ticket comment.
**Tóm tắt (VI):** Dùng prompt Feynman khi không hiểu agent/LLM; có sẵn prompt cho từng chủ đề khó; tự tóm tắt lại trước khi commit.

View File

@@ -0,0 +1,499 @@
# Onboarding — Quan (Frontend / Mobile PWA)
**Start here:** [ONBOARDING_INDEX.md](./ONBOARDING_INDEX.md) — [glossary](./ONBOARDING_INDEX.md#12-glossary-of-abbreviations) · [Feynman prompts (INDEX §9)](./ONBOARDING_INDEX.md#9-bonus--feynman-technique-with-agents)
**Date:** 7 Jul 2026
**Timebox:** ~2 months (ends ~early Aug 2026)
**Lane:** Mobile UI, Figma + Stitch design acceleration, patient PWA
---
## 1. Your mission
You continue the **UI layer** from stub routes toward Sprint 4 patient PWA checkpoints. You ship **visible screens in the browser** — not backend, not LLM internals.
Dave provides two design accelerators:
| Tool | Link | Role |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| **Figma** | [MSK_Workspace_UIX-Design](https://www.figma.com/design/uakk3T4efS1AIckfTEFSZO/MSK_Workspace_UIX-Design?node-id=0-1) | Design authority — 35 screens, tokens, Vietnamese copy |
| **Google Stitch** | [Whiteboard](https://stitch.withgoogle.com/projects/8188910238055386921) | Acceleration — board has **web mocks**; you design **mobile variants** |
**Your agency:** Pick Figma-first, Stitch-first, or hybrid **per screen** — one line in the PR is enough. Dave assigns tickets; **Week 1 default ticket is Q-1 on the live `/` worklist** (highest yield).
**Week 1 principle — ship fast, correct, get feedback:**
| Priority | Rule |
| --------------- | ----------------------------------------------------------------------------------------------------- |
| **Speed** | Tools (Figma + Stitch) = **1 day max**, then code same day |
| **Yield** | Week 1 targets the **live route** `/` (`PatientWorklistPage`) mobile at 375px — not orphan stub pages |
| **Correctness** | `design-tokens.ts`, live navigation, no hardcoded colors, no pasted Stitch HTML |
| **Feedback** | Draft PR or screenshots to Dave **end of Day 1**; iterate Day 23 from review |
**Stitch (quick rules):**
1. Board has web mocks — use Stitch to sketch **mobile layout**, export screenshot for Cursor/Kilo
2. Agent writes React; you review every line
3. Figma wins on tokens and Vietnamese copy
**Tóm tắt (VI):** Tuần 1 = 1 ngày học tool, cùng ngày code mobile cho `/` worklist; PR/screenshot cuối ngày 1; sửa theo feedback Dave — không khảo sát tự do nhiều ngày.
---
## 2. Your codebase scope
### In scope — touch freely
```
CODEBASE/frontend/
├── spec/ ← read: uix_prototype_coverage, figma-component-specs, DESIGN
└── implementation/src/
├── pages/ ← your primary workspace
├── components/atoms/
├── components/molecules/
├── components/layout/
├── styles/design-tokens.ts ← token source of truth
├── App.tsx ← routes
└── public/assets/stitch/ ← exported Stitch references
```
### Read-only — understand, don't refactor
| File / folder | Why |
| ------------------------------------------ | -------------------------------- |
| `organisms/DiagnosticCanvas.tsx` | Dave owns DICOM canvas |
| `molecules/ClinicalChatPanel.tsx` | Read layout only — Phuc owns LLM |
| `lib/llm/`, `workers/` | Phuc's lane |
| `hooks/useClinicalChat.ts` | Know it exists |
| `data/mockData.ts`, `lib/patientStore.tsx` | How worklist gets data |
### Out of scope
`CODEBASE/backend/`, `CODEBASE/ml/`, `CODEBASE/knowledge/`, `CODEBASE/infra/`, `LEGACY/`
### Live routes today
```
/ → PatientWorklistPage.tsx ← Q-1, Q-6 priority
/workspace/:patientId → ClinicalWorkspacePage.tsx ← polish only; Q-2 overlay
+ 35 stub routes in src/pages/ ← Figma parity over time
```
**Tóm tắt (VI):** Sửa pages, atoms, molecules, tokens; không đụng canvas, chat LLM, backend.
---
## 3. What to read, what to learn
### Must read (Day 1 morning only — then code)
| # | Document | Time |
| --- | ----------------------------------------------------------------- | ---------------------------------- |
| 1 | `uix_prototype_coverage.md` | 20 min — **which routes are live** |
| 2 | `figma-component-specs.md` — Priority 12 + node for `/` worklist | 15 min |
| 3 | `design-tokens.ts` + skim `DESIGN.md` | 15 min |
| 4 | `AGENT_SKILL/secrets_and_phi_safety/SKILL.md` | 10 min |
Read `component-inventory.md`, `coding_convention`, and persona §3.4 **while coding**, not before Day 1 PR.
**Tóm tắt (VI):** Sáng ngày 1 chỉ đọc 4 tài liệu ngắn; chiều cùng ngày bắt đầu code.
### Should read (when ticket assigned)
| Ticket | Read |
| ---------- | ---------------------------------------------------------------------------------------- |
| Q-1, Q-6 | Figma node for screen; matching `src/pages/*.tsx` stub |
| Q-2 | `uix_prototype_coverage.md` §Safety; `SafetyEscalationOverlay.tsx` |
| Q-3, Q-4 | `CONTEXT_VISION_SCOPE.md` §3.4; `frontend_spec.md` PoC § |
| Q-5 | `figma-component-specs.md` — error boundaries item |
| Q-8 | [Stitch board](https://stitch.withgoogle.com/projects/8188910238055386921) + live routes |
| Any screen | `Design_Material/FR_25_UC_DESIGN/Use_Case/UC-*.md` |
| Layout | `session_memory/26_Jun_26/Proj_context_26_Jun_26.md` — 65/35 ASCII |
### Learn deeper (keywords)
> Abbreviations: [Glossary INDEX §12](./ONBOARDING_INDEX.md#12-glossary-of-abbreviations)
| Keywords | When |
| ---------------------------------------------------------- | ------------------- |
| `atomic design`, `atoms molecules organisms` | component-inventory |
| `React functional components`, `props`, `composition` | all Q tickets |
| `design tokens`, `CSS variables`, `responsive breakpoints` | Q-1, Q-3 |
| `mobile-first`, `375px viewport`, `touch target 44px` | Q-3, Q-6 |
| `accessibility`, `WCAG contrast`, `font scaling` | Q-4, Q-6 |
| `error boundaries` | Q-5 |
| `PWA`, `service worker` | Sprint 4 (later) |
| `Figma Dev Mode`, `Inspect` | Q-1 |
| `Stitch design-to-code`, `HTML export as reference` | Q-8 |
| `Vite`, `npm run dev` | daily |
| `Vietnamese clinical UX`, `jargon-free patient copy` | Q-6 |
**Tóm tắt (VI):** Đọc thêm theo ticket khi code; không ôm 7 tài liệu trước PR đầu.
---
## 4. Dev toolkit
| Category | Tool |
| ------------------- | ------------------------------------------------------------------------------------------------- |
| Dev server | `npm run dev``http://localhost:5173` |
| Design authority | [Figma](https://www.figma.com/design/uakk3T4efS1AIckfTEFSZO/MSK_Workspace_UIX-Design?node-id=0-1) |
| Design acceleration | [Stitch whiteboard](https://stitch.withgoogle.com/projects/8188910238055386921) |
| Coding agent | Cursor / Kilo |
| Mobile test | Chrome DevTools → iPhone SE 375×667 |
| PR evidence | Before/after screenshots desktop + 375px |
```bash
cd PILOT_PROJECT/workspace/sprint_1_2/CODEBASE/frontend/implementation
npm install
npm run dev
```
**Optional:** Stitch SDK — `public/assets/stitch/README.md` (OAuth via Dave)
### Agent prompt template
```
Implement [screen name] for mobile (375px).
Reference: Figma node [id] from figma-component-specs.md.
Layout hint: [Stitch mobile export screenshot or description].
Use existing atoms from component-inventory.md.
Use design-tokens.ts — no hardcoded colors.
Route: [path from figma-component-specs].
Match navigation in uix_prototype_coverage.md.
```
**Tóm tắt (VI):** npm run dev + Figma + Stitch + Cursor; test 375px; PR kèm screenshot.
---
## 5. Figma + Stitch workflow
### When to use which
| Source | Best for | Caveat |
| ------------- | --------------------------------------------- | --------------------------------------- |
| **Figma** | Tokens, spacing, VI copy, auth/patient cards | Slower from scratch — use Inspect |
| **Stitch** | Rapid mobile layout; agent reference material | Web mock ≠ mobile — create mobile frame |
| **Live code** | Navigation truth | `uix_prototype_coverage.md` |
**Legacy Stitch desktop** (`public/assets/stitch/README.md`, project `17628337625105266704`) — reference only.
**Primary Stitch:** [project 8188910238055386921](https://stitch.withgoogle.com/projects/8188910238055386921)
### Workflow choices (pick one per screen — state in PR)
| ID | When | Steps |
| -------------------- | -------------------------------------------------- | --------------------------------------------- |
| **A — Figma-first** | Screen in node map; tokens matter | Inspect → atoms → `src/pages/` |
| **B — Stitch-first** | Mobile patient screen; Stitch mobile variant ready | Stitch mobile design → export → agent → React |
| **C — Hybrid** | Stitch layout close; Figma has correct copy | Stitch structure + Figma polish → code |
**Rule:** Final PR must match live navigation (`/``/workspace/:id`), not a disconnected mock.
**Tóm tắt (VI):** Chọn A/B/C mỗi màn hình; Figma thắng khi khác Stitch; khớp route thật.
---
## 6. Week 1 sprint — tools in 1 day, ship mobile same week
> **No multi-day exploration.** Figma/Stitch are means to an end — the end is a **merged mobile PR** on a live screen.
### Day 1 — Tools (morning) + code (afternoon)
| Block | Time | Do |
| ------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Setup** | 30 min | `npm install && npm run dev` — open `/` on desktop + DevTools 375px |
| **Figma** | 45 min | Open [Figma](https://www.figma.com/design/uakk3T4efS1AIckfTEFSZO/MSK_Workspace_UIX-Design?node-id=0-1) → Inspect **Patient List** node `82-3155` (maps to live worklist) |
| **Stitch** | 45 min | Open [Stitch board](https://stitch.withgoogle.com/projects/8188910238055386921) → one mobile-frame sketch OR pick closest web mock → **screenshot for agent** |
| **Must-read** | 60 min | Section 3 four docs only |
| **Code** | Rest of day | **Q-1 on live `/`** — mobile layout for `PatientWorklistPage.tsx` (or `patient-list-main.tsx` if Dave assigns stub route). Cursor/Kilo + agent template (§4). **375px from the start.** |
| **EOD** | 15 min | Send Dave: draft PR **or** before/after screenshots at 375px + 1-line workflow choice (A/B/C) |
**Not acceptable as Week 1 deliverable:** gap lists, alignment essays, clicking all 35 routes, Storybook setup.
### Day 2 — Feedback loop
- Apply Dave's review on spacing, tokens, navigation, touch targets
- Push PR updates; aim for **merge-ready** by EOD if scope was ≤200 lines
### Day 35 — Second yield
- Merge Week 1 PR OR complete merge after small fixes
- Start **second screen** (patient card stub or Q-3 polish on same `/` route)
- Optional: update one Stitch frame to match what you shipped (Q-8 lite — no separate doc)
### Week 1 success criteria
| Check | Pass |
| ------------------------------ | ---------------------------------------------- |
| Live route `/` usable at 375px | No horizontal scroll |
| Uses `design-tokens.ts` | No raw hex in new code |
| Dave reviewed once | By end of Day 1 or 2 AM |
| Runnable demo | `npm run dev` — you walk Dave through tap flow |
**Tóm tắt (VI):** Ngày 1: tool buổi sáng, code buổi chiều, gửi Dave tối ngày 1; ngày 2 sửa feedback; ngày 35 merge + màn hình thứ 2.
---
## 7. What exists vs what you build
| Exists | You increment |
| ---------------------------------- | -------------------------------------------- |
| WS-25 workspace PoC (functional) | Mobile polish, not rewrite |
| 35 Figma stub routes | Q-1: 2 priority screens |
| Stitch web mocks | Q-8 lite: update frame after ship (optional) |
| `SafetyEscalationOverlay` (orphan) | Q-2: wire it |
| No error boundaries | Q-5 |
| No patient PWA | Q-6 |
### WS-25 layout (read-only context)
Clinician workspace is 65/35 split — your zone is **patient list, cards, patient dashboard**, not DICOM canvas:
```
┌─ TopBar ─────────────────────────────────────┐
├─ Zone A (65%) DICOM │ Zone B (35%) sidebar ┤ ← Dave/Phuc zone
└────────────────────────────────────────────────┘
Your zone: / worklist, /patients, patient-card-*, /home
```
See `session_memory/26_Jun_26/Proj_context_26_Jun_26.md` for full ASCII.
**Tóm tắt (VI):** Workspace clinician đã có; bạn làm patient list, card, dashboard mobile.
---
## 8. Your backlog (detail)
| ID | Gap | Priority screens / files |
| ------- | ----------------------- | --------------------------------------------------------------------------------------- |
| **Q-1** | Stub → Figma fidelity | Home, Login, Patient List, Patient Cards (see `figma-component-specs.md` §Priorities) |
| **Q-2** | Safety overlay unrouted | `SafetyEscalationOverlay.tsx`, `App.tsx` |
| **Q-3** | No 375px pass | Assigned patient screen |
| **Q-4** | Accessibility | Patient-facing view |
| **Q-5** | Error boundaries | `App.tsx` workspace route |
| **Q-6** | Patient dashboard | `patient-card-*.tsx`, FR-26 preview |
| **Q-7** | Zero-GPU UI banner | Dave provides detection hook |
| **Q-8** | Stitch sync (lite) | Update Stitch frame **after** you ship — optional in PR, not a standalone proposal gate |
**Figma priorities (from spec):**
1. Home, Login, Create Account
2. Patient List + Cards
3. Workspace shell (read-only for you)
4. Auth flows
5. AI View, Chat, Ultrasound
6. Remaining screens
**Tóm tắt (VI):** Ưu tiên Home, Patient List, Patient Card trước.
---
## 9. SMART milestones
### M1 — Week 1 ship (Jul 7 Jul 11) ← **start here**
| | |
| -------------- | ------------------------------------------------------------------------------------------------------- |
| **Ticket** | **Q-1 on live `/` worklist** (Dave confirms file: `PatientWorklistPage.tsx` or `patient-list-main.tsx`) |
| **Specific** | Mobile implementation at 375px on a **live** route — not a disconnected stub |
| **Measurable** | Draft PR Day 1 EOD; merged or merge-ready by Day 5; before/after screenshots |
| **Achievable** | ≤200 lines; tools only Day 1 morning |
| **Relevant** | CP-5 — fastest visible yield |
| **Feedback** | Dave review ≤24h after first push |
### M2 — Second screen (Jul 14 Jul 25)
| | |
| --------------- | ------------------------------------------------------------------- |
| **Ticket** | Q-1 #2 (patient card / home) or Q-3 deepen mobile on shipped screen |
| **Deliverable** | 2nd merged PR |
| **Relevant** | CP-5 / CP-6 |
### M3 — Mobile hardening (Jul 28 Aug 11)
| | |
| --------------- | -------------------------------------------------------------- |
| **Ticket** | Q-3 or Q-2 |
| **Deliverable** | 375px recording; taps ≥44px; optional Q-8 Stitch frame updated |
| **Relevant** | CP-6 |
### M4 — Accessibility (Aug 4 Aug 21) *optional*
| | |
| --------------- | ----------------------------------------- |
| **Ticket** | Q-4 |
| **Deliverable** | High contrast, large type on patient view |
| **Relevant** | CP-7 |
**Tóm tắt (VI):** M1 = tuần 1 merge mobile `/` worklist; không có M0 khảo sát riêng; M2M4 mở rộng sau khi có feedback.
---
## 10. Per-ticket workflow
```
1. Dave assigns Q-n
2. Pick workflow A / B / C — write in PR description
3. Capture Figma + Stitch reference screenshots
4. Cursor/Kilo: use agent prompt template (Section 4)
5. Review every line of the diff — understand before commit
6. npm run dev → screenshot desktop + 375px
7. PR: Q-n, CP-n, workflow choice, design links
```
**Tóm tắt (VI):** Mỗi ticket: chọn workflow → agent hỗ trợ → review diff → screenshot → PR.
---
## 11. Boundaries
**Touch:** `src/pages/`, `src/components/` (atoms/molecules/layout), `src/styles/`, `App.tsx`, Stitch whiteboard, `public/assets/stitch/`
**Do not touch:** `src/lib/llm/`, `src/workers/`, backend, ml
**Figma/Stitch edits:** OK to update Stitch frames and Figma comments; ask Dave before changing shared Figma components.
**Tóm tắt (VI):** Chỉ frontend UI; hỏi Dave trước khi sửa component Figma dùng chung.
---
## 12. Definition of done
- [ ] Week 1: live `/` worklist mobile at 375px shipped or merge-ready
- [ ] Dave feedback incorporated within 48h of first push
- [ ] ≥2 screens merged by end of Month 1 (M1 + M2)
- [ ] Each PR: workflow A/B/C (one line) + 375px screenshot
- [ ] Never committed `secrets/` or PHI
**Tóm tắt (VI):** Tuần 1 phải có mobile `/` chạy được + feedback Dave; không cần báo cáo alignment dài.
---
## 13. Agent prompt examples (optimize / change based on usecase)
**Learn the codebase:**
```
Explain PatientWorklistPage.tsx in simple terms: what components it uses,
where data comes from, and which Figma node it maps to.
```
**Start Q-1 (Figma-first):**
```
Implement /home per Figma node 33-143 in figma-component-specs.md.
Reuse Button and Badge from component-inventory. Use design-tokens.ts only.
Current stub: src/pages/home.tsx
```
**Week 1 — ship live worklist mobile (Day 1 afternoon):**
```
Mobile pass for PatientWorklistPage at 375px.
Figma node 82-3155. Stitch screenshot attached.
Use design-tokens.ts and existing Button, Badge, PatientCard from component-inventory.
No horizontal scroll. Touch targets >= 44px.
Do not change routing — stay on /.
Show me the minimal diff.
```
**Start Q-3 (mobile pass):**
```
Make patient-list-main.tsx usable at 375px: no horizontal scroll,
touch targets at least 44px. Show me which CSS to change.
```
---
## 14. Bonus — Feynman Technique prompts
Use when React layout, routing, or design workflow feels confusing **before** you commit.
Master template: [INDEX §9](./ONBOARDING_INDEX.md#9-bonus--feynman-technique-with-agents).
### Hard concepts → ready-made prompts
**Live route vs stub route (Week 1 critical):**
```
Feynman explain the difference between our LIVE routes (/ and /workspace/:id)
and the 35 Figma STUB routes in src/pages/. Why does Week 1 target
PatientWorklistPage on / ? Use a "demo kitchen vs blueprint room" analogy.
Quiz me.
```
**PatientWorklistPage data flow:**
```
Feynman explain PatientWorklistPage.tsx: where patient list data comes from
(mockData vs patientStore), what happens when I tap "Mở phiên chẩn đoán",
and which components re-render. ≤8 steps. No jargon without definition. Quiz me.
```
**design-tokens.ts vs hardcoded CSS:**
```
Feynman explain why we use design-tokens.ts instead of #hex in components.
Show one wrong example and one right example from our codebase.
Quiz me on how to add a new spacing value correctly.
```
**375px mobile layout:**
```
Feynman explain mobile-first pass at 375px for our worklist: flex vs grid,
overflow-x, and 44px touch targets. What is the smallest change to
PatientListShell or PatientWorklistPage to stop horizontal scroll?
Teach only — no code until I answer your quiz.
```
**Figma vs Stitch vs code (your workflow A/B/C):**
```
Feynman explain when I should use Figma-first vs Stitch-first vs hybrid
for a patient screen. What is "reference only" Stitch export vs production code?
Quiz me with one scenario: patient card on mobile.
```
**React atoms vs molecules (component-inventory):**
```
Feynman explain atomic design using OUR component-inventory.md: when is
something a Button (atom) vs PatientHeader (molecule)? Give 2 examples
from patient list screen. Quiz me.
```
### Feynman before commit rule
If the agent wrote code and you **cannot** Feynman-explain the diff back to a friend in 30 seconds → don't commit yet; run one Feynman prompt on the confusing file.
**Tóm tắt (VI):** Hỏi agent giải thích đơn giản khi khó hiểu React/route/mobile; có prompt mẫu; không commit code mình chưa giải thích lại được.

View File

@@ -11,15 +11,11 @@ Knee Ultrasound Analysis API được xây dựng bằng FastAPI, chuyên phục
### 1.1 Các chức năng chính ### 1.1 Các chức năng chính
* **Phân loại góc chụp siêu âm (Angle Classification):** Tự động nhận dạng mặt cắt/góc chụp từ ảnh đầu vào bao gồm các nhãn: `med-lat`, `post-trans`, `sup-trans-flex`, và `sup-up-long`. - **Phân loại góc chụp siêu âm (Angle Classification):** Tự động nhận dạng mặt cắt/góc chụp từ ảnh đầu vào bao gồm các nhãn: `med-lat`, `post-trans`, `sup-trans-flex`, và `sup-up-long`.
- **Phát hiện viêm (Inflammation Detection):** Xác định sự hiện diện của tình trạng viêm khớp gối qua hai góc chụp chính là `sup-up-long``post-trans`.
* **Phát hiện viêm (Inflammation Detection):** Xác định sự hiện diện của tình trạng viêm khớp gối qua hai góc chụp chính là `sup-up-long``post-trans`. - **Phân đoạn ảnh ngữ nghĩa (Segmentation):** Tách biệt các cấu trúc giải phẫu đích (dịch khớp, gân, xương, màng hoạt dịch...) thành các phân vùng mặt nạ màu riêng biệt.
- **Đo độ dày tự động (Thickness Measurement):** Tự động tính toán khoảng cách hình học theo đơn vị milimét ($mm$) giữa các phân vùng mô mềm đã được phân đoạn (chỉ áp dụng đối với mặt cắt góc `sup-up-long`).
* **Phân đoạn ảnh ngữ nghĩa (Segmentation):** Tách biệt các cấu trúc giải phẫu đích (dịch khớp, gân, xương, màng hoạt dịch...) thành các phân vùng mặt nạ màu riêng biệt. - **Đánh giá mức độ viêm (Severity Analysis):** Xếp hạng thang điểm mức độ nghiêm trọng của viêm từ cấp độ 0 (Rất nhẹ) đến cấp độ 3 (Nặng) dựa trên tỷ lệ diện tích dịch khớp và sự tăng sinh màng hoạt dịch.
* **Đo độ dày tự động (Thickness Measurement):** Tự động tính toán khoảng cách hình học theo đơn vị milimét ($mm$) giữa các phân vùng mô mềm đã được phân đoạn (chỉ áp dụng đối với mặt cắt góc `sup-up-long`).
* **Đánh giá mức độ viêm (Severity Analysis):** Xếp hạng thang điểm mức độ nghiêm trọng của viêm từ cấp độ 0 (Rất nhẹ) đến cấp độ 3 (Nặng) dựa trên tỷ lệ diện tích dịch khớp và sự tăng sinh màng hoạt dịch.
@@ -27,11 +23,13 @@ Knee Ultrasound Analysis API được xây dựng bằng FastAPI, chuyên phục
Khi nhận tập tin ảnh từ máy trạm (Client), hệ thống sẽ thực hiện phân nhánh xử lý logic động dựa trên kết quả của khối phân loại góc chụp: Khi nhận tập tin ảnh từ máy trạm (Client), hệ thống sẽ thực hiện phân nhánh xử lý logic động dựa trên kết quả của khối phân loại góc chụp:
| Góc chụp phát hiện | Quy trình xử lý chi tiết trong Backend pipeline | | Góc chụp phát hiện | Quy trình xử lý chi tiết trong Backend pipeline |
| --- | --- | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`post-trans`** | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh POST $\rightarrow$ Trả kết quả JSON & Mask.| | `post-trans` | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh POST $\rightarrow$ Trả kết quả JSON & Mask. |
| **`sup-up-long`** | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh SUP $\rightarrow$ Đo độ dày mô $\rightarrow$ Đánh giá mức độ nặng $\rightarrow$ Trả kết quả.| | `sup-up-long` | Phân loại góc $\rightarrow$ Phát hiện viêm $\rightarrow$ Phân đoạn ảnh SUP $\rightarrow$ Đo độ dày mô $\rightarrow$ Đánh giá mức độ nặng $\rightarrow$ Trả kết quả. |
| **`med-lat`** or **`sup-trans-flex`** | Chỉ thực hiện phân loại góc $\rightarrow$ Trả kết quả trực tiếp (Bỏ qua nhánh phân đoạn & đo lường).| | `med-lat` or `sup-trans-flex` | Chỉ thực hiện phân loại góc $\rightarrow$ Trả kết quả trực tiếp (Bỏ qua nhánh phân đoạn & đo lường). |
```plantuml ```plantuml
@startuml @startuml
@@ -70,19 +68,27 @@ stop
--- ---
## 2. Hướng dẫn cài đặt & Triển khai môi trường ## 2. Hướng dẫn cài đặt & Triển khai môi trường
### 2.1 Yêu cầu hệ thống tối thiểu ### 2.1 Yêu cầu hệ thống tối thiểu
| Thành phần cấu phần | Thông số kỹ thuật yêu cầu tối thiểu | | Thành phần cấu phần | Thông số kỹ thuật yêu cầu tối thiểu |
| --- | --- | | ------------------------- | ----------------------------------------------------------------------------- |
| **Hệ điều hành** | Ubuntu 20.04+ / Windows 10+ / macOS 12+ | | **Hệ điều hành** | Ubuntu 20.04+ / Windows 10+ / macOS 12+ |
| **Môi trường Python** | Phiên bản 3.10 cố định | | **Môi trường Python** | Phiên bản 3.10 cố định |
| **Bộ nhớ RAM** | 16 GB trở lên | | **Bộ nhớ RAM** | 16 GB trở lên |
| **Bộ xử lý đồ họa (GPU)** | NVIDIA GPU hỗ trợ nền tảng CUDA 12.4 (Khuyến nghị để tối ưu tốc độ) | | **Bộ xử lý đồ họa (GPU)** | NVIDIA GPU hỗ trợ nền tảng CUDA 12.4 (Khuyến nghị để tối ưu tốc độ) |
| **Dung lượng VRAM** | Tối thiểu 8 GB (Khuyến nghị 16 GB nếu chạy song song đồng thời nhiều mô hình)| | **Dung lượng VRAM** | Tối thiểu 8 GB (Khuyến nghị 16 GB nếu chạy song song đồng thời nhiều mô hình) |
| **Ổ cứng lưu trữ** | Tối thiểu 15 GB dung lượng trống (Dành cho bộ cài đặt và file weights `.pth`) | | **Ổ cứng lưu trữ** | Tối thiểu 15 GB dung lượng trống (Dành cho bộ cài đặt và file weights `.pth`) |
| **Bộ công cụ bổ trợ** | CUDA Toolkit 12.4 & cuDNN 9.x tương thích tương ứng| | **Bộ công cụ bổ trợ** | CUDA Toolkit 12.4 & cuDNN 9.x tương thích tương ứng |
### 2.2 Khởi tạo môi trường ảo ### 2.2 Khởi tạo môi trường ảo
@@ -102,7 +108,7 @@ conda activate vkist-ultrasound
``` ```
*Hoặc khởi tạo nhanh bằng mô-đun thư viện chuẩn `venv` nếu hệ thống chưa cài đặt Anaconda*: *Hoặc khởi tạo nhanh bằng mô-đun thư viện chuẩn* `venv` *nếu hệ thống chưa cài đặt Anaconda*:
```bash ```bash
# Trên nền tảng hệ điều hành Linux / macOS # Trên nền tảng hệ điều hành Linux / macOS
@@ -114,6 +120,8 @@ venv\Scripts\activate
``` ```
### 2.3 Cài đặt các gói thư viện phụ thuộc (Dependencies) ### 2.3 Cài đặt các gói thư viện phụ thuộc (Dependencies)
Thực hiện cài đặt các thư viện lõi quy định trong tệp cấu hình: Thực hiện cài đặt các thư viện lõi quy định trong tệp cấu hình:
@@ -133,8 +141,8 @@ pip install torch==2.5.0+cu124 torchvision==0.20.0+cu124 --index-url https://dow
> ⚠️ **LƯU Ý QUAN TRỌNG VỀ PACKAGE NATTEN:** > ⚠️ **LƯU Ý QUAN TRỌNG VỀ PACKAGE NATTEN:**
> Dòng cấu hình cài đặt gói `natten==0.17.3+torch250cu124` mặc định đã bị gắn chú thích (`#` comment out) trong tệp `requirements.txt`. Nếu bạn sử dụng các kiến trúc mạng Transformer nâng cao yêu cầu gói này, bắt buộc cài đặt thủ công qua liên kết phân phối bánh xe (wheels) chính thức: > Dòng cấu hình cài đặt gói `natten==0.17.3+torch250cu124` mặc định đã bị gắn chú thích (`#` comment out) trong tệp `requirements.txt`. Nếu bạn sử dụng các kiến trúc mạng Transformer nâng cao yêu cầu gói này, bắt buộc cài đặt thủ công qua liên kết phân phối bánh xe (wheels) chính thức:
> `pip install natten==0.17.3+torch250cu124 -f https://shi-labs.com/natten/wheels/` > `pip install natten==0.17.3+torch250cu124 -f https://shi-labs.com/natten/wheels/`
>
>
### 2.4 Cấu trúc cây thư mục dự án chuẩn ### 2.4 Cấu trúc cây thư mục dự án chuẩn
@@ -168,6 +176,8 @@ project/
``` ```
### 2.5 Khởi động máy chủ dịch vụ ### 2.5 Khởi động máy chủ dịch vụ
Thực thi lệnh chạy máy chủ tại thư mục gốc: Thực thi lệnh chạy máy chủ tại thư mục gốc:
@@ -187,26 +197,23 @@ python app.py
``` ```
* **Giao diện Web UI kiểm thử trực quan:** `http://localhost:8000` - **Giao diện Web UI kiểm thử trực quan:** `http://localhost:8000`
- **Tài liệu API tương tác tự động (Swagger UI):** `http://localhost:8000/docs`
* **Tài liệu API tương tác tự động (Swagger UI):** `http://localhost:8000/docs`
--- ---
## 3. Tài liệu đặc tả API (API Reference) ## 3. Tài liệu đặc tả API (API Reference)
### 3.1 Trạng thái hoạt động (Health Check) ### 3.1 Trạng thái hoạt động (Health Check)
* **Endpoint:** `GET /api/health` - **Endpoint:** `GET /api/health`
- **Chức năng:** Kiểm tra tính sẵn sàng phục vụ của cụm dịch vụ API Backend.
- **Định dạng dữ liệu phản hồi (Response JSON):**
* **Chức năng:** Kiểm tra tính sẵn sàng phục vụ của cụm dịch vụ API Backend.
* **Định dạng dữ liệu phản hồi (Response JSON):**
```json ```json
{ {
"status": "healthy" "status": "healthy"
@@ -222,28 +229,36 @@ Mã hóa dữ liệu đầu vào dưới dạng tệp tin `multipart/form-data`
#### Các tham số yêu cầu (Request Parameters) #### Các tham số yêu cầu (Request Parameters)
| Tham số cấu hình | Phương thức truyền | Kiểu dữ liệu | Giá trị mặc định | Định nghĩa chức năng chi tiết | | Tham số cấu hình | Phương thức truyền | Kiểu dữ liệu | Giá trị mặc định | Định nghĩa chức năng chi tiết |
| --- | --- | --- | --- | --- | | -------------------- | ------------------ | ------------ | --------------------- | ----------------------------------------------------------------------------------------- |
| **`image`** | Multipart Form | Binary File | *Bắt buộc* | Tệp tin ảnh siêu âm đầu gối cần xử lý (Hỗ trợ mở rộng định dạng: `.jpg`, `.png`, `.bmp`).| | `image` | Multipart Form | Binary File | *Bắt buộc* | Tệp tin ảnh siêu âm đầu gối cần xử lý (Hỗ trợ mở rộng định dạng: `.jpg`, `.png`, `.bmp`). |
| **`angle_model`** | Query String | String | `convnext` | Tên định danh mô hình đảm nhận tác vụ phân loại góc chụp.| | `angle_model` | Query String | String | `convnext` | Tên định danh mô hình đảm nhận tác vụ phân loại góc chụp. |
| **`inflammation_model`** | Query String | String | `efficientnet_b0` | Mô hình phát hiện tình trạng viêm (Hiện tại cố định cấu hình mạng).| | `inflammation_model` | Query String | String | `efficientnet_b0` | Mô hình phát hiện tình trạng viêm (Hiện tại cố định cấu hình mạng). |
| **`segment_model_sup`** | Query String | String | `deeplabv3` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `sup-up-long`.| | `segment_model_sup` | Query String | String | `deeplabv3` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `sup-up-long`. |
| **`segment_model_post`** | Query String | String | `deeplabv3_resnet101` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `post-trans`.| | `segment_model_post` | Query String | String | `deeplabv3_resnet101` | Mô hình phân đoạn cấu trúc giải phẫu dành cho mặt cắt góc `post-trans`. |
#### Danh sách định danh mô hình khả dụng trong hệ thống #### Danh sách định danh mô hình khả dụng trong hệ thống
| Phân nhóm Task | Tên tham số truyền vào | Kiến trúc mạng nơ-ron gốc | Mô tả đặc tính đầu ra | | Phân nhóm Task | Tên tham số truyền vào | Kiến trúc mạng nơ-ron gốc | Mô tả đặc tính đầu ra |
| --- | --- | --- | --- | | ---------------------- | ---------------------- | ------------------------- | ------------------------------------------------------ |
| **Phân loại Góc chụp** | `convnext` | ConvNeXt Tiny | Phân cấp phân loại ra 4 lớp nhãn đầu ra.| | **Phân loại Góc chụp** | `convnext` | ConvNeXt Tiny | Phân cấp phân loại ra 4 lớp nhãn đầu ra. |
| | `densenet` | DenseNet-121 | Mạng kết nối dày đặc.| | | `densenet` | DenseNet-121 | Mạng kết nối dày đặc. |
| | `resnet50` | ResNet-50 | Kiến trúc mạng dư thừa tiêu chuẩn.| | | `resnet50` | ResNet-50 | Kiến trúc mạng dư thừa tiêu chuẩn. |
| | `efficientnet_b2` | EfficientNet-B2 | Tối ưu hóa đa quy mô tài nguyên mạng.| | | `efficientnet_b2` | EfficientNet-B2 | Tối ưu hóa đa quy mô tài nguyên mạng. |
| | `swin` | Swin Transformer V2-S | Kiến trúc Attention cửa sổ dịch chuyển.| | | `swin` | Swin Transformer V2-S | Kiến trúc Attention cửa sổ dịch chuyển. |
| **Phân đoạn góc SUP** | `deeplabv3` | DeepLabV3 ResNet-50 | Trích xuất đặc trưng đa tỷ lệ với 7 lớp đầu ra.| | **Phân đoạn góc SUP** | `deeplabv3` | DeepLabV3 ResNet-50 | Trích xuất đặc trưng đa tỷ lệ với 7 lớp đầu ra. |
| | `unet_resnet101` | UNet + ResNet-101 | Kiến trúc Encoder-Decoder kết hợp ResNet.| | | `unet_resnet101` | UNet + ResNet-101 | Kiến trúc Encoder-Decoder kết hợp ResNet. |
| | `efficientfeedback` | EfficientFeedbackNetwork | Thiết kế tùy biến riêng có liên kết phản hồi dữ liệu.| | | `efficientfeedback` | EfficientFeedbackNetwork | Thiết kế tùy biến riêng có liên kết phản hồi dữ liệu. |
| | `unet3plus` | UNet3+ with Attention | Cơ chế Attention kết hợp kết nối toàn diện Full-scale.| | | `unet3plus` | UNet3+ with Attention | Cơ chế Attention kết hợp kết nối toàn diện Full-scale. |
| **Phân đoạn góc POST** | `deeplabv3_resnet101` | DeepLabV3 ResNet-101 | Cấu trúc chuyên sâu phân đoạn góc nhìn mặt sau.| | **Phân đoạn góc POST** | `deeplabv3_resnet101` | DeepLabV3 ResNet-101 | Cấu trúc chuyên sâu phân đoạn góc nhìn mặt sau. |
#### Cấu trúc dữ liệu JSON phản hồi (Response Body Schema) #### Cấu trúc dữ liệu JSON phản hồi (Response Body Schema)
@@ -300,9 +315,11 @@ Mã hóa dữ liệu đầu vào dưới dạng tệp tin `multipart/form-data`
``` ```
#### Ví dụ mã triển khai gọi dịch vụ (Client Invocations) #### Ví dụ mã triển khai gọi dịch vụ (Client Invocations)
* **Sử dụng lệnh Client cURL CLI:** - **Sử dụng lệnh Client cURL CLI:**
```bash ```bash
curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_model_sup=deeplabv3" \ curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_model_sup=deeplabv3" \
@@ -310,7 +327,7 @@ curl -X POST "http://localhost:8000/api/analyze?angle_model=convnext&segment_mod
``` ```
* **Triển khai ứng dụng gọi qua script Python (Requests):** - **Triển khai ứng dụng gọi qua script Python (Requests):**
```python ```python
import requests import requests
@@ -334,53 +351,71 @@ print("Số liệu đo lường hình học:", parsed_result.get("measurement"))
--- ---
## 4. Các thông số cấu hình lõi hệ thống ## 4. Các thông số cấu hình lõi hệ thống
### 4.1 Hằng số hệ thống trong `app.py` ### 4.1 Hằng số hệ thống trong `app.py`
| Tên định danh hằng số | Giá trị mặc định | Diễn giải chức năng kỹ thuật | | Tên định danh hằng số | Giá trị mặc định | Diễn giải chức năng kỹ thuật |
| --- | --- | --- | | --------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `UPLOAD_FOLDER` | `'uploads'` | Đường dẫn cục bộ lưu trữ file ảnh thô nhận từ máy trạm.| | `UPLOAD_FOLDER` | `'uploads'` | Đường dẫn cục bộ lưu trữ file ảnh thô nhận từ máy trạm. |
| `RESULTS_FOLDER` | `'results'` | Đường dẫn lưu ảnh màu sau phân đoạn (Color Mask Overlayed).| | `RESULTS_FOLDER` | `'results'` | Đường dẫn lưu ảnh màu sau phân đoạn (Color Mask Overlayed). |
| `TEMPLATES_FOLDER` | `'templates'` | Thư mục chứa mã nguồn giao diện phân tích Web UI.| | `TEMPLATES_FOLDER` | `'templates'` | Thư mục chứa mã nguồn giao diện phân tích Web UI. |
| `PIXEL_TO_MM` | $\frac{45.0}{655.0} \approx 0.0687$ | Hệ số chuyển đổi từ độ phân giải pixel sang kích thước thực tế ($mm$). Phụ thuộc cố định vào cấu hình đầu ra của phần cứng máy quét siêu âm.| | `PIXEL_TO_MM` | $\frac{45.0}{655.0} \approx 0.0687$ | Hệ số chuyển đổi từ độ phân giải pixel sang kích thước thực tế ($mm$). Phụ thuộc cố định vào cấu hình đầu ra của phần cứng máy quét siêu âm. |
| `DEFAULT_MEASURE_IDS` | `[1, 5]` | Danh sách mảng chứa ID nhãn lớp cấu trúc giải phẫu kích hoạt thuật toán đo độ dày: `1 = effusion` (Dịch khớp), `5 = synovium` (Màng hoạt dịch).| | `DEFAULT_MEASURE_IDS` | `[1, 5]` | Danh sách mảng chứa ID nhãn lớp cấu trúc giải phẫu kích hoạt thuật toán đo độ dày: `1 = effusion` (Dịch khớp), `5 = synovium` (Màng hoạt dịch). |
| `device` | `cuda` hoặc `cpu` | Khối phần cứng thực thi tính toán đồ họa (Tự động thiết lập dựa trên tính khả dụng của driver NVIDIA).| | `device` | `cuda` hoặc `cpu` | Khối phần cứng thực thi tính toán đồ họa (Tự động thiết lập dựa trên tính khả dụng của driver NVIDIA). |
### 4.2 Cấu hình Pipeline tiền xử lý và biến đổi ma trận ảnh (Transforms) ### 4.2 Cấu hình Pipeline tiền xử lý và biến đổi ma trận ảnh (Transforms)
Hệ thống phân tách ảnh đầu vào thành các luồng biến đổi riêng biệt trước khi nạp vào tensor mô hình tùy thuộc vào mục tiêu xử lý chuyên biệt: Hệ thống phân tách ảnh đầu vào thành các luồng biến đổi riêng biệt trước khi nạp vào tensor mô hình tùy thuộc vào mục tiêu xử lý chuyên biệt:
| Luồng xử lý Pipeline ảnh | Kích thước chuyển đổi (Resize) | Quy định chuẩn hóa phân phối ma trận (Normalization) | | Luồng xử lý Pipeline ảnh | Kích thước chuyển đổi (Resize) | Quy định chuẩn hóa phân phối ma trận (Normalization) |
| --- | --- | --- | | ------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------- |
| **Phân loại góc & Phát hiện viêm** | <br>$224 \times 224$ pixel | Áp dụng phân phối phân cấp:<br> $\text{mean} = [0.485, 0.456, 0.406]$, <br> $\text{std} = [0.229, 0.224, 0.225]$ | | **Phân loại góc & Phát hiện viêm** | $224 \times 224$ pixel | Áp dụng phân phối phân cấp: $\text{mean} = [0.485, 0.456, 0.406]$, $\text{std} = [0.229, 0.224, 0.225]$ |
| **Phân đoạn cấu trúc (Segmentation)** | <br>$512 \times 512$ pixel | Không áp dụng chuẩn hóa phân phối (Chỉ thực thi hàm chuyển đổi tensor `ToTensor()`) | | **Phân đoạn cấu trúc (Segmentation)** | $512 \times 512$ pixel | Không áp dụng chuẩn hóa phân phối (Chỉ thực thi hàm chuyển đổi tensor `ToTensor()`) |
--- ---
## 5. Ràng buộc kỹ thuật & Quy tắc thiết kế hệ thống ## 5. Ràng buộc kỹ thuật & Quy tắc thiết kế hệ thống
### 5.1 Quản lý và giải phóng tài nguyên bộ nhớ GPU (VRAM Leak Warning) ### 5.1 Quản lý và giải phóng tài nguyên bộ nhớ GPU (VRAM Leak Warning)
Trong phiên bản hiện tại, logic xử lý nội tại của API kích hoạt các hàm `load_angle_model()`, `load_inflammation_model()`, và `load_segmentation_model_*()` trực tiếp bên trong vòng đời của mỗi phiên request nhận về. Hành vi này ép buộc GPU liên tục nạp lại dữ liệu tệp `.pth` vào VRAM cho mỗi giao dịch HTTP, sinh ra độ trễ (Overhead) I/O lớn và tiềm ẩn nguy cơ tràn bộ nhớ hệ thống. Khi triển khai môi trường Production, bắt buộc phải tái cấu trúc chuyển các hàm này thành Singleton dịch vụ (Tải một lần duy nhất lúc khởi động tiến trình Web Server). Trong phiên bản hiện tại, logic xử lý nội tại của API kích hoạt các hàm `load_angle_model()`, `load_inflammation_model()`, và `load_segmentation_model_*()` trực tiếp bên trong vòng đời của mỗi phiên request nhận về. Hành vi này ép buộc GPU liên tục nạp lại dữ liệu tệp `.pth` vào VRAM cho mỗi giao dịch HTTP, sinh ra độ trễ (Overhead) I/O lớn và tiềm ẩn nguy cơ tràn bộ nhớ hệ thống. Khi triển khai môi trường Production, bắt buộc phải tái cấu trúc chuyển các hàm này thành Singleton dịch vụ (Tải một lần duy nhất lúc khởi động tiến trình Web Server).
### 5.2 Ràng buộc phi tuyến tính của tham số vật lý `PIXEL_TO_MM` ### 5.2 Ràng buộc phi tuyến tính của tham số vật lý `PIXEL_TO_MM`
Hằng số quy đổi $\text{PIXEL\_TO\_MM} = \frac{45.0}{655.0}$ là một giá trị được cấu hình cứng (Hardcoded) trong mã nguồn, đặc trưng duy nhất cho một dòng máy siêu âm lâm sàng có tỷ lệ hiển thị $45mm$ tương đương với độ phân giải vùng quét $655\text{ px}$. Khi hệ thống thu thập ảnh siêu âm từ các thiết bị chuẩn đoán hình ảnh khác, hoặc thay đổi độ phân giải ảnh xuất ra, số liệu đo khoảng cách tổn thương sẽ sai lệch nghiêm trọng nếu hằng số này không được hiệu chuẩn lại thông qua ma trận nội quan của máy quét mới. Hằng số quy đổi $\text{PIXELTOMM} = \frac{45.0}{655.0}$ là một giá trị được cấu hình cứng (Hardcoded) trong mã nguồn, đặc trưng duy nhất cho một dòng máy siêu âm lâm sàng có tỷ lệ hiển thị $45mm$ tương đương với độ phân giải vùng quét $655\text{ px}$. Khi hệ thống thu thập ảnh siêu âm từ các thiết bị chuẩn đoán hình ảnh khác, hoặc thay đổi độ phân giải ảnh xuất ra, số liệu đo khoảng cách tổn thương sẽ sai lệch nghiêm trọng nếu hằng số này không được hiệu chuẩn lại thông qua ma trận nội quan của máy quét mới.
### 5.3 Quy tắc ánh xạ phân lớp (Class Remapping Matrix) đối với mô hình Custom ### 5.3 Quy tắc ánh xạ phân lớp (Class Remapping Matrix) đối với mô hình Custom
Hai mô hình tùy biến sâu phục vụ mặt cắt góc nhìn phía trên bánh chè (`UNet3+``EfficientFeedback Network`) được huấn luyện trên tập dữ liệu đặc thù sở hữu thứ tự cấu trúc mảng nhãn đầu ra lệch pha hoàn toàn so với kiến trúc phân cấp chuẩn của hệ thống. Để thống nhất dữ liệu trả về cho Client, khối Backend API thực hiện cơ chế tự động chuyển đổi chỉ mục mảng (Index Remapping) theo bảng đặc tả logic dưới đây: Hai mô hình tùy biến sâu phục vụ mặt cắt góc nhìn phía trên bánh chè (`UNet3+``EfficientFeedback Network`) được huấn luyện trên tập dữ liệu đặc thù sở hữu thứ tự cấu trúc mảng nhãn đầu ra lệch pha hoàn toàn so với kiến trúc phân cấp chuẩn của hệ thống. Để thống nhất dữ liệu trả về cho Client, khối Backend API thực hiện cơ chế tự động chuyển đổi chỉ mục mảng (Index Remapping) theo bảng đặc tả logic dưới đây:
| Chỉ mục Mô hình gốc (Output Model Index) | Chỉ mục chuẩn hóa hệ thống (Standard System Index) | Tên nhãn lớp giải phẫu tương ứng (Anatomical Label Class) | | Chỉ mục Mô hình gốc (Output Model Index) | Chỉ mục chuẩn hóa hệ thống (Standard System Index) | Tên nhãn lớp giải phẫu tương ứng (Anatomical Label Class) |
| --- | --- | --- | | ---------------------------------------- | -------------------------------------------------- | --------------------------------------------------------- |
| `0` | `0` | **`background`** (Nền ảnh không chứa cấu trúc) | | `0` | `0` | `background` (Nền ảnh không chứa cấu trúc) |
| `1` | `2` | **`fat`** (Lớp mô mỡ dưới da) | | `1` | `2` | `fat` (Lớp mô mỡ dưới da) |
| `2` | `6` | **`tendon`** (Cấu trúc gân cơ) | | `2` | `6` | `tendon` (Cấu trúc gân cơ) |
| `3` | `1` | **`effusion`** (Vùng tụ dịch khớp gối ổ viêm) | | `3` | `1` | `effusion` (Vùng tụ dịch khớp gối ổ viêm) |
| `4` | `4` | **`femur`** (Ranh giới cấu trúc xương đùi) | | `4` | `4` | `femur` (Ranh giới cấu trúc xương đùi) |
| `5` | `5` | **`synovium`** (Màng hoạt dịch bao quanh khớp) | | `5` | `5` | `synovium` (Màng hoạt dịch bao quanh khớp) |
| `6` | `3` | **`fat-pat`** (Tổ chức mỡ Hoffa) | | `6` | `3` | `fat-pat` (Tổ chức mỡ Hoffa) |
### 5.4 Cơ chế tự động dọn dẹp tập tin tồn đọng (Garbage Collection Task) ### 5.4 Cơ chế tự động dọn dẹp tập tin tồn đọng (Garbage Collection Task)
@@ -388,8 +423,12 @@ Các tập tin ảnh thô tải lên thư mục `uploads/` và ảnh xử lý nh
--- ---
## 6. Giải pháp mở rộng tính năng mã nguồn (Backend Optimization Guide) ## 6. Giải pháp mở rộng tính năng mã nguồn (Backend Optimization Guide)
### 6.1 Tăng tốc độ phản hồi bằng Cơ chế Caching Mô hình Toàn cục ### 6.1 Tăng tốc độ phản hồi bằng Cơ chế Caching Mô hình Toàn cục
Thay thế kiến trúc nạp tải mô hình cũ bằng một kho lưu trữ Cache tĩnh trong bộ nhớ RAM, tối ưu hóa thời gian xử lý request từ mức giây xuống mức mili-giây: Thay thế kiến trúc nạp tải mô hình cũ bằng một kho lưu trữ Cache tĩnh trong bộ nhớ RAM, tối ưu hóa thời gian xử lý request từ mức giây xuống mức mili-giây:
@@ -409,11 +448,13 @@ def get_cached_angle_model(selected_model_name: str):
``` ```
### 6.2 Thêm mới một kiến trúc phân loại góc chụp (Ví dụ: Vision Transformer - ViT) ### 6.2 Thêm mới một kiến trúc phân loại góc chụp (Ví dụ: Vision Transformer - ViT)
Để tích hợp một mạng nơ-ron mới vào hệ thống xử lý, tuân thủ nghiêm ngặt quy trình 3 bước sau: Để tích hợp một mạng nơ-ron mới vào hệ thống xử lý, tuân thủ nghiêm ngặt quy trình 3 bước sau:
* **Bước 1:** Bổ sung khối xử lý điều kiện rẽ nhánh logic vào hàm khởi tạo mô hình `load_angle_model()`: - **Bước 1:** Bổ sung khối xử lý điều kiện rẽ nhánh logic vào hàm khởi tạo mô hình `load_angle_model()`:
```python ```python
elif model_name == 'vit': elif model_name == 'vit':
@@ -428,9 +469,8 @@ elif model_name == 'vit':
``` ```
* **Bước 2:** Di chuyển tệp trọng số huấn luyện nhị phân của mạng (`best_vit_b16.pth`) vào chính xác không gian lưu trữ của thư mục `/models/`. - **Bước 2:** Di chuyển tệp trọng số huấn luyện nhị phân của mạng (`best_vit_b16.pth`) vào chính xác không gian lưu trữ của thư mục `/models/`.
- **Bước 3:** Ứng dụng phía Client có thể kích hoạt mạng mới bằng cách truyền giá trị định danh qua tham số URL: `/api/analyze?angle_model=vit`.
* **Bước 3:** Ứng dụng phía Client có thể kích hoạt mạng mới bằng cách truyền giá trị định danh qua tham số URL: `/api/analyze?angle_model=vit`.
@@ -460,6 +500,8 @@ async def analyze_batch_images(images: List[UploadFile] = File(...)):
``` ```
### 6.4 Bản đóng gói container hóa ứng dụng (Production Dockerfile) ### 6.4 Bản đóng gói container hóa ứng dụng (Production Dockerfile)
Đóng gói toàn bộ ML Stack bao gồm trình điều khiển GPU NVIDIA CUDA để triển khai đồng bộ trên các hạ tầng Cloud hoặc máy chủ On-Premise của bệnh viện: Đóng gói toàn bộ ML Stack bao gồm trình điều khiển GPU NVIDIA CUDA để triển khai đồng bộ trên các hạ tầng Cloud hoặc máy chủ On-Premise của bệnh viện:
@@ -490,15 +532,16 @@ CMD ["python", "app.py"]
``` ```
* **Lệnh khởi dựng Image hệ thống:** `docker build -t medical-api-service .` - **Lệnh khởi dựng Image hệ thống:** `docker build -t medical-api-service .`
- **Lệnh kích hoạt Container chia sẻ tài nguyên phần cứng GPU vật lý:**
* **Lệnh kích hoạt Container chia sẻ tài nguyên phần cứng GPU vật lý:**
```bash ```bash
docker run --gpus all -p 8000:8000 -v $(pwd)/models:/app/models medical-api-service docker run --gpus all -p 8000:8000 -v $(pwd)/models:/app/models medical-api-service
``` ```
### 6.5 Bộ chuyển đổi tiếp nhận trực tiếp luồng dữ liệu ảnh y tế chuẩn DICOM ### 6.5 Bộ chuyển đổi tiếp nhận trực tiếp luồng dữ liệu ảnh y tế chuẩn DICOM
Mở rộng chức năng cho phép hệ thống API đọc trực tiếp tệp tin ảnh gốc dạng `.dcm` trích xuất trực tiếp từ các thiết bị siêu âm chuẩn lâm sàng trong bệnh viện mà không cần qua bước chuyển đổi định dạng thủ công: Mở rộng chức năng cho phép hệ thống API đọc trực tiếp tệp tin ảnh gốc dạng `.dcm` trích xuất trực tiếp từ các thiết bị siêu âm chuẩn lâm sàng trong bệnh viện mà không cần qua bước chuyển đổi định dạng thủ công:
@@ -527,16 +570,21 @@ async def analyze_dicom_file(file: UploadFile = File(...)):
--- ---
## Phụ lục: Đặc tả Dữ liệu định lượng lâm sàng ## Phụ lục: Đặc tả Dữ liệu định lượng lâm sàng
### Phụ lục A: Bảng phân định mã màu mặt nạ phân đoạn ngữ nghĩa (Color Map Legend) ### Phụ lục A: Bảng phân định mã màu mặt nạ phân đoạn ngữ nghĩa (Color Map Legend)
1. Cấu trúc Mặt cắt mặt trên bánh chè - Góc SUP (`sup-up-long`) 1. Cấu trúc Mặt cắt mặt trên bánh chè - Góc SUP (`sup-up-long`)
Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu gối phục vụ thuật toán tính toán độ dày dịch tụ. Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu gối phục vụ thuật toán tính toán độ dày dịch tụ.
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc | | Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
| --- | --- | --- | --- | | ------------------ | ---------------------------- | --------------------- | -------------------------- |
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen (Không chứa dữ liệu) | | `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen (Không chứa dữ liệu) |
| `effusion` | Vùng dịch khớp tụ ổ viêm | `[255, 0, 0]` | 🟥 Đỏ | | `effusion` | Vùng dịch khớp tụ ổ viêm | `[255, 0, 0]` | 🟥 Đỏ |
| `fat` | Tổ chức mô mỡ dưới da | `[255, 255, 0]` | 🟨 Vàng | | `fat` | Tổ chức mô mỡ dưới da | `[255, 255, 0]` | 🟨 Vàng |
@@ -545,15 +593,15 @@ Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu g
| `synovium` | Lớp màng hoạt dịch tăng sinh | `[255, 0, 255]` | 🟪 Tím | | `synovium` | Lớp màng hoạt dịch tăng sinh | `[255, 0, 255]` | 🟪 Tím |
| `tendon` | Vùng bó gân cơ | `[0, 0, 255]` | 🟦 Xanh dương | | `tendon` | Vùng bó gân cơ | `[0, 0, 255]` | 🟦 Xanh dương |
> 🔄 **QUY TẮC CHUYỂN ĐỔI CHUYỂN GÓC (SUP $\rightarrow$ POST):**
> Khi hệ thống chuyển đổi trạng thái phân tích sang mặt cắt phía sau khớp gối (Góc `POST`), ma trận thuật toán phân đoạn sẽ tự động tái cấu trúc màu sắc ngữ nghĩa: Vùng tổn thương chứa **`effusion`** (màu đỏ) sẽ chuyển trạng thái biểu diễn thành **`baker's cyst`** (Kén Baker), và tổ chức cấu trúc vùng **`fat-pat`** (màu lam sáng) sẽ hoán đổi ý nghĩa thành vùng **`muscle`** (Cơ bắp vùng khoeo).
>
>
2. Cấu trúc Mặt cắt mặt sau vùng khoeo chân - Góc POST (`post-trans`) > 🔄 **QUY TẮC CHUYỂN ĐỔI CHUYỂN GÓC (SUP $\rightarrow$ POST):**
> Khi hệ thống chuyển đổi trạng thái phân tích sang mặt cắt phía sau khớp gối (Góc `POST`), ma trận thuật toán phân đoạn sẽ tự động tái cấu trúc màu sắc ngữ nghĩa: Vùng tổn thương chứa `effusion` (màu đỏ) sẽ chuyển trạng thái biểu diễn thành `baker's cyst` (Kén Baker), và tổ chức cấu trúc vùng `fat-pat` (màu lam sáng) sẽ hoán đổi ý nghĩa thành vùng `muscle` (Cơ bắp vùng khoeo).
1. Cấu trúc Mặt cắt mặt sau vùng khoeo chân - Góc POST (`post-trans`)
| Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc | | Từ khóa nhãn (Key) | Cấu trúc giải phẫu đích | Mã màu hiển thị (RGB) | Trực quan hóa màu sắc |
| --- | --- | --- | --- | | ------------------ | ---------------------------------------- | --------------------- | --------------------- |
| `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen | | `background` | Nền ảnh siêu âm | `[0, 0, 0]` | ⬛ Đen |
| `baker's cyst` | Tổ chức kén hoạt dịch vùng khoeo (Baker) | `[255, 0, 0]` | 🟥 Đỏ | | `baker's cyst` | Tổ chức kén hoạt dịch vùng khoeo (Baker) | `[255, 0, 0]` | 🟥 Đỏ |
| `fat` | Lớp mô mỡ | `[255, 255, 0]` | 🟨 Vàng | | `fat` | Lớp mô mỡ | `[255, 255, 0]` | 🟨 Vàng |
@@ -562,17 +610,21 @@ Góc SUP tập trung khoanh vùng các lớp mô mềm phía trước đầu g
| `synovium` | Màng hoạt dịch mặt sau | `[255, 0, 255]` | 🟪 Tím | | `synovium` | Màng hoạt dịch mặt sau | `[255, 0, 255]` | 🟪 Tím |
| `tendon` | Hệ thống gân cơ mặt sau | `[0, 0, 255]` | 🟦 Xanh dương | | `tendon` | Hệ thống gân cơ mặt sau | `[0, 0, 255]` | 🟦 Xanh dương |
--- ---
### Phụ lục B: Thang điểm đánh giá mức độ nghiêm trọng của ổ viêm (Clinical Severity Score) ### Phụ lục B: Thang điểm đánh giá mức độ nghiêm trọng của ổ viêm (Clinical Severity Score)
Hệ thống chấm điểm toán học tự động căn cứ trên trọng số diện tích và độ dày phân tách để đưa ra kết luận mức độ bệnh lý lâm sàng thông qua phương trình tuyến tính tổng hợp: Hệ thống chấm điểm toán học tự động căn cứ trên trọng số diện tích và độ dày phân tách để đưa ra kết luận mức độ bệnh lý lâm sàng thông qua phương trình tuyến tính tổng hợp:
$$\text{combined\_score} = \text{effusion\_score} \times 0.6 + \text{synovium\_score} \times 0.4$$ $$\text{combinedscore} = \text{effusionscore} \times 0.6 + \text{synoviumscore} \times 0.4$$
Dựa trên kết quả giá trị của biến số $\text{combined\_score}$, hệ thống tự động phân cấp thành 4 ngưỡng trạng thái lâm sàng tương ứng: Dựa trên kết quả giá trị của biến số $\text{combinedscore}$, hệ thống tự động phân cấp thành 4 ngưỡng trạng thái lâm sàng tương ứng:
- **Mức 0 - Rất nhẹ ($\text{score} < 3$):** Trạng thái dịch khớp cấu trúc màng hoạt dịch nằm hoàn toàn trong giới hạn sinh bình thường của thể.
- **Mức 1 - Nhẹ ($\text{score}$ từ $3$ đến $7.9$):** Xuất hiện hiện tượng tụ dịch khớp lớp mỏng, màng hoạt dịch dấu hiệu tăng sinh nhẹ cấu trúc màng.
- **Mức 2 - Trung bình ($\text{score}$ từ $8$ đến $15$):** Lượng dịch tụ khớp gối mức độ vừa phải, màng hoạt dịch bắt đầu phì đại tăng sinh nét.
- **Mức 3 - Nặng ($\text{score} > 15$):** Lớp tụ dịch khớp gối dày kích thước lớn, màng hoạt dịch tăng sinh phì đại mạnh, lan rộng diện tích cấu trúc giải phẫu xung quanh.
* **Mức 0 - Rất nhẹ ($\text{score} < 3$):** Trạng thái dịch khớp cấu trúc màng hoạt dịch nằm hoàn toàn trong giới hạn sinh bình thường của thể.
* **Mức 1 - Nhẹ ($\text{score}$ từ $3$ đến $7.9$):** Xuất hiện hiện tượng tụ dịch khớp lớp mỏng, màng hoạt dịch dấu hiệu tăng sinh nhẹ cấu trúc màng.
* **Mức 2 - Trung bình ($\text{score}$ từ $8$ đến $15$):** Lượng dịch tụ khớp gối mức độ vừa phải, màng hoạt dịch bắt đầu phì đại tăng sinh nét.
* **Mức 3 - Nặng ($\text{score} > 15$):** Lớp tụ dịch khớp gối dày kích thước lớn, màng hoạt dịch tăng sinh phì đại mạnh, lan rộng diện tích cấu trúc giải phẫu xung quanh.

View File

@@ -0,0 +1,233 @@
import asyncio
import logging
from contextlib import asynccontextmanager
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
from fastapi.security import OAuth2PasswordBearer
from fastapi.responses import StreamingResponse
from datetime import datetime
from data.spec.schemas import (
AnalysisJobSubmit, JobStatus, PipelineStep, StepEvent,
ModelCatalog, ModelRegistrationResult, HealthStatus,
AnalysisJobSyncSubmit, JobResult, ErrorResponse,
)
from backend.implementation.analysis_jobs import service as analysis_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1", tags=["analysis"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
_event_queues: dict[str, asyncio.Queue] = {}
_queue_lock = asyncio.Lock()
@asynccontextmanager
async def _get_queue(job_id: str):
async with _queue_lock:
if job_id not in _event_queues:
_event_queues[job_id] = asyncio.Queue()
yield _event_queues[job_id]
def _get_queue_sync(job_id: str) -> asyncio.Queue:
if job_id not in _event_queues:
_event_queues[job_id] = asyncio.Queue()
return _event_queues[job_id]
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
def _sse_format(event: StepEvent) -> str:
lines = [f"event: {event.event_type}"]
payload = event.model_dump(mode="json")
lines.append(f"data: {payload}")
lines.append("")
lines.append("")
return "\n".join(lines)
@router.post(
"/analysis-jobs",
response_model=dict,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def submit_analysis_job(
payload: AnalysisJobSubmit,
user_id: str = Depends(_verify_jwt_token),
):
try:
job_id = await analysis_service.submit_job(
session_id=payload.session_id,
params=payload.params or {},
model_versions=payload.model_versions,
)
return {"job_id": job_id}
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.get(
"/analysis-jobs/{job_id}",
response_model=JobStatus,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def get_job_status(job_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await analysis_service.job_status(job_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.get(
"/analysis-jobs/{job_id}/steps",
response_model=list[PipelineStep],
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def get_job_steps(job_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await analysis_service.job_steps(job_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/analysis",
response_model=JobResult,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def submit_sync_analysis(
payload: AnalysisJobSyncSubmit,
user_id: str = Depends(_verify_jwt_token),
):
try:
return await analysis_service.submit_sync(
session_id=payload.session_id,
params=payload.params or {},
model_versions=payload.model_versions,
)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.get(
"/analysis-jobs/{job_id}/stream",
responses={
401: {"model": ErrorResponse},
404: {"model": ErrorResponse},
},
)
async def stream_job_events(job_id: str, user_id: str = Depends(_verify_jwt_token)):
queue = _get_queue_sync(job_id)
async def event_generator():
try:
while True:
event = await queue.get()
yield _sse_format(event)
if event.event_type == "completed" or event.event_type == "failed":
break
except asyncio.CancelledError:
logger.info(f"SSE stream cancelled for job_id={job_id}")
finally:
async with _queue_lock:
_event_queues.pop(job_id, None)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
"Connection": "keep-alive",
},
)
@router.post(
"/internal/analysis-jobs/{job_id}/events",
status_code=status.HTTP_202_ACCEPTED,
include_in_schema=False,
)
async def internal_push_event(job_id: str, event: dict):
queue = _get_queue_sync(job_id)
try:
step_event = StepEvent(
step_id=event.get("step_id", ""),
job_id=job_id,
event_type=event.get("event_type", "progress"),
task_type=event.get("task_type", ""),
status=event.get("status", "running"),
data=event.get("data"),
timestamp=datetime.now(),
)
await queue.put(step_event)
except Exception as exc:
logger.error(f"Failed to push event for job {job_id}: {exc}")
return {"queued": True}
@router.get(
"/health",
response_model=HealthStatus,
include_in_schema=False,
)
async def health_check():
try:
return await analysis_service.health()
except NotImplementedError:
return HealthStatus(
status="ok",
version="0.1.0",
dependencies={},
uptime_seconds=0.0,
)
@router.get(
"/model-registry",
response_model=ModelCatalog,
responses={401: {"model": ErrorResponse}},
)
async def list_models(user_id: str = Depends(_verify_jwt_token)):
try:
return await analysis_service.list_registered_models()
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.post(
"/models/register",
response_model=ModelRegistrationResult,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}},
)
async def register_model(
model_id: str,
file: UploadFile | None = None,
user_id: str = Depends(_verify_jwt_token),
):
try:
return await analysis_service.register_model(model_id, file)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))

View File

@@ -0,0 +1,82 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import LoginRequest, Token, UserProfile, UserUpdateRequest, RefreshRequest, ErrorResponse
from backend.implementation.auth import service as auth_service
router = APIRouter(prefix="/api/v1", tags=["auth"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
profile = await auth_service.me(token)
return profile.user_id
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.post(
"/auth/login",
response_model=Token,
responses={401: {"model": ErrorResponse}},
)
async def login(payload: LoginRequest):
try:
return await auth_service.login(payload.username, payload.password)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc))
@router.post(
"/auth/logout",
status_code=status.HTTP_204_NO_CONTENT,
)
async def logout(token: str = Depends(oauth2_scheme)):
try:
await auth_service.logout(token)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.post(
"/auth/refresh",
response_model=Token,
responses={401: {"model": ErrorResponse}},
)
async def refresh(payload: RefreshRequest):
try:
return await auth_service.refresh(payload.refresh_token)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc))
@router.get(
"/users/me",
response_model=UserProfile,
responses={401: {"model": ErrorResponse}},
)
async def get_me(user_id: str = Depends(verify_jwt_token)):
try:
return await auth_service.me(user_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.patch(
"/users/me",
response_model=UserProfile,
responses={401: {"model": ErrorResponse}},
)
async def update_me(payload: UserUpdateRequest, user_id: str = Depends(verify_jwt_token)):
try:
return await auth_service.update_me(user_id, payload.model_dump(exclude_none=True))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))

View File

@@ -0,0 +1,47 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import IngestionRecord, RecordDetail, ErrorResponse
from backend.implementation.ingestion_history import service as ingestion_service
router = APIRouter(prefix="/api/v1", tags=["ingestion-history"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.get(
"/ingestion-history",
response_model=list[IngestionRecord],
responses={401: {"model": ErrorResponse}},
)
async def list_ingestion_records(user_id: str = Depends(_verify_jwt_token)):
try:
return await ingestion_service.list_records(user_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.get(
"/ingestion-history/{record_id}",
response_model=RecordDetail,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def get_ingestion_record(record_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await ingestion_service.get_record(record_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))

View File

@@ -0,0 +1,59 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import NotificationItem, NotificationPreferences, ErrorResponse
from backend.implementation.notification import service as notification_service
router = APIRouter(prefix="/api/v1", tags=["notification"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.get(
"/notifications",
response_model=list[NotificationItem],
responses={401: {"model": ErrorResponse}},
)
async def list_notifications(user_id: str = Depends(_verify_jwt_token), filters: dict | None = None):
try:
return await notification_service.list_notifications(user_id, filters)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.patch(
"/notifications/{notification_id}/read",
status_code=status.HTTP_204_NO_CONTENT,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def mark_read(notification_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
await notification_service.mark_read(notification_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/notifications/preferences",
status_code=status.HTTP_204_NO_CONTENT,
responses={401: {"model": ErrorResponse}},
)
async def set_preferences(payload: NotificationPreferences, user_id: str = Depends(_verify_jwt_token)):
try:
await notification_service.set_preferences(user_id, payload.model_dump(exclude_none=True))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))

View File

@@ -0,0 +1,91 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import Patient, PatientCreate, PatientListResponse, ErrorResponse
from backend.implementation.patient import service as patient_service
router = APIRouter(prefix="/api/v1", tags=["patient"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.get(
"/patients",
response_model=PatientListResponse,
responses={401: {"model": ErrorResponse}},
)
async def list_patients(user_id: str = Depends(_verify_jwt_token)):
try:
items = await patient_service.list_patients(user_id)
return PatientListResponse(items=items, total=len(items))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.post(
"/patients",
response_model=Patient,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def create_patient(payload: PatientCreate, user_id: str = Depends(_verify_jwt_token)):
try:
return await patient_service.create_patient(payload.model_dump(exclude_none=True))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.get(
"/patients/{patient_id}",
response_model=Patient,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def get_patient(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await patient_service.get_patient(patient_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.get(
"/patients/{patient_id}/sessions",
response_model=list[dict],
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def list_patient_sessions(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await patient_service.list_sessions(patient_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.get(
"/patients/{patient_id}/history",
response_model=list[dict],
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def patient_ingestion_history(patient_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await patient_service.ingestion_history(patient_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))

View File

@@ -0,0 +1,320 @@
import asyncio
from typing import Any
import httpx
import logging
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
from fastapi.security import OAuth2PasswordBearer
from fastapi.responses import StreamingResponse
from PILOT_PROJECT.workspace.sprint_1_2.CODEBASE.data.spec.schemas.safety_schemas import ChatResponse
from data.spec.schemas import (
HeatmapResult, RationaleResult, ChatEvent, DriftCheckResult,
EvidenceList, ActivationMeta, AnnotationArtifact, EscalationTicket,
GuardrailResult, ErrorResponse, CorrectionSubmit, CorrectionRecord,
)
from backend.implementation.safety import service as safety_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1", tags=["safety"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
def _sse_chat_format(event: ChatEvent) -> str:
lines = [f"event: {event.event_type}"]
payload = event.model_dump(mode="json")
lines.append(f"data: {payload}")
lines.append("")
lines.append("")
return "\n".join(lines)
@router.post(
"/sessions/{session_id}/explanations/gradcam",
response_model=HeatmapResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def gradcam(session_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await safety_service.gradcam(session_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/explanations/rationale",
response_model=RationaleResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def rationale(
session_id: str,
redaction_hash: str | None = None,
user_id: str = Depends(_verify_jwt_token)
):
try:
return await safety_service.rationale(session_id, redaction_hash)
except HTTPException:
raise
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/safety/circuit-breaker",
status_code=status.HTTP_204_NO_CONTENT,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def circuit_breaker(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
try:
await safety_service.circuit_break(session_id, payload.get("flag", False))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/chat/socratic",
response_model=ChatResponse,
responses={401: {"model": ErrorResponse}, 404: {"model: ErrorResponse}},
)
async def socratic_chat(
session_id: str,
payload: dict,
redaction_hash: str | None = None,
user_id: str = Depends(_verify_jwt_token)
):
try:
return await safety_service.socratic_chat(
session_id,
payload.get("prompt", ""),
redaction_hash=redaction_hash
)
except HTTPException:
raise
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/drift/check",
response_model=DriftCheckResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def drift_check(session_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await safety_service.drift_check(session_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/rag/evidence",
response_model=EvidenceList,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def rag_evidence(session_id: str, user_id: str = Depends(_verify_jwt_token)):
try:
return await safety_service.rag_evidence(session_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/activations",
response_model=ActivationMeta,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def activations(session_id: str, params: dict, user_id: str = Depends(_verify_jwt_token)):
try:
return await safety_service.activations(session_id, params)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/annotations/artifacts",
response_model=AnnotationArtifact,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def upload_artifact(
session_id: str,
file: UploadFile = File(...),
user_id: str = Depends(_verify_jwt_token),
):
try:
return await safety_service.upload_artifact(session_id, file)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/ground-truth",
status_code=status.HTTP_204_NO_CONTENT,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def ground_truth(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
try:
await safety_service.ground_truth(session_id, payload.get("label", {}))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/escalation",
response_model=EscalationTicket,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def escalate(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
try:
return await safety_service.escalate(session_id, payload.get("reason", ""))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/annotations/morphology",
status_code=status.HTTP_204_NO_CONTENT,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def morphology(session_id: str, payload: dict, user_id: str = Depends(_verify_jwt_token)):
try:
await safety_service.morphology(session_id, payload.get("annotation", {}))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/safety/guardrail-check",
response_model=GuardrailResult,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def guardrail_check(payload: dict, user_id: str = Depends(_verify_jwt_token)):
try:
session_id = payload.get("session_id", "")
return await safety_service.guardrail_check(
session_id=session_id,
prompt=payload.get("prompt", ""),
score=float(payload.get("score", 0.0)),
)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except (ValueError, TypeError) as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.post(
"/sessions/{session_id}/feedback",
response_model=CorrectionRecord,
status_code=status.HTTP_201_CREATED,
responses={
401: {"model": ErrorResponse},
404: {"model": ErrorResponse},
422: {"model": ErrorResponse},
},
)
async def submit_correction(
session_id: str,
payload: CorrectionSubmit,
user_id: str = Depends(_verify_jwt_token),
):
try:
return await safety_service.submit_correction(session_id, payload.dict())
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.get(
"/sessions/{session_id}/chat/stream",
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def chat_stream(
session_id: str,
prompt: str,
redaction_hash: str | None = None,
user_id: str = Depends(_verify_jwt_token),
):
async def generate():
try:
async for chunk in safety_service.chat_stream(session_id, prompt, redaction_hash):
event = ChatEvent(
session_id=session_id,
event_type="chunk",
content=chunk,
is_final=False,
)
yield _sse_chat_format(event)
final_event = ChatEvent(
session_id=session_id,
event_type="completed",
content="",
is_final=True,
)
yield _sse_chat_format(final_event)
except HTTPException as exc:
error_event = ChatEvent(
session_id=session_id,
event_type="error",
content=exc.detail,
is_final=True,
)
yield _sse_chat_format(error_event)
except NotImplementedError:
fallback_event = ChatEvent(
session_id=session_id,
event_type="error",
content="Chat streaming service not yet implemented",
is_final=True,
)
yield _sse_chat_format(fallback_event)
except asyncio.CancelledError:
logger.info(f"Chat stream cancelled for session_id={session_id}")
return StreamingResponse(
generate(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
"Connection": "keep-alive",
},
)

View File

@@ -0,0 +1,214 @@
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status, UploadFile, File
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import (
Session, SessionDetail, SessionCreate, SessionPatchReview,
FrameMetadata, PersistResult, ExportResult, ScrubResult,
ReportCreate, ReportSignRequest, ReportSyncEMRRequest, SyncResult,
ErrorResponse,
)
from backend.implementation.session import service as session_service
router = APIRouter(prefix="/api/v1", tags=["session"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.post(
"/sessions",
response_model=Session,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def create_session(payload: SessionCreate, user_id: str = Depends(verify_jwt_token)):
try:
return await session_service.create_session(
user_id=user_id,
patient_id=payload.patient_id,
case_id=getattr(payload, "case_id", None),
)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.get(
"/sessions/{session_id}",
response_model=SessionDetail,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def get_session(session_id: str, user_id: str = Depends(verify_jwt_token)):
try:
return await session_service.get_session(session_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/frames",
response_model=FrameMetadata,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def add_frame(
session_id: str,
file: UploadFile = File(...),
frame_number: int | None = None,
user_id: str = Depends(verify_jwt_token),
):
try:
return await session_service.add_frame(session_id, file, frame_number)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.patch(
"/sessions/{session_id}/review",
response_model=Session,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def patch_review(
session_id: str,
payload: SessionPatchReview,
user_id: str = Depends(verify_jwt_token),
):
try:
return await session_service.patch_review(session_id, payload.model_dump(exclude_none=True))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
@router.post(
"/reports",
response_model=dict,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def create_report(payload: ReportCreate, user_id: str = Depends(verify_jwt_token)):
try:
result = await session_service.persist(payload.session_id, payload.payload)
return {"report_id": result.session_id, "status": result.status, "updated_at": result.updated_at.isoformat()}
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/reports/{report_id}/sign",
response_model=dict,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def sign_report(
report_id: str,
payload: ReportSignRequest,
user_id: str = Depends(verify_jwt_token),
):
try:
result = await session_service.persist(report_id, {"signed": True, "signature": payload.signature})
return {"report_id": report_id, "signed": True, "updated_at": result.updated_at.isoformat()}
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/reports/{report_id}/emr-sync",
response_model=SyncResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def sync_emr(
report_id: str,
payload: ReportSyncEMRRequest,
user_id: str = Depends(verify_jwt_token),
):
from datetime import datetime
try:
result = await session_service.persist(report_id, {"emr_sync": True})
return SyncResult(
report_id=report_id,
emr_status="pending",
emr_reference=None,
synced_at=datetime.utcnow(),
)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/persist",
response_model=PersistResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def persist_session(
session_id: str,
review: dict,
user_id: str = Depends(verify_jwt_token),
):
try:
return await session_service.persist(session_id, review)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/export-pdf",
response_model=ExportResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def export_pdf(
session_id: str,
params: dict,
user_id: str = Depends(verify_jwt_token),
):
try:
return await session_service.export_pdf(session_id, params)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))
@router.post(
"/sessions/{session_id}/scrub-validate",
response_model=ScrubResult,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def scrub_validate(
session_id: str,
metadata: dict,
user_id: str = Depends(verify_jwt_token),
):
try:
return await session_service.scrub_validate(session_id, metadata)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))

View File

@@ -0,0 +1,47 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import UserSettings, SettingsUpdate, ErrorResponse
from backend.implementation.settings import service as settings_service
router = APIRouter(prefix="/api/v1", tags=["settings"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def _verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.get(
"/settings",
response_model=UserSettings,
responses={401: {"model": ErrorResponse}},
)
async def get_settings(user_id: str = Depends(_verify_jwt_token)):
try:
return await settings_service.get_settings(user_id)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
@router.patch(
"/settings",
response_model=UserSettings,
responses={401: {"model": ErrorResponse}, 422: {"model": ErrorResponse}},
)
async def update_settings(payload: SettingsUpdate, user_id: str = Depends(_verify_jwt_token)):
try:
return await settings_service.update_settings(user_id, payload.model_dump(exclude_none=True))
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))

View File

@@ -0,0 +1,40 @@
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from data.spec.schemas import AnomalyReport, AnomalyRecord, ErrorResponse
from backend.implementation.telemetry import service as telemetry_service
router = APIRouter(prefix="/api/v1", tags=["telemetry"])
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
async def verify_jwt_token(token: str = Depends(oauth2_scheme)) -> str:
try:
from backend.api.auth_api import verify_jwt_token as _verify
return await _verify(token)
except HTTPException:
raise
except Exception:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
@router.post(
"/analysis-jobs/{job_id}/anomalies",
response_model=AnomalyRecord,
status_code=status.HTTP_201_CREATED,
responses={401: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
async def report_anomaly(
job_id: str,
payload: AnomalyReport,
user_id: str = Depends(verify_jwt_token),
):
try:
return await telemetry_service.report_anomaly(job_id, payload.data)
except NotImplementedError as exc:
raise HTTPException(status_code=status.HTTP_501_NOT_IMPLEMENTED, detail=str(exc))
except LookupError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc))

View File

@@ -0,0 +1,90 @@
"""
Standalone FastAPI server for CV inference (Modal Triton).
Run from CODEBASE root:
PYTHONPATH=. python -m backend.cv_inference_server
Or the backward-compatible launcher:
PYTHONPATH=. python backend/tests/test_fast_api_proxy.py
Default: http://127.0.0.1:8001 — point the frontend Vite proxy here (see .env.development).
Env:
TRITON_ENDPOINT Modal Triton KServe v2 HTTP URL
CV_INFERENCE_HOST bind host (default 127.0.0.1)
CV_INFERENCE_PORT bind port (default 8001)
ANGLE_MODEL / INFLAMMATION_MODEL / SEGMENT_MODEL optional overrides
CV_PIPELINE_VERSION cache invalidation fingerprint (default poc-v2-spec-cv)
"""
from __future__ import annotations
import logging
import os
from contextlib import asynccontextmanager
# Must run before backend imports — config reads TRITON_ENDPOINT at import time.
DEFAULT_TRITON_ENDPOINT = "https://dtj-tran--triton-s3-service-unified-triton-server.modal.run"
os.environ.setdefault("TRITON_ENDPOINT", DEFAULT_TRITON_ENDPOINT)
import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
import asyncio
from backend.routers import cv_inference
logger = logging.getLogger(__name__)
DEFAULT_TRITON_ENDPOINT = "https://dtj-tran--triton-s3-service-unified-triton-server.modal.run"
@asynccontextmanager
async def lifespan(app: FastAPI):
logger.info("Starting CV inference service on Triton: %s", os.getenv("TRITON_ENDPOINT"))
from backend.services.triton_warmup import warmup_triton_models
warmup_task = asyncio.create_task(warmup_triton_models())
yield
logger.info("Shutting down CV inference service")
def create_app() -> FastAPI:
app = FastAPI(
title="VKIST CV Inference Service",
version="0.2.0",
description=(
"Spec-compliant musculoskeletal ultrasound CV pipeline "
"(CLAHE → angle → inflammation → conditional segmentation)."
),
lifespan=lifespan,
)
app.add_middleware(
CORSMiddleware,
allow_origins=os.getenv(
"CORS_ORIGINS",
"http://localhost:3000,http://localhost:5173,http://localhost:4173,http://127.0.0.1:5173",
).split(","),
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(cv_inference.router)
return app
app = create_app()
def main() -> None:
logging.basicConfig(level=logging.INFO)
host = os.getenv("CV_INFERENCE_HOST", os.getenv("SEGMENT_TEST_HOST", "127.0.0.1"))
port = int(os.getenv("CV_INFERENCE_PORT", os.getenv("SEGMENT_TEST_PORT", "8001")))
logger.info("CV inference service listening on %s:%s", host, port)
uvicorn.run(app, host=host, port=port, log_level="info")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,48 @@
import logging
from typing import NamedTuple, Optional
from dataclasses import dataclass
logger = logging.getLogger(__name__)
@dataclass
class DriftResult:
score: float
is_drifted: bool
threshold: float
@dataclass
class GuardrailResult:
verdict: str # "PASS" | "MITIGATE"
reason: Optional[str] = None
class BERTAdapter:
"""
Adapter for BERT-based safety checks (drift, referee, guardrails).
Current implementation provides stubs.
"""
def __init__(self):
self.drift_threshold = 0.7
def drift_check(self, text: str) -> DriftResult:
"""
Checks if the input text drifts from the clinical domain.
"""
# Stub: Always return no drift
return DriftResult(score=0.9, is_drifted=False, threshold=self.drift_threshold)
def referee_check(self, text: str, retrieved_chunks: list) -> bool:
"""
RAG-Referee: Validates if LLM response is grounded in provided chunks.
"""
# Stub: Always return grounded
return True
def guardrail_check(self, text: str) -> GuardrailResult:
"""
Token/Chunk level guardrail check for hallucinations or scope violations.
"""
# Stub: Always return PASS
return GuardrailResult(verdict="PASS")
def get_bert_adapter() -> BERTAdapter:
return BERTAdapter()

View File

@@ -0,0 +1,71 @@
import logging
from typing import Any, AsyncGenerator, List, Optional
from langchain_google_vertexai import VertexAI
from langchain.callbacks.base import BaseCallbackHandler
from langchain.schema import LLMResult
from backend.implementation import config
logger = logging.getLogger(__name__)
class AuditCallbackHandler(BaseCallbackHandler):
"""
Langchain callback to enforce audit logging before LLM calls per NFR-16a.
"""
def __init__(self, session_id: str, metadata: Optional[dict] = None):
self.session_id = session_id
self.metadata = metadata or {}
def on_llm_start(self, serialized: Any, prompts: List[str], **kwargs) -> None:
# MANDATORY: Write egress_consent + egress_redact_manifest to immutable audit log
# In a real implementation, this would call a database service to commit to Postgres.
logger.info(f"[AUDIT] Pre-egress audit commit for session {self.session_id}. "
f"Prompts: {prompts}. Metadata: {self.metadata}")
def on_llm_end(self, response: LLMResult, **kwargs) -> None:
# Log actual egress event after completion
logger.info(f"[AUDIT] LLM egress completed for session {self.session_id}")
def on_llm_error(self, error: Exception, **kwargs) -> None:
logger.error(f"[AUDIT] LLM error for session {self.session_id}: {str(error)}")
class VertexAILangchainAdapter:
def __init__(self):
self.llm = VertexAI(
model_name=config.VERTEX_AI_MODEL,
project_id=config.VERTEX_AI_PROJECT,
location=config.VERTEX_AI_LOCATION,
max_output_tokens=256,
temperature=0.2,
top_p=0.8,
top_k=40,
)
async def generate(self, prompt: str, session_id: str, metadata: Optional[dict] = None) -> str:
import asyncio
loop = asyncio.get_event_loop()
callback_handler = AuditCallbackHandler(session_id, metadata)
def _sync_generate():
result = self.llm.generate(
prompts=[prompt],
callbacks=[callback_handler]
)
return result.generations[0][0].text
return await loop.run_in_executor(None, _sync_generate)
async def stream_generate(self, prompt: str, session_id: str, metadata: Optional[dict] = None) -> AsyncGenerator[str, None]:
import asyncio
loop = asyncio.get_event_loop()
callback_handler = AuditCallbackHandler(session_id, metadata)
def _sync_stream():
return self.llm.stream(prompt, callbacks=[callback_handler])
stream = await loop.run_in_executor(None, _sync_stream)
for chunk in stream:
yield chunk
def get_llm_adapter() -> VertexAILangchainAdapter:
return VertexAILangchainAdapter()

View File

@@ -0,0 +1,40 @@
import redis
import logging
from backend.implementation import config
logger = logging.getLogger(__name__)
class RedisClient:
"""
Singleton Redis client for managing session state and consult_mode.
"""
_instance = None
def __new__(cls):
if cls._instance is None:
cls._instance = super(RedisClient, cls).__new__(cls)
try:
cls._instance.client = redis.Redis(
host=config.REDIS_HOST,
port=config.REDIS_PORT,
db=config.REDIS_DB,
decode_responses=True
)
logger.info("Connected to Redis at %s:%s", config.REDIS_HOST, config.REDIS_PORT)
except Exception as e:
logger.error("Failed to connect to Redis: %s", e)
cls._instance.client = None
return cls._instance
def get(self, key: str):
return self.client.get(key) if self.client else None
def set(self, key: str, value: str, ex: int = None):
if self.client:
self.client.set(key, value, ex=ex)
def exists(self, key: str) -> bool:
return bool(self.client.exists(key)) if self.client else False
def get_redis_client() -> RedisClient:
return RedisClient()

View File

@@ -0,0 +1,139 @@
import asyncio
import json
from typing import Any
import numpy as np
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
class TritonAdapter:
def __init__(self, endpoint_url: str, timeout: float = 60.0):
self.endpoint_url = endpoint_url.rstrip("/")
self.timeout = timeout
self._session = self._build_session()
@staticmethod
def _build_session() -> requests.Session:
session = requests.Session()
retry = Retry(
total=0,
connect=0,
read=0,
redirect=0,
status=0,
raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry, pool_connections=20, pool_maxsize=50)
session.mount("http://", adapter)
session.mount("https://", adapter)
return session
async def close(self):
await asyncio.to_thread(self._session.close)
async def infer(
self, model_name: str, inputs: dict, outputs: list[str] | None = None
) -> dict:
return await asyncio.to_thread(
self._infer_sync, model_name, inputs, outputs
)
def _infer_sync(
self, model_name: str, inputs: dict, outputs: list[str] | None = None
) -> dict:
metadata_inputs = []
binary_chunks = []
for name, spec in inputs.items():
data = spec["data"]
shape = spec.get("shape", [])
datatype = spec.get("datatype", "FP32")
arr = np.asarray(data, dtype=np.float32)
binary = arr.tobytes()
metadata_inputs.append(
{
"name": name,
"shape": shape,
"datatype": datatype,
"parameters": {"binary_data_size": len(binary)},
}
)
binary_chunks.append(binary)
metadata_outputs = [{"name": o} for o in (outputs or [])]
metadata = {
"inputs": metadata_inputs,
"outputs": metadata_outputs,
}
metadata_bytes = json.dumps(metadata).encode("utf-8")
body = metadata_bytes + b"".join(binary_chunks)
headers = {
"Inference-Header-Content-Length": str(len(metadata_bytes)),
"Content-Type": "application/octet-stream",
}
url = f"{self.endpoint_url}/v2/models/{model_name}/infer"
response = self._session.post(url, data=body, headers=headers, timeout=self.timeout)
response.raise_for_status()
return self._parse_binary_response(response.headers, response.content)
@staticmethod
def _parse_binary_response(headers: dict, body: bytes) -> dict:
header_len = int(headers.get("Inference-Header-Content-Length", "0"))
metadata = json.loads(body[:header_len].decode("utf-8"))
result = {}
offset = 0
for output in metadata.get("outputs", []):
name = output["name"]
shape = output.get("shape", [])
params = output.get("parameters", {})
binary_size = params.get("binary_data_size", 0)
if binary_size > 0:
chunk = body[header_len + offset : header_len + offset + binary_size]
arr = np.frombuffer(chunk, dtype=np.float32).reshape(shape)
result[name] = arr.tolist()
offset += binary_size
return result
async def model_ready(self, model_name: str) -> bool:
return await asyncio.to_thread(self._model_ready_sync, model_name)
def _model_ready_sync(self, model_name: str) -> bool:
url = f"{self.endpoint_url}/v2/models/{model_name}"
response = self._session.get(url, timeout=self.timeout)
if response.status_code == 404:
return False
response.raise_for_status()
data = response.json()
return data.get("ready", False)
async def list_models(self) -> list[dict]:
return await asyncio.to_thread(self._list_models_sync)
# def _list_models_sync(self) -> list[dict]:
# url = f"{self.endpoint_url}/v2/models"
# response = requests.get(url, timeout=self.timeout)
# response.raise_for_status()
# data = response.json()
# return data.get("models", [])
def _list_models_sync(self) -> list[dict]:
# 1. Change the endpoint to Triton's repository index path
url = f"{self.endpoint_url}/v2/repository/index"
# 2. Change requests.get to requests.post with an empty json payload {}
response = self._session.post(url, json={}, timeout=self.timeout)
response.raise_for_status()
data = response.json()
# KServe v2 returns a list directly: [{"name": "model_a", "version": "1", "state": "READY"}]
return data

View File

@@ -0,0 +1,323 @@
import asyncio
import io
import base64
import uuid
import logging
import numpy as np
from datetime import datetime
from typing import Any
from data.spec.schemas import (
AnalysisJobSubmit, JobStatus, JobResult, PipelineStep,
StepEvent, ModelCatalog, ModelRegistrationResult,
HealthStatus,
)
from PIL import Image
from backend.implementation.preprocessing.clahe import apply_clahe
from backend.implementation.preprocessing.tensor_prep import (
prepare_angle_tensor,
prepare_inflammation_tensor,
prepare_segmentation_tensor,
)
from backend.implementation.postprocessing.measurement import calculate_thickness
from backend.implementation.postprocessing.severity import calculate_severity
from backend.implementation.postprocessing.overlay import create_overlay
from backend.implementation.postprocessing.calibration import (
calibration_config_from_params,
interpret_angle_logits,
interpret_inflammation_logits,
)
from backend.implementation.config import (
get_model_name,
get_segmentation_model,
get_angle_type,
TRITON_ENDPOINT,
)
from backend.implementation.adapters.triton_adapter import TritonAdapter
logger = logging.getLogger(__name__)
_job_registry: dict[str, dict] = {}
_job_lock = asyncio.Lock()
def _interpret_angle_result(result: dict, params: dict | None = None) -> dict:
logits = result.get("logits", [])
if not logits:
raise ValueError("Empty angle logits")
config = calibration_config_from_params(params)
return interpret_angle_logits(logits, config)
def _interpret_inflammation_result(result: dict, params: dict | None = None) -> dict:
logits = result.get("logits", [])
if not logits:
raise ValueError("Empty inflammation logits")
config = calibration_config_from_params(params)
return interpret_inflammation_logits(logits, config)
def _process_segmentation_result(result: dict, angle_class: str) -> tuple:
logits = result.get("logits", [])
if not logits:
raise ValueError("Empty segmentation logits")
logits_arr = np.array(logits)
if logits_arr.ndim < 3:
raise ValueError("Unexpected segmentation output shape")
preds = logits_arr.argmax(axis=1)[0]
angle_type = get_angle_type(angle_class)
if angle_type == "sup":
class_map = {
0: "background", 1: "effusion", 2: "fat", 3: "fat-pat",
4: "femur", 5: "synovium", 6: "tendon",
}
else:
class_map = {
0: "background", 1: "fat", 2: "tendon", 3: "muscle",
4: "femur", 5: "artery", 6: "baker's cyst",
}
masks = {}
for class_id, class_name in class_map.items():
masks[class_name] = (preds == class_id).astype(np.uint8)
return preds, masks
async def _get_triton_adapter() -> TritonAdapter:
return TritonAdapter(endpoint_url=TRITON_ENDPOINT)
def _encode_image_to_base64(image_pil: Image.Image) -> str:
buffered = io.BytesIO()
image_pil.save(buffered, format="PNG")
return base64.b64encode(buffered.getvalue()).decode()
async def _run_pipeline(image_pil: Image.Image, session_id: str, params: dict, model_versions: dict | None = None) -> dict:
enhanced_pil = apply_clahe(image_pil)
angle_tensor = prepare_angle_tensor(image_pil)
inflammation_tensor = prepare_inflammation_tensor(image_pil)
segmentation_tensor = prepare_segmentation_tensor(image_pil)
triton = await _get_triton_adapter()
angle_model = get_model_name("angle", model_versions)
angle_result = await triton.infer(
model_name=angle_model,
inputs={"input": {"data": angle_tensor.tolist(), "shape": list(angle_tensor.shape), "datatype": "FP32"}},
)
angle_interpreted = _interpret_angle_result(angle_result, params)
result = {
"angle": {
"class": angle_interpreted["class"],
"confidence": angle_interpreted["confidence"],
"calibration": angle_interpreted["calibration"],
},
"models_used": {"angle": angle_model},
}
if angle_interpreted["class"] in ("post-trans", "sup-up-long"):
inflam_model = get_model_name("inflammation", model_versions)
inflammation_result = await triton.infer(
model_name=inflam_model,
inputs={"input": {"data": inflammation_tensor.tolist(), "shape": list(inflammation_tensor.shape), "datatype": "FP32"}},
)
inflammation_interpreted = _interpret_inflammation_result(inflammation_result, params)
result["inflammation"] = {
"detected": inflammation_interpreted["detected"],
"confidence": inflammation_interpreted["confidence"],
"calibration": inflammation_interpreted["calibration"],
}
result["models_used"]["inflammation"] = inflam_model
if inflammation_interpreted["detected"]:
seg_model_name = get_segmentation_model(angle_interpreted["class"], model_versions)
seg_result = await triton.infer(
model_name=seg_model_name,
inputs={"input": {"data": segmentation_tensor.tolist(), "shape": list(segmentation_tensor.shape), "datatype": "FP32"}},
)
preds, masks = _process_segmentation_result(seg_result, angle_interpreted["class"])
angle_type = get_angle_type(angle_interpreted["class"])
measurement = calculate_thickness(masks, image_pil.size)
severity = calculate_severity(masks, image_pil.size)
segmented_overlay = create_overlay(image_pil, masks, measurement, angle_type)
result.update({
"measurement": measurement,
"severity": severity,
"segmentation": {
"performed": True,
"classes_detected": [k for k, v in masks.items() if np.sum(v) > 0],
"angle_type": angle_type,
},
"images": {
"enhanced": _encode_image_to_base64(enhanced_pil),
"segmented": _encode_image_to_base64(segmented_overlay),
},
})
result["models_used"]["segmentation"] = seg_model_name
else:
from backend.implementation.pipeline.cv_spec_pipeline import (
build_segmentation_skipped,
build_severity_zero,
)
result["segmentation"] = build_segmentation_skipped("no_inflammation")
result["severity"] = build_severity_zero("no_inflammation")
result["images"] = {"enhanced": _encode_image_to_base64(enhanced_pil)}
else:
from backend.implementation.pipeline.cv_spec_pipeline import (
build_segmentation_skipped,
build_severity_zero,
)
result["segmentation"] = build_segmentation_skipped("angle_only")
result["severity"] = build_severity_zero("angle_only")
result["images"] = {"enhanced": _encode_image_to_base64(enhanced_pil)}
return result
async def submit_sync(session_id: str, params: dict, model_versions: dict | None = None) -> JobResult:
if "local_image_path" in params:
image_pil = Image.open(params["local_image_path"]).convert("RGB")
elif "local_image_bytes" in params:
image_pil = Image.open(io.BytesIO(params["local_image_bytes"])).convert("RGB")
else:
from backend.implementation.session import service as session_service
frame_metadata = await session_service.get_frame(session_id, params.get("frame_id"))
raise NotImplementedError("S3 frame retrieval not yet integrated; use local_image_path for testing")
pipeline_result = await _run_pipeline(image_pil, session_id, params, model_versions)
job_id = str(uuid.uuid4())
return JobResult(
job_id=job_id,
session_id=session_id,
status="completed",
result=pipeline_result,
duration_ms=0,
)
async def submit_job(session_id: str, params: dict, model_versions: dict | None = None) -> str:
job_id = str(uuid.uuid4())
async with _job_lock:
_job_registry[job_id] = {
"session_id": session_id,
"params": params,
"model_versions": model_versions,
"status": "queued",
"result": None,
"steps": [],
"created_at": datetime.now(),
}
async def _background():
try:
await push_step_event(job_id, {
"step_id": str(uuid.uuid4()),
"job_id": job_id,
"event_type": "progress",
"task_type": "analysis",
"status": "running",
})
async with _job_lock:
_job_registry[job_id]["status"] = "running"
result = await submit_sync(session_id, params, model_versions)
async with _job_lock:
_job_registry[job_id].update({
"status": "completed",
"result": result.model_dump(),
})
await push_step_event(job_id, {
"step_id": str(uuid.uuid4()),
"job_id": job_id,
"event_type": "completed",
"task_type": "analysis",
"status": "completed",
"data": {"job_result": result.model_dump()},
})
except Exception as exc:
logger.exception(f"Job {job_id} failed")
async with _job_lock:
_job_registry[job_id].update({
"status": "failed",
"result": {"error": str(exc)},
})
await push_step_event(job_id, {
"step_id": str(uuid.uuid4()),
"job_id": job_id,
"event_type": "failed",
"task_type": "analysis",
"status": "failed",
"data": {"error": str(exc)},
})
asyncio.create_task(_background())
return job_id
async def job_status(job_id: str) -> JobStatus:
async with _job_lock:
job = _job_registry.get(job_id)
if not job:
raise LookupError(f"Job {job_id} not found")
return JobStatus(
job_id=job_id,
session_id=job["session_id"],
status=job["status"],
result=job.get("result"),
steps=job.get("steps", []),
created_at=job["created_at"],
updated_at=datetime.now(),
)
async def job_steps(job_id: str) -> list[PipelineStep]:
async with _job_lock:
job = _job_registry.get(job_id)
if not job:
raise LookupError(f"Job {job_id} not found")
return job.get("steps", [])
async def list_registered_models() -> ModelCatalog:
return ModelCatalog(models=[], total=0)
async def register_model(model_id: str, file: Any) -> ModelRegistrationResult:
raise NotImplementedError("Model registration not yet implemented")
async def health() -> HealthStatus:
try:
triton = await _get_triton_adapter()
models = await triton.list_models()
ready = any(m.get("state") == "READY" for m in models)
status = "ok" if ready else "degraded"
return HealthStatus(
status=status,
version="0.1.0",
dependencies={"triton": str(ready)},
uptime_seconds=0.0,
)
except Exception as exc:
logger.warning(f"Health check failed: {exc}")
return HealthStatus(status="error", version="0.1.0", dependencies={"triton": "False"}, uptime_seconds=0.0)
async def push_step_event(job_id: str, event: dict) -> None:
from backend.api.analysis_api import _event_queues, _queue_lock
async with _queue_lock:
if job_id not in _event_queues:
_event_queues[job_id] = asyncio.Queue()
step_event = StepEvent(
step_id=event.get("step_id", ""),
job_id=job_id,
event_type=event.get("event_type", "progress"),
task_type=event.get("task_type", ""),
status=event.get("status", "running"),
data=event.get("data"),
timestamp=datetime.now(),
)
await _event_queues[job_id].put(step_event)

View File

@@ -0,0 +1,21 @@
from data.spec.schemas import LoginRequest, Token, UserProfile, UserUpdateRequest
async def login(username: str, password: str) -> Token:
raise NotImplementedError("Auth service not yet implemented")
async def logout(token: str) -> None:
raise NotImplementedError("Auth service not yet implemented")
async def refresh(refresh_token: str) -> Token:
raise NotImplementedError("Auth service not yet implemented")
async def me(token: str) -> UserProfile:
raise NotImplementedError("Auth service not yet implemented")
async def update_me(token: str, updates: dict) -> UserProfile:
raise NotImplementedError("Auth service not yet implemented")

View File

@@ -0,0 +1,75 @@
import os
from pathlib import Path
from typing import Dict
SECRETS_DIR = Path(__file__).resolve().parent.parent.parent.parent.parent.parent / "secrets"
def _load_secret(name: str, filename: str) -> str:
file_path = SECRETS_DIR / filename
env_file = os.getenv(f"{name}_FILE")
if env_file:
resolved = Path(env_file)
if resolved.exists():
with open(resolved, "r", encoding="utf-8") as f:
return f.read().strip()
if file_path.exists():
with open(file_path, "r", encoding="utf-8") as f:
return f.read().strip()
raise RuntimeError(
f"Required secret {name} not found at {file_path} or via {name}_FILE env var"
)
# Endpoints (environment-provided, no hardcoded fallback for production)
MODAL_MEDGEMMA_ENDPOINT = os.getenv("MODAL_MEDGEMMA_ENDPOINT")
VERTEX_AI_GEMINI_ENDPOINT = os.getenv("VERTEX_AI_GEMINI_ENDPOINT")
# Secrets (must be present in PILOT_PROJECT/secrets or env)
GCP_ACCESS_TOKEN = _load_secret("GCP_ACCESS_TOKEN", "gcp_access_token.txt")
MEDGEMMA_API_KEY = _load_secret("MEDGEMMA_API_KEY", "modal_api_key.txt")
PROJECT_ID = os.getenv("VERTEX_AI_PROJECT", "vkist-project")
LOCATION = os.getenv("VERTEX_AI_LOCATION", "asia-southeast1")
TRITON_ENDPOINT = os.getenv("TRITON_ENDPOINT", "http://localhost:8000")
TEMP_DIR = os.getenv("TEMP_DIR", "/tmp/analysis_jobs")
# LLM Configuration
VERTEX_AI_PROJECT = os.getenv("VERTEX_AI_PROJECT", "vkist-project")
VERTEX_AI_LOCATION = os.getenv("VERTEX_AI_LOCATION", "asia-southeast1")
VERTEX_AI_MODEL = os.getenv("VERTEX_AI_MODEL", "medgemma")
# Redis Configuration
REDIS_HOST = os.getenv("REDIS_HOST", "localhost")
REDIS_PORT = int(os.getenv("REDIS_PORT", "6379"))
REDIS_DB = int(os.getenv("REDIS_DB", "0"))
DEFAULT_MODEL_VERSIONS = {
"angle": "angle_classify_convnext_tiny",
"inflammation": "inflammation_model_efficientnet_b0_ultrasound_2_cls",
"segmentation_sup": "segmentation_model_unet_resnet101",
"segmentation_post": "segmentation_model_post_deeplabv3_resnet101",
}
CLAHE_CLIP_LIMIT = float(os.getenv("CLAHE_CLIP_LIMIT", "2.0"))
CLAHE_TILE_SIZE = tuple(int(x) for x in os.getenv("CLAHE_TILE_SIZE", "8,8").split(","))
def get_model_name(task: str, model_versions: Dict[str, str] | None = None) -> str:
if model_versions and task in model_versions:
return model_versions[task]
return DEFAULT_MODEL_VERSIONS.get(task, task)
def get_angle_type(angle_class: str) -> str:
if angle_class in ("sup-trans-flex", "sup-up-long"):
return "sup"
if angle_class == "post-trans":
return "post"
return "other"
def get_segmentation_model(angle_class: str, model_versions: Dict[str, str] | None = None) -> str:
angle_type = get_angle_type(angle_class)
task = "segmentation_sup" if angle_type == "sup" else "segmentation_post"
return get_model_name(task, model_versions)

View File

@@ -0,0 +1,10 @@
from data.spec.schemas import IngestionRecord, RecordDetail
from typing import Any
async def list_records(user_id: str) -> list[IngestionRecord]:
raise NotImplementedError("Ingestion history service not yet implemented")
async def get_record(record_id: str) -> RecordDetail:
raise NotImplementedError("Ingestion history service not yet implemented")

View File

@@ -0,0 +1,13 @@
from data.spec.schemas import NotificationItem, NotificationPreferences
async def list_notifications(user_id: str, filters: dict | None = None) -> list[NotificationItem]:
raise NotImplementedError("Notification service not yet implemented")
async def mark_read(notification_id: str) -> None:
raise NotImplementedError("Notification service not yet implemented")
async def set_preferences(user_id: str, prefs: dict) -> None:
raise NotImplementedError("Notification service not yet implemented")

View File

@@ -0,0 +1,21 @@
from data.spec.schemas import Patient, PatientCreate, PatientListResponse
async def list_patients(user_id: str) -> list[Patient]:
raise NotImplementedError("Patient service not yet implemented")
async def create_patient(data: dict) -> Patient:
raise NotImplementedError("Patient service not yet implemented")
async def get_patient(patient_id: str) -> Patient:
raise NotImplementedError("Patient service not yet implemented")
async def list_sessions(patient_id: str) -> list[dict]:
raise NotImplementedError("Patient service not yet implemented")
async def ingestion_history(patient_id: str) -> list[dict]:
raise NotImplementedError("Patient service not yet implemented")

View File

@@ -0,0 +1,13 @@
"""CV inference orchestration (Sprint 12 spec)."""
from backend.implementation.pipeline.cv_spec_pipeline import (
BRANCH_ANGLE_CLASSES,
build_segmentation_skipped,
build_severity_zero,
)
__all__ = [
"BRANCH_ANGLE_CLASSES",
"build_segmentation_skipped",
"build_severity_zero",
]

View File

@@ -0,0 +1,35 @@
"""Shared CV pipeline helpers — Sprint 12 architecture spec §7."""
from __future__ import annotations
# Angles that may run inflammation → conditional segmentation.
BRANCH_ANGLE_CLASSES = frozenset({"post-trans", "sup-up-long"})
def build_severity_zero(reason: str) -> dict:
descriptions = {
"angle_only": "Góc quét không yêu cầu phân đoạn viêm",
"no_inflammation": "Không phát hiện viêm — bỏ qua phân đoạn",
}
return {
"level": 0,
"severity": "Rất nhẹ",
"color": "#28a745",
"description": descriptions.get(reason, "Không phân đoạn"),
"effusion": {"pixels": 0, "ratio": 0.0, "thickness": 0},
"synovium": {"pixels": 0, "ratio": 0.0},
"combined_score": 0.0,
"reason": reason,
}
def build_segmentation_skipped(reason: str) -> dict:
notes = {
"angle_only": "Chỉ phân loại góc — med-lat / sup-trans-flex",
"no_inflammation": "Không phát hiện viêm — bỏ qua phân đoạn",
}
return {
"performed": False,
"reason": reason,
"note": notes.get(reason, reason),
}

View File

@@ -0,0 +1,230 @@
"""Temperature-scaled softmax, entropy guardrails, and risk-first prediction payloads."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
import numpy as np
ANGLE_CLASSES = ["med-lat", "post-trans", "sup-trans-flex", "sup-up-long"]
INFLAMMATION_CLASSES = ["no_inflammation", "inflammation"]
CALIBRATION_TIERS = frozenset({"aggressive", "standard", "conservative"})
# Legacy API aliases
CALIBRATION_MODES = CALIBRATION_TIERS | frozenset({"screening", "diagnostic"})
TIER_RECOMMENDED_T = {
"aggressive": 0.7,
"standard": 1.4,
"conservative": 2.2,
# legacy → tier
"screening": 2.2,
"diagnostic": 1.4,
}
TIER_BOUNDARY_AGGRESSIVE_MAX = (0.7 + 1.4) / 2
TIER_BOUNDARY_STANDARD_MAX = (1.4 + 2.2) / 2
@dataclass
class CalibrationConfig:
"""User-adjustable calibration context (maps to UI mode / clinical prior)."""
temperature: float = 1.4
mode: str = "standard"
clinical_suspicion: float = 0.0
alpha_margin: float = 0.05
ood_entropy_threshold: float = 0.88
def __post_init__(self) -> None:
self.clinical_suspicion = float(np.clip(self.clinical_suspicion, 0.0, 1.0))
if self.mode not in CALIBRATION_TIERS:
if self.mode in ("screening",):
self.mode = "conservative"
elif self.mode in ("diagnostic",):
self.mode = "standard"
else:
self.mode = "standard"
if self.temperature <= 0:
self.temperature = 1.0
def logits_to_array(logits: Any) -> np.ndarray:
arr = np.asarray(logits, dtype=np.float32).reshape(-1)
if arr.size == 0:
raise ValueError("Empty logits")
return arr
def resolve_tier_from_temperature(temperature: float) -> str:
if temperature <= TIER_BOUNDARY_AGGRESSIVE_MAX:
return "aggressive"
if temperature <= TIER_BOUNDARY_STANDARD_MAX:
return "standard"
return "conservative"
def effective_temperature(config: CalibrationConfig) -> float:
if config.temperature > 0:
return max(0.25, float(config.temperature))
return TIER_RECOMMENDED_T.get(config.mode, 1.4)
def temperature_scaled_softmax(logits: np.ndarray, temperature: float) -> np.ndarray:
scaled = logits / max(temperature, 1e-6)
shifted = scaled - np.max(scaled)
exp = np.exp(shifted)
return exp / np.sum(exp)
def shannon_entropy(probs: np.ndarray) -> float:
safe = np.clip(probs, 1e-12, 1.0)
return float(-np.sum(safe * np.log(safe)))
def normalized_entropy(probs: np.ndarray) -> float:
if probs.size <= 1:
return 0.0
return shannon_entropy(probs) / float(np.log(probs.size))
def ambiguous_class_set(probs: np.ndarray, class_names: list[str], alpha_margin: float) -> list[str]:
idx_sorted = np.argsort(probs)[::-1]
max_prob = float(probs[idx_sorted[0]])
return [class_names[i] for i in idx_sorted if float(probs[i]) >= max_prob - alpha_margin]
def estimate_misclassification_rate(max_prob: float) -> float:
"""Placeholder empirical mapping until validation-set calibration bins are wired."""
if max_prob >= 0.95:
return 0.05
if max_prob >= 0.90:
return 0.08
if max_prob >= 0.85:
return 0.12
if max_prob >= 0.75:
return 0.18
if max_prob >= 0.65:
return 0.25
return 0.35
def _risk_framing_vi(
predicted_class: str,
class_names: list[str],
probs: np.ndarray,
decision_state: str,
error_rate: float,
ambiguous_set: list[str],
norm_entropy: float,
) -> str:
if decision_state == "ood_warning":
return (
"Mô hình chưa được huấn luyện với loại ảnh tương tự, nên kết quả AI có thể không đáng tin. "
"Hãy kiểm tra chất lượng ảnh và đối chiếu lâm sàng trước khi dựa vào nhãn tự động."
)
if decision_state == "ambiguous":
alt = ", ".join(c for c in ambiguous_set if c != predicted_class)
return (
f"Dự đoán chính: {predicted_class}. Tập mơ hồ (α): {', '.join(ambiguous_set)}"
+ (f" — các lựa chọn khả dĩ gồm {alt}." if alt else ".")
+ " Cần đối chiếu lâm sàng trước khi khóa kết quả."
)
return (
f"Dự đoán: {predicted_class}. "
f"Trong các ca có phân bố thống kê tương tự, tỷ lệ phân loại sai ước tính ~{error_rate * 100:.0f}% "
f"(entropy chuẩn hóa {norm_entropy:.2f})."
)
def decision_state_from(probs: np.ndarray, norm_entropy: float, config: CalibrationConfig) -> str:
if norm_entropy >= config.ood_entropy_threshold:
return "ood_warning"
ambiguous = ambiguous_class_set(probs, [str(i) for i in range(probs.size)], config.alpha_margin)
if len(ambiguous) > 1:
return "ambiguous"
return "confident"
def interpret_classification_logits(
logits: Any,
class_names: list[str],
config: CalibrationConfig | None = None,
) -> dict[str, Any]:
if len(class_names) == 0:
raise ValueError("class_names must not be empty")
cfg = config or CalibrationConfig()
arr = logits_to_array(logits)
if arr.size != len(class_names):
raise ValueError(f"Expected {len(class_names)} logits, got {arr.size}")
temperature = effective_temperature(cfg)
tier = resolve_tier_from_temperature(temperature)
probs = temperature_scaled_softmax(arr, temperature)
pred_idx = int(np.argmax(probs))
predicted_class = class_names[pred_idx]
max_prob = float(probs[pred_idx])
entropy = shannon_entropy(probs)
norm_entropy = normalized_entropy(probs)
ambiguous = ambiguous_class_set(probs, class_names, cfg.alpha_margin)
state = decision_state_from(probs, norm_entropy, cfg)
error_rate = estimate_misclassification_rate(max_prob)
risk_vi = _risk_framing_vi(
predicted_class,
class_names,
probs,
state,
error_rate,
ambiguous,
norm_entropy,
)
class_probabilities = {
name: round(float(probs[i]) * 100, 2) for i, name in enumerate(class_names)
}
return {
"class": predicted_class,
"confidence": round(max_prob * 100, 2),
"calibration": {
"raw_logits": [round(float(x), 6) for x in arr.tolist()],
"temperature": round(temperature, 4),
"base_temperature": cfg.temperature,
"mode": tier,
"clinical_suspicion": round(cfg.clinical_suspicion, 3),
"alpha_margin": cfg.alpha_margin,
"class_probabilities": class_probabilities,
"entropy": round(entropy, 4),
"normalized_entropy": round(norm_entropy, 4),
"ambiguous_set": ambiguous,
"decision_state": state,
"predicted_error_rate": round(error_rate, 4),
"risk_framing_vi": risk_vi,
},
}
def interpret_angle_logits(logits: Any, config: CalibrationConfig | None = None) -> dict[str, Any]:
return interpret_classification_logits(logits, ANGLE_CLASSES, config)
def interpret_inflammation_logits(logits: Any, config: CalibrationConfig | None = None) -> dict[str, Any]:
payload = interpret_classification_logits(logits, INFLAMMATION_CLASSES, config)
detected = payload["class"] == "inflammation"
payload["detected"] = detected
return payload
def calibration_config_from_params(params: dict[str, Any] | None) -> CalibrationConfig:
if not params:
return CalibrationConfig()
calibration = params.get("calibration") or {}
return CalibrationConfig(
temperature=float(calibration.get("temperature", 1.0)),
mode=str(calibration.get("mode", "standard")),
clinical_suspicion=float(calibration.get("clinical_suspicion", 0.0)),
alpha_margin=float(calibration.get("alpha_margin", 0.05)),
ood_entropy_threshold=float(calibration.get("ood_entropy_threshold", 0.88)),
)

View File

@@ -0,0 +1,123 @@
__all__ = ["calculate_thickness", "get_mask_bounding_box", "find_max_continuous_segment"]
import numpy as np
import cv2
SEGMENT_CLASSES_SUPRAPAT = {
0: "background", 1: "effusion", 2: "fat", 3: "fat-pat",
4: "femur", 5: "synovium", 6: "tendon"
}
SEGMENT_CLASSES_POST = {
0: "background", 1: "fat", 2: "tendon", 3: "muscle",
4: "femur", 5: "artery", 6: "baker's cyst"
}
PIXEL_TO_MM = 45.0 / 655.0
def get_mask_bounding_box(mask, dist_percent: float = 0.01):
if mask is None or np.sum(mask) == 0:
return None
mask_uint8 = mask.astype(np.uint8)
if np.max(mask_uint8) == 1:
mask_uint8 *= 255
img_width = mask_uint8.shape[1]
dist_threshold = img_width * dist_percent
kernel = np.ones((5, 5), np.uint8)
clean_mask = cv2.morphologyEx(mask_uint8, cv2.MORPH_OPEN, kernel)
contours, _ = cv2.findContours(clean_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
if not contours:
return None
contour_info = sorted(
[{"cnt": cnt, "area": cv2.contourArea(cnt)} for cnt in contours],
key=lambda x: x["area"], reverse=True,
)
main_block = contour_info[0]
max_area = main_block["area"]
if max_area < 50:
return None
main_mask = np.zeros_like(mask_uint8)
cv2.drawContours(main_mask, [main_block["cnt"]], -1, 255, -1)
dist_map = cv2.distanceTransform(255 - main_mask, cv2.DIST_L2, 3)
significant_contours = [main_block["cnt"]]
area_threshold = max_area / 4.0
for i in range(1, len(contour_info)):
other = contour_info[i]
other_mask = np.zeros_like(mask_uint8)
cv2.drawContours(other_mask, [other["cnt"]], -1, 255, -1)
min_dist = np.min(dist_map[other_mask > 0])
if other["area"] >= area_threshold or min_dist <= dist_threshold:
significant_contours.append(other["cnt"])
all_points = np.concatenate(significant_contours)
x, y, w, h = cv2.boundingRect(all_points)
return x, y, w, h
def find_max_continuous_segment(col_array):
padded = np.concatenate(([0], col_array, [0]))
diffs = np.diff(padded)
starts = np.where(diffs == 1)[0]
ends = np.where(diffs == -1)[0]
if len(starts) == 0:
return 0, -1, -1
lengths = ends - starts
max_idx = int(np.argmax(lengths))
max_len = int(lengths[max_idx])
return max_len, int(starts[max_idx]), int(ends[max_idx])
def calculate_thickness(masks: dict, image_size, measure_ids=None):
if measure_ids is None:
measure_ids = [1, 5]
width, height = image_size
mask_all_labels = np.zeros((height, width), dtype=np.uint8)
mask_measure = np.zeros((height, width), dtype=np.uint8)
has_any_label = False
if "fat-pat" in masks:
class_map = SEGMENT_CLASSES_SUPRAPAT
else:
class_map = SEGMENT_CLASSES_POST
for class_id, class_name in class_map.items():
if class_name not in masks or class_name == "background":
continue
mask = masks[class_name]
if np.sum(mask) > 0:
has_any_label = True
mask_all_labels = np.logical_or(mask_all_labels, mask).astype(np.uint8)
if class_id in measure_ids:
mask_measure = np.logical_or(mask_measure, mask).astype(np.uint8)
if not has_any_label or np.sum(mask_measure) == 0:
return None
bbox_all = get_mask_bounding_box(mask_all_labels)
if bbox_all is None:
return None
x_all, y_all, w_all, h_all = bbox_all
roi_start = x_all + (w_all // 3)
roi_end = x_all + (2 * w_all // 3)
roi_strip = mask_measure[:, roi_start:roi_end]
global_max_len_px = 0
best_x_rel = 0
best_y_start = 0
best_y_end = 0
for x in range(roi_strip.shape[1]):
col = roi_strip[:, x]
if not np.any(col):
continue
length, y_s, y_e = find_max_continuous_segment(col)
if length > global_max_len_px:
global_max_len_px = length
best_x_rel = x
best_y_start = y_s
best_y_end = y_e
if global_max_len_px == 0:
return None
thickness_mm = global_max_len_px * PIXEL_TO_MM
real_x = roi_start + best_x_rel
return {
"thickness_px": int(global_max_len_px),
"thickness_mm": float(round(thickness_mm, 2)),
"x": int(real_x),
"y_start": int(best_y_start),
"y_end": int(best_y_end),
"roi_start": int(roi_start),
"roi_end": int(roi_end),
"bbox": {"x": int(x_all), "y": int(y_all), "w": int(w_all), "h": int(h_all)},
}

View File

@@ -0,0 +1,89 @@
__all__ = ["create_overlay"]
from PIL import Image, ImageDraw
import numpy as np
import cv2
COLOR_MAP_SUP = {
"background": [0, 0, 0],
"effusion": [255, 0, 0],
"fat": [255, 255, 0],
"fat-pat": [0, 255, 255],
"femur": [0, 255, 0],
"synovium": [255, 0, 255],
"tendon": [0, 0, 255],
}
COLOR_MAP_POST = {
"background": [0, 0, 0],
"baker's cyst": [255, 0, 0],
"fat": [255, 255, 0],
"muscle": [0, 255, 255],
"femur": [0, 255, 0],
"artery": [255, 0, 255],
"synovium": [255, 0, 255],
"tendon": [0, 0, 255],
}
def create_overlay(image_pil: Image.Image, masks: dict, measurement, angle_type: str = "sup") -> Image.Image:
if masks is None:
return image_pil
color_map = COLOR_MAP_SUP if angle_type == "sup" else COLOR_MAP_POST
img_array = np.array(image_pil)
overlay = img_array.copy()
for class_name, mask in masks.items():
if class_name in color_map and np.sum(mask) > 0:
color = color_map[class_name]
for i in range(3):
overlay[:, :, i] = np.where(
mask > 0,
(overlay[:, :, i] * 0.6 + color[i] * 0.4).astype(np.uint8),
overlay[:, :, i],
)
overlay_pil = Image.fromarray(overlay)
draw = ImageDraw.Draw(overlay_pil)
for class_name in ["effusion", "synovium"]:
mask = masks.get(class_name)
if mask is not None and np.sum(mask) > 0:
mask_uint8 = (mask * 255).astype(np.uint8)
contours, _ = cv2.findContours(mask_uint8, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for contour in contours:
points = contour.reshape(-1, 2).tolist()
if len(points) > 2:
points = [(int(p[0]), int(p[1])) for p in points]
draw.line(points + [points[0]], fill=(255, 255, 255), width=3)
if measurement and angle_type == "sup":
x = measurement["x"]
y_start = measurement["y_start"]
y_end = measurement["y_end"]
thickness_mm = measurement["thickness_mm"]
roi_start = measurement["roi_start"]
roi_end = measurement["roi_end"]
bbox = measurement["bbox"]
draw.rectangle(
[bbox["x"], bbox["y"], bbox["x"] + bbox["w"], bbox["y"] + bbox["h"]],
outline=(0, 255, 0), width=3,
)
h = image_pil.size[1]
draw.line([(roi_start, 0), (roi_start, h)], fill=(0, 255, 255), width=2)
draw.line([(roi_end, 0), (roi_end, h)], fill=(0, 255, 255), width=2)
draw.line([(x, y_start), (x, y_end)], fill=(255, 0, 0), width=4)
radius = 4
draw.ellipse([x - radius, y_start - radius, x + radius, y_start + radius],
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
draw.ellipse([x - radius, y_end - radius, x + radius, y_end + radius],
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
text = f"{thickness_mm:.2f} mm"
try:
from PIL import ImageFont
font = ImageFont.load_default()
bbox_text = draw.textbbox((0, 0), text, font=font)
text_w = bbox_text[2] - bbox_text[0]
text_h = bbox_text[3] - bbox_text[1]
except Exception:
text_w, text_h = 100, 20
text_x = x + 8
text_y = y_start - text_h - 8
draw.rectangle([text_x - 2, text_y - 2, text_x + text_w + 2, text_y + text_h + 2], fill=(0, 0, 0))
draw.text((text_x, text_y), text, fill=(255, 0, 0))
return overlay_pil

View File

@@ -0,0 +1,64 @@
__all__ = ["calculate_severity"]
import numpy as np
SEVERITY_LEVELS = [
(15, 3, "Nặng", "#dc3545", "Dịch khớp dày, màng hoạt dịch tăng sinh rõ"),
(8, 2, "Trung bình", "#fd7e14", "Dịch khớp trung bình, màng hoạt dịch tăng sinh vừa"),
(3, 1, "Nhẹ", "#ffc107", "Dịch khớp mỏng, màng hoạt dịch tăng sinh nhẹ"),
(0, 0, "Rất nhẹ", "#28a745", "Lượng dịch và màng hoạt dịch trong giới hạn bình thường"),
]
def calculate_severity(masks: dict, image_size) -> dict | None:
if not masks:
return None
width, height = image_size
total_pixels = width * height
effusion_mask = masks.get("effusion", np.zeros((height, width), dtype=np.uint8))
effusion_pixels = int(np.sum(effusion_mask))
effusion_ratio = (effusion_pixels / total_pixels) * 100
effusion_thickness = 0
if effusion_pixels > 0:
rows_with_effusion = np.any(effusion_mask > 0, axis=1)
if np.any(rows_with_effusion):
effusion_thickness = int(np.sum(rows_with_effusion))
synovium_mask = masks.get("synovium", np.zeros((height, width), dtype=np.uint8))
synovium_pixels = int(np.sum(synovium_mask))
synovium_ratio = (synovium_pixels / total_pixels) * 100
effusion_score = min(effusion_thickness / height * 100, 100)
synovium_score = synovium_ratio
combined_score = effusion_score * 0.6 + synovium_score * 0.4
for threshold, level, severity, color, description in SEVERITY_LEVELS:
if combined_score > threshold:
return {
"level": int(level),
"severity": severity,
"color": color,
"description": description,
"effusion": {
"pixels": effusion_pixels,
"ratio": float(round(effusion_ratio, 2)),
"thickness": effusion_thickness,
},
"synovium": {
"pixels": synovium_pixels,
"ratio": float(round(synovium_ratio, 2)),
},
"combined_score": float(round(combined_score, 2)),
}
return {
"level": 0,
"severity": "Rất nhẹ",
"color": "#28a745",
"description": "Lượng dịch và màng hoạt dịch trong giới hạn bình thường",
"effusion": {
"pixels": effusion_pixels,
"ratio": float(round(effusion_ratio, 2)),
"thickness": effusion_thickness,
},
"synovium": {
"pixels": synovium_pixels,
"ratio": float(round(synovium_ratio, 2)),
},
"combined_score": float(round(combined_score, 2)),
}

View File

@@ -0,0 +1,13 @@
__all__ = ["apply_clahe"]
import cv2
import numpy as np
from PIL import Image
def apply_clahe(image_pil: Image.Image, clip_limit: float = 2.0, tile_grid_size: tuple[int, int] = (8, 8)) -> Image.Image:
img_array = np.array(image_pil)
gray = cv2.cvtColor(img_array, cv2.COLOR_RGB2GRAY)
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=tile_grid_size)
enhanced_gray = clahe.apply(gray)
enhanced_rgb = cv2.cvtColor(enhanced_gray, cv2.COLOR_GRAY2RGB)
return Image.fromarray(enhanced_rgb)

View File

@@ -0,0 +1,35 @@
__all__ = ["prepare_angle_tensor", "prepare_inflammation_tensor", "prepare_segmentation_tensor"]
import numpy as np
from PIL import Image
from .transforms import Resize, Normalize
ANGLE_TRANSFORM = Resize((224, 224))
ANGLE_NORMALIZE = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
INFLAMMATION_TRANSFORM = Resize((224, 224))
INFLAMMATION_NORMALIZE = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
SEGMENTATION_TRANSFORM = Resize((512, 512))
def _to_nchw(arr_hwc: np.ndarray) -> np.ndarray:
arr = arr_hwc.transpose(2, 0, 1)
return np.expand_dims(arr, axis=0)
def prepare_angle_tensor(image_pil: Image.Image) -> np.ndarray:
img = ANGLE_TRANSFORM(image_pil)
arr = ANGLE_NORMALIZE(img)
return _to_nchw(arr)
def prepare_inflammation_tensor(image_pil: Image.Image) -> np.ndarray:
img = INFLAMMATION_TRANSFORM(image_pil)
arr = INFLAMMATION_NORMALIZE(img)
return _to_nchw(arr)
def prepare_segmentation_tensor(image_pil: Image.Image) -> np.ndarray:
img = SEGMENTATION_TRANSFORM(image_pil)
arr = np.asarray(img).astype(np.float32) / 255.0
return _to_nchw(arr)

View File

@@ -0,0 +1,22 @@
__all__ = ["Resize", "Normalize"]
from PIL import Image
import numpy as np
class Resize:
def __init__(self, size: tuple[int, int]):
self.size = size
def __call__(self, image: Image.Image) -> Image.Image:
return image.resize(self.size, Image.Resampling.BILINEAR)
class Normalize:
def __init__(self, mean: list[float], std: list[float]):
self.mean = np.array(mean, dtype=np.float32)
self.std = np.array(std, dtype=np.float32)
def __call__(self, image_pil: Image.Image) -> np.ndarray:
arr = np.asarray(image_pil).astype(np.float32) / 255.0
arr = (arr - self.mean) / self.std
return arr

View File

@@ -0,0 +1,148 @@
from typing import Any, AsyncGenerator
import logging
from fastapi import HTTPException, status
from data.spec.schemas import (
HeatmapResult, RationaleResult, ChatResponse, DriftCheckResult,
EvidenceList, ActivationMeta, AnnotationArtifact, EscalationTicket,
GuardrailResult, CorrectionRecord,
)
from backend.implementation.adapters.llm_adapter import get_llm_adapter
from backend.implementation.adapters.bert_adapter import get_bert_adapter
from backend.implementation.adapters.redis_adapter import get_redis_client
logger = logging.getLogger(__name__)
llm_adapter = get_llm_adapter()
bert_adapter = get_bert_adapter()
redis_client = get_redis_client()
async def _verify_pre_egress(session_id: str, redaction_hash: str | None = None):
"""
Enforce NFR-16a Pre-Egress Checklist.
"""
# 1. Consent Verification
consent_key = f"consent:{session_id}"
if not redis_client.exists(consent_key):
logger.error(f"Consent missing for session {session_id}")
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="User consent for cloud LLM egress is required."
)
# 2. Redaction Verification (if hash provided)
if redaction_hash:
# In real impl: Run Presidio on the prompt and compare hashes
# For now, we assume a simple check or stub
if redaction_hash == "FAIL_HASH":
logger.error(f"Redaction hash mismatch for session {session_id}")
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Redaction verification failed. PHI may be present."
)
# Note: Audit Log commit is handled via the LLM adapter's AuditCallbackHandler
# to ensure it happens exactly before the call.
async def gradcam(session_id: str) -> HeatmapResult:
raise NotImplementedError("Safety service not yet implemented")
async def rationale(session_id: str, redaction_hash: str | None = None) -> RationaleResult:
# Pre-egress check
await _verify_pre_egress(session_id, redaction_hash)
# 1. Fetch session context (simplified for stub)
context = {"grade": "moderate", "joint_site": "wrist"}
# 2. Construct prompt
prompt = f"Based on MOH guidelines, explain the synovitis grade {context['grade']} for {context['joint_site']}..."
# 3. Call LLM adapter
text = await llm_adapter.generate(prompt, session_id)
redis_client.set(f"consult_mode:{session_id}", "tier_3")
return RationaleResult(text=text)
async def circuit_break(session_id: str, flag: bool) -> None:
if flag:
logger.warning(f"Circuit breaker triggered for session {session_id}")
return None
async def socratic_chat(session_id: str, prompt: str, redaction_hash: str | None = None) -> ChatResponse:
# Pre-egress check
await _verify_pre_egress(session_id, redaction_hash)
# 1. Retrieve conversation history (stub)
history = []
# 2. Construct prompt
full_prompt = f"History: {history}\nUser: {prompt}\nAssistant: "
# 3. Call LLM adapter
response_text = await llm_adapter.generate(full_prompt, session_id)
# 4. BERT Referee check (stub)
is_grounded = bert_adapter.referee_check(response_text, [])
if not is_grounded:
response_text = "I'm sorry, I couldn't verify this answer against the guidelines."
# Post-egress: update consult mode
redis_client.set(f"consult_mode:{session_id}", "tier_3")
return ChatResponse(response=response_text)
async def drift_check(session_id: str) -> DriftCheckResult:
res = bert_adapter.drift_check("mock clinical text")
return DriftCheckResult(score=res.score, is_drifted=res.is_drifted)
async def rag_evidence(session_id: str) -> EvidenceList:
raise NotImplementedError("Safety service not yet implemented")
async def activations(session_id: str, params: dict) -> ActivationMeta:
raise NotImplementedError("Safety service not yet implemented")
async def upload_artifact(session_id: str, file: Any) -> AnnotationArtifact:
raise NotImplementedError("Safety service not yet implemented")
async def ground_truth(session_id: str, label: dict) -> None:
raise NotImplementedError("Safety service not yet implemented")
async def escalate(session_id: str, reason: str) -> EscalationTicket:
raise NotImplementedError("Safety service not yet implemented")
async def morphology(session_id: str, annotation: dict) -> None:
raise NotImplementedError("Safety service not yet implemented")
async def guardrail_check(session_id: str, prompt: str, score: float) -> GuardrailResult:
res = bert_adapter.guardrail_check(prompt)
return GuardrailResult(verdict=res.verdict, reason=res.reason)
async def submit_correction(session_id: str, correction: dict) -> CorrectionRecord:
raise NotImplementedError("Safety correction service not yet implemented")
async def chat_stream(session_id: str, prompt: str, redaction_hash: str | None = None) -> AsyncGenerator[str, None]:
# Pre-egress check
await _verify_pre_egress(session_id, redaction_hash)
async for chunk in llm_adapter.stream_generate(prompt, session_id):
res = bert_adapter.guardrail_check(chunk)
if res.verdict == "MITIGATE":
yield "[Content Filtered]"
return
yield chunk
# Post-egress
redis_client.set(f"consult_mode:{session_id}", "tier_3")

View File

@@ -0,0 +1,34 @@
from datetime import datetime
from data.spec.schemas import (
Session, SessionCreate, SessionDetail, SessionPatchReview,
FrameMetadata, PersistResult, ExportResult, ScrubResult,
)
from typing import Any
async def create_session(user_id: str, patient_id: str, case_id: str | None = None) -> Session:
raise NotImplementedError("Session service not yet implemented")
async def get_session(session_id: str) -> SessionDetail:
raise NotImplementedError("Session service not yet implemented")
async def add_frame(session_id: str, file: Any, frame_number: int | None = None) -> FrameMetadata:
raise NotImplementedError("Session service not yet implemented")
async def patch_review(session_id: str, review: dict) -> Session:
raise NotImplementedError("Session service not yet implemented")
async def persist(session_id: str, review: dict) -> PersistResult:
raise NotImplementedError("Session service not yet implemented")
async def export_pdf(session_id: str, params: dict) -> ExportResult:
raise NotImplementedError("Session service not yet implemented")
async def scrub_validate(session_id: str, metadata: dict) -> ScrubResult:
raise NotImplementedError("Session service not yet implemented")

View File

@@ -0,0 +1,9 @@
from data.spec.schemas import UserSettings, SettingsUpdate
async def get_settings(user_id: str) -> UserSettings:
raise NotImplementedError("Settings service not yet implemented")
async def update_settings(user_id: str, updates: dict) -> UserSettings:
raise NotImplementedError("Settings service not yet implemented")

View File

@@ -0,0 +1,50 @@
import asyncio
import base64
import io
import logging
from typing import Any
from PIL import Image
from backend.implementation.postprocessing.calibration import calibration_config_from_params
from backend.services.cv_inference_service import CvInferenceOptions, run_single
from backend.services.celery_app import celery_app
logger = logging.getLogger(__name__)
def _decode_base64_to_pil(b64_str: str) -> Image.Image:
raw = base64.b64decode(b64_str)
return Image.open(io.BytesIO(raw)).convert("RGB")
@celery_app.task(bind=True, name="cv_inference.run_chunk", max_retries=2, default_retry_delay=10)
def run_cv_chunk(self, chunk_payload: dict) -> list[dict]:
images_b64 = chunk_payload.get("images_b64", [])
frame_ids = chunk_payload.get("frame_ids", [])
calibration_params = chunk_payload.get("calibration", {})
model_versions = chunk_payload.get("model_versions")
if not images_b64:
return []
images = [_decode_base64_to_pil(b64) for b64 in images_b64]
calibration = calibration_config_from_params(calibration_params)
options = CvInferenceOptions(
calibration=calibration,
model_versions=model_versions,
use_cache=False,
)
async def _run():
return await asyncio.gather(*[
run_single(img, frame_id=fid, options=options)
for img, fid in zip(images, frame_ids)
])
try:
results = asyncio.run(_run())
return results
except Exception as exc:
logger.exception("Celery chunk task failed: %s", exc)
raise self.retry(exc=exc, countdown=10, max_retries=2)

View File

@@ -0,0 +1,6 @@
from data.spec.schemas import AnomalyRecord
from typing import Any
async def report_anomaly(session_id: str, data: dict) -> AnomalyRecord:
raise NotImplementedError("Telemetry service not yet implemented")

View File

@@ -0,0 +1,27 @@
"""Triton batching helpers — aligned with config.pbtxt max_batch_size: 8."""
from __future__ import annotations
import os
from collections.abc import Iterator, Sequence
from typing import TypeVar
T = TypeVar("T")
TRITON_MAX_BATCH_SIZE = int(os.getenv("TRITON_MAX_BATCH_SIZE", "8"))
def chunk_sequence(items: Sequence[T], batch_size: int | None = None) -> Iterator[list[T]]:
"""Split a sequence into chunks of at most ``batch_size`` (default: TRITON_MAX_BATCH_SIZE)."""
size = batch_size if batch_size is not None else TRITON_MAX_BATCH_SIZE
if size < 1:
raise ValueError(f"batch_size must be >= 1, got {size}")
for start in range(0, len(items), size):
yield list(items[start : start + size])
def batch_count(item_count: int, batch_size: int | None = None) -> int:
"""Number of Triton infer calls needed (e.g. 10 images -> 2 batches when size=8)."""
if item_count <= 0:
return 0
size = batch_size if batch_size is not None else TRITON_MAX_BATCH_SIZE
return (item_count + size - 1) // size

View File

@@ -0,0 +1,29 @@
import logging
import sys
from logging.handlers import RotatingFileHandler
from pathlib import Path
LOG_DIR = Path("logs")
LOG_DIR.mkdir(exist_ok=True)
LOG_FORMAT = "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s"
LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
def setup_logging():
root = logging.getLogger()
root.setLevel(logging.DEBUG)
root.handlers.clear()
formatter = logging.Formatter(LOG_FORMAT, datefmt=LOG_DATE_FORMAT)
console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.INFO)
console.setFormatter(formatter)
root.addHandler(console)
file_handler = RotatingFileHandler(
LOG_DIR / "app.log",
maxBytes=10 * 1024 * 1024,
backupCount=5,
encoding="utf-8",
)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
root.addHandler(file_handler)
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("uvicorn.access").setLevel(logging.INFO)

View File

@@ -0,0 +1,89 @@
import logging
import os
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.exceptions import RequestValidationError
from starlette.exceptions import HTTPException as StarletteHTTPException
from backend.api import (
auth_api,
patient_api,
session_api,
analysis_api,
safety_api,
notification_api,
settings_api,
ingestion_api,
telemetry_api,
)
from backend.routers import cloud_orchestrate, cloud_consult, agent_tools
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
logger.info("Starting medical imaging AI platform API")
yield
logger.info("Shutting down medical imaging AI platform API")
app = FastAPI(
title="Medical Imaging & AI Safety Platform",
description="Clinical diagnostic imaging platform with AI safety analysis",
version="0.1.0",
lifespan=lifespan,
)
app.add_middleware(
CORSMiddleware,
allow_origins=os.getenv(
"CORS_ORIGINS",
"http://localhost:3000,http://localhost:5173,http://localhost:5174,http://localhost:4173",
).split(","),
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request, exc):
from fastapi.responses import JSONResponse
from data.spec.schemas import ErrorResponse
return JSONResponse(
status_code=422,
content=ErrorResponse(detail=str(exc), code="VALIDATION_ERROR").model_dump(),
)
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request, exc):
from fastapi.responses import JSONResponse
from data.spec.schemas import ErrorResponse
content = ErrorResponse(detail=exc.detail, code="HTTP_ERROR").model_dump()
return JSONResponse(status_code=exc.status_code, content=content)
@app.exception_handler(NotImplementedError)
async def not_implemented_handler(request, exc):
from fastapi.responses import JSONResponse
from data.spec.schemas import ErrorResponse
content = ErrorResponse(detail=str(exc), code="NOT_IMPLEMENTED").model_dump()
return JSONResponse(status_code=501, content=content)
app.include_router(cloud_orchestrate.router)
app.include_router(cloud_consult.router)
app.include_router(agent_tools.router)
app.include_router(auth_api.router)
app.include_router(patient_api.router)
app.include_router(session_api.router)
app.include_router(analysis_api.router)
app.include_router(safety_api.router)
app.include_router(notification_api.router)
app.include_router(settings_api.router)
app.include_router(ingestion_api.router)
app.include_router(telemetry_api.router)

Some files were not shown because too many files have changed in this diff Show More