update
This commit is contained in:
DatTT127
12
README.md
12
README.md
@@ -17,21 +17,19 @@ This repository contains the research, design, and implementation materials for
|
||||
PILOT_PROJECT/
|
||||
├── .gitignore # Git ignore rules (includes OS/editor temp files, dependencies, and `secrets/` folder)
|
||||
├── AGENT_SKILL # Skills file for the agentic automation system
|
||||
├── Reading_docs/ # Reference and background materials
|
||||
├── PROJ_LEVEL_READING/ # reading document on prj level (project-vision, dev-plan, user-analysis)
|
||||
│ ├── PLAN/ # Project plans and timelines
|
||||
│ ├── Requirement_Analysis/ # Stakeholder and functional requirements
|
||||
│ ├── Technical_Brainstorming/ # Brainstorming notes, sketches, whitepapers
|
||||
│ └── User_Analysis/ # User personas, workflows, and usability research
|
||||
├── workspace/ # Primary working area for sprints and development
|
||||
│ ├── sprint_1_2/ # Example sprint folder
|
||||
│ │ ├── CAVEAT_TASK.md # Known limitations and caveats
|
||||
│ │ ├── CONTEXT.md # Sprint context and goals
|
||||
│ │ ├── DESIGN_MATERIAL/ # UI/UX mockups, wireframes, style guides
|
||||
│ │ ├── docs/ # Generated or supplemental documentation
|
||||
│ │ ├── PROJECT_VIS.md # Project visualization and architecture diagrams
|
||||
│ │ ├── LEGACY # The legacy material - for archieved / pre-existed project
|
||||
│ │ ├── CODEBASE # The project codebase
|
||||
│ │ ├── Design_Material/ # The system design - API - and UIX design material
|
||||
│ │ ├── SOFTWARE_SYSTEM_DESIGN_FR_25.md # Detailed software design specification
|
||||
│ │ ├── SOLUTION_ARCHITECTURE_SPEC.md # Solution architecture overview
|
||||
│ │ └── visualization/ # Charts, diagrams, and visual assets
|
||||
│ │ └── VISUALIZATION/ # Charts, diagrams, and visual assets
|
||||
│ └── ... # Additional sprint folders as needed
|
||||
├── secrets/ # **Developer‑managed secrets** (NOT tracked by git)
|
||||
│ │ # Each developer should maintain their own copy of this folder
|
||||
|
||||
579
proj_level_reading/ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md
Normal file
579
proj_level_reading/ARCHITECT/SOFTWARE_ARCHITECTURE_SPEC.md
Normal file
@@ -0,0 +1,579 @@
|
||||
# Software Architecture Specification — VKIST MSK Platform
|
||||
|
||||
**Scope:** Sprint 1-2 (FR-25 Synovitis Grading + Multi-Modal NLP Integration)
|
||||
**Parent:** [SOLUTION_ARCHITECTURE_SPEC.md](./SOLUTION_ARCHITECTURE_SPEC.md), [SPRINT_1_2_ARCHITECTURE_SPEC.md](../workspace/sprint_1_2/SPRINT_1_2_ARCHITECTURE_SPEC.md)
|
||||
**Workflow:** Understand → Model (C4) → Specify → Decompose → Plan
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
Build a reproducible, air-gapped-first musculoskeletal ultrasound analysis platform that performs automated synovitis grading (FR-25) with Vietnamese-language NLP explanations, auditable RAG citations, and HITL finalization — deployable on a single-hospital K3s cluster under ≤10 Mbps LAN constraints, with ≤150 MB idle app bundle and ≤1.5 s inference latency.
|
||||
|
||||
---
|
||||
|
||||
## 2. Requirements
|
||||
|
||||
### Functional (Sprint 1-2)
|
||||
- [ ] FR-25: Load knee DICOM → segment joint structures → measure synovium thickness → grade synovitis 0-3
|
||||
- [ ] Grad-CAM overlay on primary viewport (zero extra clicks)
|
||||
- [ ] Circuit-breaker Socratic dialogue (radiologist challenges AI grade before finalizing)
|
||||
- [ ] BERT drift monitor against baseline MOH corpus
|
||||
- [ ] RAG-Referee validates every LLM-generated explanation against top-k retrieved MOH guideline chunks
|
||||
- [ ] Decree 13 PII scrubbing on all outbound text (client-side + FastAPI middleware)
|
||||
- [ ] ladybugDB ontology traversal for anatomical entity disambiguation
|
||||
- [ ] GemmaE2B/MedGemma Vietnamese LLM consult (browser WebLLM local OR cloud Vertex AI) with MOH guideline citations
|
||||
- [ ] Circular 46/2018 PDF report generation
|
||||
- [ ] Immutable audit log append (NFR-17) and HITL digital signature gate (NFR-19)
|
||||
|
||||
### Non-Functional (Critical)
|
||||
- [ ] NFR-4: 150 MB idle app bundle
|
||||
- [ ] NFR-5: ≤1.5 s inference (on-prem Triton)
|
||||
- [ ] NFR-7: ≤200 ms TTFT token streaming
|
||||
- [ ] NFR-8: Fault-tolerant across Wi-Fi drops (state preserved)
|
||||
- [ ] NFR-14: No client-side GPU/neural accelerator required
|
||||
- [ ] NFR-15: Circular 46 EMR compliance
|
||||
- [ ] NFR-16: Air-gapped primary; NFR-16a PoC fallback with redaction/consent/audit
|
||||
- [ ] NFR-17: Immutable audit log
|
||||
- [ ] NFR-18: 100% LLM text cites MOH protocol via RAG
|
||||
- [ ] NFR-19: HITL digital signature before FINALIZED/ARCHIVED
|
||||
|
||||
---
|
||||
|
||||
## 3. Constraints & System Context
|
||||
|
||||
- **On-prem:** K3s on Dell PowerEdge (single hospital, ≤10 Mbps LAN)
|
||||
- **Model:** GemmaE2B-Q4 (~1.3 GB) distributed via intranet CDN → GCP CDN fallback → WASM local (WebLLM) or cloud MedGemma on Vertex AI (NFR-16a) → MOH templates
|
||||
- **Data:** Postgres + pgvector, Redis (5 constrained data types), MinIO, IndexedDB client prefs
|
||||
- **Auth:** Keycloak + RBAC inside K3s; GitLab/Jira on cloud VM (NFR-16a exception)
|
||||
- **CI/CD:** Jenkins inside K3s → cloud GitLab via SSH
|
||||
- **Failover:** NGINX + Keepalived VIP (≤2s switch)
|
||||
|
||||
---
|
||||
|
||||
## 4. C4 Models
|
||||
|
||||
### 4.1 Context Diagram (Tier 1)
|
||||
|
||||
See master: `SOLUTION_ARCHITECTURE_SPEC.md §8.1`
|
||||
Actors: Radiologist (UP5), Senior Expert (UP1), Support (UP4), Admin
|
||||
External: PACS, EMR/HIS, Triton, ladybugDB, pgvector, GemmaE2B/MedGemma, EmbeddingGemma
|
||||
|
||||
### 4.2 Container Diagram (Tier 2)
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_Software_Architecture_Containers"
|
||||
!include <C4/C4_Container>
|
||||
|
||||
title C4 Container Diagram - VKIST MSK Platform Software Architecture
|
||||
|
||||
Person(radiologist, "Diagnostic Radiologist (UP5)", "Primary user: loads DICOM, reviews grading, finalizes reports.")
|
||||
Person(admin, "System Administrator", "K3s ops, model updates, observability, failover.")
|
||||
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Clinical protocol validation, threshold approval.")
|
||||
Person_Ext(support_staff, "Support Staff (UP4)", "Patient registration, case queue.")
|
||||
|
||||
System_Boundary(hospital_lan, "Hospital LAN (Air-Gapped, ≤10 Mbps)") {
|
||||
Container(pwa, "React PWA Frontend", "React 18, TypeScript, Zustand, LiteRT, MediaPipe, Dexie.js", "View-angle validation via WASM; DICOM preview; Grad-CAM overlay; Decree 13 scrubber; IndexedDB for encrypted sessions and user prefs.")
|
||||
Container(nginx, "Active-Passive Gateway", "NGINX + Keepalived", "SSL termination, VIP load balancing, instant failover (≤2s).")
|
||||
|
||||
System_Boundary(k3s_cluster, "K3s Orchestration Cluster") {
|
||||
System_Boundary(edge_servers, "Application Server Cluster") {
|
||||
Container(edge_inference, "Edge Inference Service", "FastAPI (Python 3.12, Uvicorn)", "DICOM ingest, 3-step ML pipeline orchestration, Grad-CAM generation, report assembly, consult SSE streaming.")
|
||||
Container(rag_svc, "RAG & Knowledge Service", "FastAPI (Python 3.12, asyncpg)", "pgvector top-k retrieval, ladybugDB ontology wrapper, GemmaE2B/MedGemma consult route, RAG-Referee validation.")
|
||||
Container(audit_svc, "Audit & EMR Service", "Node.js / FastAPI Worker", "Immutable append-only audit events; HL7/FHIR EMR push with outbox retry.")
|
||||
Container(api_gw, "API Gateway", "Envoy", "TLS termination, rate limiting, routing, OIDC validation pass-through.")
|
||||
Container(auth_svc, "Identity & Access", "Keycloak", "OIDC, RBAC, realm vkist-msk.")
|
||||
Container(obs_stack, "Observability Stack", "Prometheus + Grafana", "Metrics scrape, dashboards, alerting.")
|
||||
}
|
||||
}
|
||||
|
||||
System_Boundary(db_vm, "Database VM") {
|
||||
ContainerDb(postgres, "PostgreSQL + pgvector", "PostgreSQL 16 + pgvector", "EMR records, spatial markers, MOH guideline HNSW index, session embeddings, audit ledger.")
|
||||
ContainerDb(redis, "Redis Cache", "Redis 7 (AOF + RDB)", "JWT sessions, MOH guideline chunks, DICOM metadata, consult_mode state, rate-limit counters.")
|
||||
ContainerDb(minio, "MinIO Object Store", "MinIO S3-compatible", "DICOM payloads, segmentation overlays, Grad-CAM heatmaps, report PDFs, model weight staging.")
|
||||
}
|
||||
}
|
||||
|
||||
System_Ext(triton, "Triton Inference Server", "NVIDIA Triton, ONNX/TensorRT, gRPC 8001", "3-step ML pipeline (angle → inflammation → segmentation) + embedding extraction.")
|
||||
System_Ext(emr, "Hospital EMR / HIS", "HL7/FHIR", "Finalized report storage, prescription sync, ground-truth records.")
|
||||
System_Ext(pacs, "PACS / Ultrasound Device", "DICOM/C-MOVE", "Image capture and retrieval.")
|
||||
System_Ext(gcp_cdn, "GCP CDN Emergency Fallback", "Signed-URL Cloud CDN (ap-southeast1)", "Non-clinical model weight distribution when intranet CDN unreachable.")
|
||||
System_Ext(vertex_ai, "GCP Vertex AI", "GemmaE2B via REST", "PoC-only NFR-16a inference tier; redacted payloads only.")
|
||||
|
||||
Rel(radiologist, pwa, "Loads scan, reviews grade, finalizes report, views explanations", "HTTPS 443")
|
||||
Rel(admin, nginx, "Deploys, monitors, configures", "HTTPS 443 / SSH")
|
||||
Rel(senior_expert, pwa, "Validates protocols, approves thresholds", "HTTPS 443")
|
||||
Rel(support_staff, pwa, "Registration, case queue", "HTTPS 443")
|
||||
|
||||
Rel(pwa, nginx, "API requests + pre-validated DICOM", "HTTPS 443")
|
||||
Rel(nginx, api_gw, "Routes upstream", "HTTP 8000")
|
||||
Rel(api_gw, edge_inference, "Forwards /api/*", "HTTP 8000")
|
||||
Rel(api_gw, rag_svc, "Forwards /rag/*", "HTTP 8001")
|
||||
Rel(api_gw, auth_svc, "Validates OIDC tokens", "HTTP 8080")
|
||||
|
||||
Rel(edge_inference, triton, "3-step inference + embeddings", "gRPC 8001")
|
||||
Rel(edge_inference, postgres, "Queries guidelines, writes cases/embeddings/audit", "SQL 5432")
|
||||
Rel(edge_inference, redis, "Session, rate-limit, consult_mode state", "TCP 6379")
|
||||
Rel(edge_inference, minio, "DICOM, overlays, reports", "S3 API")
|
||||
Rel(edge_inference, auth_svc, "Token introspection", "OIDC")
|
||||
Rel(edge_inference, obs_stack, "/metrics", "HTTP 9090")
|
||||
|
||||
Rel(rag_svc, postgres, "pgvector HNSW queries", "SQL 5432")
|
||||
Rel(rag_svc, redis, "Guideline cache, pub/sub invalidation", "TCP 6379")
|
||||
Rel(rag_svc, minio, "Guideline PDF ingestion staging", "S3 API")
|
||||
|
||||
Rel(audit_svc, postgres, "Appends immutable audit events", "SQL 5432")
|
||||
Rel(audit_svc, emr, "Finalized report push", "HL7/FHIR")
|
||||
Rel(audit_svc, redis, "Outbox lock, EMR retry counters", "TCP 6379")
|
||||
|
||||
Rel(edge_inference, emr, "Report push (via audit-svc wrapper)", "HL7/FHIR")
|
||||
Rel(pwa, pacs, "Direct DICOM capture / C-MOVE", "DICOM")
|
||||
|
||||
Rel(k3s_cluster, gcp_cdn, "Model weight fetch fallback (non-clinical)", "HTTPS 443 (signed URL)")
|
||||
Rel(edge_inference, vertex_ai, "PoC-only cloud consult (NFR-16a redacted)", "HTTPS 443 / REST")
|
||||
Rel(postgres, minio, "Backup checkpoint", "S3 API")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
Container communication summary:
|
||||
|
||||
| Source | Target | Protocol | Purpose |
|
||||
|--------|--------|----------|---------|
|
||||
| PWA | NGINX | HTTPS 443 | All API requests |
|
||||
| NGINX | Envoy | HTTP 8000 | Route upstream |
|
||||
| Envoy | Edge Inference | HTTP 8000 | `/api/*` |
|
||||
| Envoy | RAG Service | HTTP 8001 | `/rag/*` |
|
||||
| Envoy | Keycloak | HTTP 8080 | OIDC validation |
|
||||
| Edge Inference | Triton | gRPC 8001 | 3-step ML pipeline + embeddings |
|
||||
| Edge Inference | Postgres | TCP 5432 | SQL + pgvector HNSW |
|
||||
| Edge Inference | Redis | TCP 6379 | Session, rate-limit, consult-mode |
|
||||
| Edge Inference | MinIO | S3 API | DICOM, overlays, reports |
|
||||
| Edge Inference | Keycloak | OIDC | Token validation |
|
||||
| RAG Service | Postgres | TCP 5432 | pgvector HNSW |
|
||||
| RAG Service | Redis | TCP 6379 | Guideline cache, pub/sub |
|
||||
| Audit Service | Postgres | TCP 5432 | Immutable audit ledger |
|
||||
| Audit Service | EMR | HL7/FHIR | Finalized report push |
|
||||
| K3s | GCP CDN | HTTPS 443 | Signed-URL model weight emergency fetch |
|
||||
|
||||
### 4.3 Component Diagrams (Tier 3)
|
||||
|
||||
**Edge Inference Service (edge-inference-svc)**
|
||||
|
||||
```plantuml
|
||||
@startuml
|
||||
!include <C4/C4_Component>
|
||||
|
||||
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
||||
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream")
|
||||
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
|
||||
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
|
||||
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates")
|
||||
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates")
|
||||
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
|
||||
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
|
||||
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
|
||||
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation")
|
||||
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
|
||||
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
|
||||
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
|
||||
}
|
||||
Rel(api, stream, "delegates", "SSE")
|
||||
Rel(api, preproc, "validates image", "sync")
|
||||
Rel(api, router, "selects tier", "sync")
|
||||
Rel(router, pipeline, "invokes", "gRPC")
|
||||
Rel(router, stream, "fallback text", "SSE")
|
||||
Rel(api, gradcam, "requests overlay", "sync")
|
||||
Rel(api, report, "generates PDF", "sync")
|
||||
Rel(api, rag, "queries MOH", "SQL")
|
||||
Rel(rag, referee, "validates citations", "sync")
|
||||
Rel(api, nlp, "scrubs output", "sync")
|
||||
Rel(api, audit, "writes event", "sync")
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 4.3 Component Diagrams (Tier 3)
|
||||
|
||||
**Edge Inference Service (edge-inference-svc)**
|
||||
|
||||
```plantuml
|
||||
@startuml
|
||||
!include <C4/C4_Component>
|
||||
|
||||
Container_Boundary(edge_svc, "edge-inference-svc (FastAPI)") {
|
||||
Component(api, "API Controller", "REST + SSE", "/api/analyze, /api/consult/stream")
|
||||
Component(stream, "SSE Token Streamer", "FastAPI StreamingResponse", "200 ms TTFT, heart-beat every 30s")
|
||||
Component(preproc, "Image Preprocessor", "OpenCV + pydicom", "CLAHE, rescale, DICOM header scrub (client-side pre-check)")
|
||||
Component(router, "Inference Router", "consult_mode state", "Tier selection: WASM→Triton→Vertex→Templates")
|
||||
Component(breaker, "Circuit Breaker", "pybreaker", "Wrap Triton + EMR calls; fail-open to templates")
|
||||
Component(pipeline, "ML Pipeline", "gRPC client", "Angle→Inflammation→Segmentation→Measurement")
|
||||
Component(gradcam, "Grad-CAM Generator", "OpenCV", "Spatial activation overlay; base64 PNG to PWA")
|
||||
Component(report, "Report Builder", "WeasyPrint / ReportLab", "Circular 46 PDF; HITL signature gate before FINALIZED")
|
||||
Component(rag, "RAG Service", "pgvector SQL", "Retrieve top-5 MOH chunks; enforce NFR-18 citation")
|
||||
Component(referee, "RAG-Referee", "BERT classifier", "Reject LLM text if citation confidence < threshold")
|
||||
Component(nlp, "NLP Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine/clean residual PII; error if unresolvable")
|
||||
Component(audit, "Audit Logger", "Append-only writer", "Every tier transition, consent, finalize, override")
|
||||
}
|
||||
Rel(api, stream, "delegates", "SSE")
|
||||
Rel(api, router, "selects tier", "sync")
|
||||
Rel(router, pipeline, "invokes", "gRPC")
|
||||
Rel(router, stream, "fallback text", "SSE")
|
||||
Rel(api, preproc, "validates image", "sync")
|
||||
Rel(api, gradcam, "requests overlay", "sync")
|
||||
Rel(api, report, "generates PDF", "sync")
|
||||
Rel(api, rag, "queries MOH", "SQL")
|
||||
Rel(rag, referee, "validates citations", "sync")
|
||||
Rel(api, nlp, "scrubs output", "sync")
|
||||
Rel(api, audit, "writes event", "sync")
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 4.4 Deployment Diagram (Tier 3)
|
||||
|
||||
```plantuml
|
||||
@startuml
|
||||
!include <C4/C4_Deployment>
|
||||
LAYOUT_LEFT_RIGHT()
|
||||
|
||||
Deployment_Node(hw, "Dell PowerEdge (Hospital)") {
|
||||
Deployment_Node(k3s, "K3s Cluster") {
|
||||
Deployment_Node(pod_edge, "Pod: edge-inference-svc") {
|
||||
Container(ci, "FastAPI", "Python 3.12, Uvicorn")
|
||||
}
|
||||
Deployment_Node(pod_rag, "Pod: rag-svc") {
|
||||
Container(cr, "FastAPI", "Python 3.12, asyncpg")
|
||||
}
|
||||
Deployment_Node(pod_audit, "Pod: audit-svc") {
|
||||
Container(ca, "Node.js", "WAL writer")
|
||||
}
|
||||
Deployment_Node(pod_gw, "Pod: api-gateway") {
|
||||
Container(cg, "Envoy", "TLS termination, rate limit")
|
||||
}
|
||||
Deployment_Node(pod_auth, "Pod: auth-svc") {
|
||||
Container(ck, "Keycloak", "OIDC, RBAC")
|
||||
}
|
||||
Deployment_Node(pod_obs, "Pod: observability") {
|
||||
Container(cp, "Prometheus")
|
||||
Container(cf, "Grafana")
|
||||
}
|
||||
}
|
||||
Deployment_Node(db_vm, "DB VM") {
|
||||
ContainerDb(pg, "PostgreSQL + pgvector HNSW")
|
||||
ContainerDb(rd, "Redis")
|
||||
ContainerDb(mn, "MinIO")
|
||||
}
|
||||
}
|
||||
Deployment_Node(triton_node, "GPU Node (Triton)", "NVIDIA A10/T4, 24 GB VRAM") {
|
||||
Container(tc, "Triton Inference Server", "ONNX/TensorRT, gRPC 8001")
|
||||
}
|
||||
Deployment_Node(ext, "External (NFR-16a governed)") {
|
||||
ContainerDb(s3v, "S3 Vectors / GCP CDN / Vertex AI")
|
||||
}
|
||||
|
||||
Rel(ci, pg, "SQL", "TCP 5432")
|
||||
Rel(ci, rd, "Cache", "TCP 6379")
|
||||
Rel(ci, mn, "Blobs", "S3 API")
|
||||
Rel(ci, tc, "Inference", "gRPC 8001")
|
||||
Rel(k3s, db_vm, "SQL", "TCP")
|
||||
Rel(ci, ck, "Auth", "OIDC")
|
||||
Rel(ci, ca, "Audit", "HTTP")
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Component Specifications
|
||||
|
||||
### 5.1 React PWA
|
||||
|
||||
| Concern | Decision |
|
||||
|---------|----------|
|
||||
| Framework | React 18 + TypeScript + Zustand |
|
||||
| Styling | Tailwind CSS (mobile-first) |
|
||||
| DICOM viewer | Cornerstone.js + custom canvas layer |
|
||||
| Grad-CAM | `<canvas>` overlay; base64 PNG from FastAPI |
|
||||
| Local model | LiteRT (MediaPipe Tasks Vision) — angle pre-classifier |
|
||||
| Client storage | Dexie.js over IndexedDB (encrypted sessions, prefs) |
|
||||
| Offline | Service Worker: cache-first for shell; network-first for API |
|
||||
| Bundle | Tree-shake Cornerstone extras; split vendor chunk |
|
||||
| PHI safety | Decree 13 regex scrubber before any network write |
|
||||
| Edge guardrail | `guardrail.worker.ts` (WebWorker): Transformers.js BERT for hallucination/mal-intention detection; prompt injection scoring; scope-breach detection. OpenRedaction + pii-filter + js-data-anonymizer run in main thread or dedicated worker for redaction pipeline. Separate from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM) with no shared WASM memory. |
|
||||
|
||||
### 5.2 FastAPI Application (edge-inference-svc)
|
||||
|
||||
```
|
||||
app/
|
||||
├── main.py # Entry, middleware, lifespan
|
||||
├── config.py # Pydantic settings (env-driven)
|
||||
├── middleware/
|
||||
│ ├── auth.py # Keycloak OIDC validation
|
||||
│ ├── phi_scrub.py # Microsoft Presidio redaction gate (NFR-16a); refine edge output; error if unresolvable PII
|
||||
│ └── audit.py # Append-only event emitter
|
||||
├── routers/
|
||||
│ ├── analyze.py # POST /api/analyze (sync 3-step pipeline)
|
||||
│ ├── consult.py # SSE /api/consult/stream (GemmaE2B/MedGemma + RAG + guardrail session management)
|
||||
│ ├── pacs.py # C-MOVE proxy + DICOM upload
|
||||
│ ├── emr.py # HL7/FHIR push with outbox
|
||||
│ └── admin.py # Model update, drift review, cache invalidation
|
||||
├── services/
|
||||
│ ├── inference_router.py # consult_mode state machine
|
||||
│ ├── triton_client.py # gRPC with retry decorator
|
||||
│ ├── rag.py # pgvector top-k + citation formatter
|
||||
│ ├── referee.py # BERT drift + RAG confidence gate
|
||||
│ ├── guardrail.py # Edge BERT violation scoring, session termination, cloud mitigation trigger
|
||||
│ ├── redaction.py # Presidio AnonymizerEngine; re-verify edge redaction; refine residual PII; error if unresolvable
|
||||
│ ├── report.py # Circular 46 PDF + HITL signature
|
||||
│ ├── ontology.py # ladybugDB C++ bindings wrapper
|
||||
│ └── audit_writer.py # WAL append
|
||||
├── models/
|
||||
│ ├── dto.py # Request/response schemas (Pydantic v2)
|
||||
│ ├── domain.py # Case, Session, Grade, Embedding entities
|
||||
│ └── enums.py # ConsultMode, Grade, Tier
|
||||
└── infra/
|
||||
├── cache.py # Redis client (5 data types only)
|
||||
├── db.py # SQLAlchemy async engine + pgvector
|
||||
└── storage.py # MinIO S3 client
|
||||
```
|
||||
|
||||
### 5.3 Knowledge Service (rag-svc)
|
||||
|
||||
Separated from inference to allow independent scaling of RAG queries:
|
||||
- Endpoints: `POST /rag/query`, `POST /rag/referee-check`, `GET /rag/guideline/{version}`
|
||||
- Reads pgvector only; no Triton dependency
|
||||
- Publishes guideline update events to Redis Pub/Sub for cache invalidation
|
||||
|
||||
### 5.4 Data Models (Postgres)
|
||||
|
||||
Key tables (migration via Alembic):
|
||||
|
||||
```
|
||||
guidelines (id, version, title, source_url, embedding vector(768), active_from, retired_at)
|
||||
sessions (session_hash, radiologist_id, case_id, consult_mode, created_at, closed_at)
|
||||
cases (case_id, patient_hash, joint_site, dicom_checksum, final_grade, finalized_at, signer_id)
|
||||
embeddings (id, session_hash, chunk_text, embedding vector(768), source, created_at)
|
||||
audit_events (id, event_hash, session_hash, actor, action, tier, consent_token, redaction_manifest, payload_hash, ts)
|
||||
emr_outbox (id, case_id, payload, status, attempts, next_retry_at)
|
||||
user_prefs (user_id, jsonb, updated_at) # synced from IndexedDB
|
||||
```
|
||||
|
||||
Unique constraint: `cases.case_id` final grade requires `signer_id != NULL` (NFR-19).
|
||||
|
||||
### 5.5 Redis Keys (Exact Schema)
|
||||
|
||||
| Key pattern | Type | TTL | Purpose |
|
||||
|-------------|------|-----|---------|
|
||||
| `session:{hash}` | String + Hash | 3600s | JWT session validation |
|
||||
| `guideline:{ver}:{chunk_id}` | String | 604800s (7d) | MOH guideline chunk |
|
||||
| `dicom:{session_hash}` | String | 43200s (12h) | Per-session DICOM headers |
|
||||
| `consult_mode:{session_hash}` | String | 7200s | Tier state (tier_1..tier_3b) |
|
||||
| `rate:{actor_id}:{window}` | String | 30s sliding | Rate-limit counter |
|
||||
|
||||
Key-space invalidation: `guideline:{ver}:*` deleted on version bump via Postgres NOTIFY listener.
|
||||
|
||||
### 5.6 Triton gRPC Contract
|
||||
|
||||
```protobuf
|
||||
service InferencePipeline {
|
||||
rpc Run3Step (DICOMBytes) returns (PipelineResult);
|
||||
rpc ExtractEmbedding (TextChunk) returns (EmbeddingVector);
|
||||
}
|
||||
|
||||
message DICOMBytes {
|
||||
bytes raw_dicom = 1;
|
||||
string session_hash = 2;
|
||||
uint32 max_vram_mb = 3;
|
||||
}
|
||||
|
||||
message PipelineResult {
|
||||
AngleClass angle = 1;
|
||||
InflammationFlag inflammation = 2;
|
||||
SegmentationMask mask = 3;
|
||||
SynoviumMeasurement measurement = 4;
|
||||
Grade grade = 5;
|
||||
bytes gradcam_png = 6;
|
||||
}
|
||||
```
|
||||
|
||||
Idempotency invariant: identical `raw_dicom + session_hash` → identical output bytes. No partial state between steps.
|
||||
|
||||
### 5.7 Frontend Guardrail Runtime
|
||||
|
||||
**WebWorker Topology**
|
||||
|
||||
| Worker | Runtime | Role | Memory Cap |
|
||||
|--------|---------|------|------------|
|
||||
| `cv.worker.ts` | LiteRT (WASM) | CV inference: angle pre-classifier, image preprocessing | Isolated WASM instance |
|
||||
| `llm.worker.ts` | WebLLM (WASM) | GemmaE2B local generation, Gemma Functions tool-calling | Isolated WASM instance (NFR-4 1.5GB heap) |
|
||||
| `guardrail.worker.ts` | Transformers.js (WASM/WebGPU) | BERT classification: hallucination, prompt-injection, scope-breach scoring | Shared Web Worker thread (no WASM memory pool) |
|
||||
|
||||
Isolation rules:
|
||||
1. No `SharedArrayBuffer` or `Atomics` between workers — no raw WASM memory sharing.
|
||||
2. Communication via `postMessage` with structured clone (no transferable object reuse for audit integrity).
|
||||
3. Unload priority on memory pressure (browser `memorywarning` event): `llm.worker.ts` first, then `guardrail.worker.ts`, then `cv.worker.ts`.
|
||||
4. IndexedDB is the only persistence layer shared across workers; written by main thread or dedicated serializer, never read inside LLM context window.
|
||||
|
||||
**Edge Guardrail Decision Contract**
|
||||
|
||||
```
|
||||
User Input → OpenRedaction + pii-filter → js-data-anonymizer → BERT Guardrail
|
||||
↓
|
||||
PASS: forward to RAG → LLM
|
||||
FAIL: terminate LLM session → cloud mitigate
|
||||
```
|
||||
|
||||
Server-side FastAPI contracts:
|
||||
- `POST /api/guardrail-check` (internal): accepts BERT score + query hash; returns `PASS|MITIGATE`.
|
||||
- `POST /api/redaction-ground-check`: accepts client manifest hash + sanitized payload; returns `PASS|CLEANED|ERROR`. `CLEANED` = server refined residual PII and continues. `ERROR` = server unable to clean → client receives structured error.
|
||||
- `POST /api/consult/stream`: now includes `X-Guardrail-Version` and `X-Redaction-Manifest-Hash` headers for audit.
|
||||
|
||||
**IndexedDB Schema Additions**
|
||||
|
||||
| Table | Key | Columns | Invalidation |
|
||||
|-------|-----|---------|--------------|
|
||||
| `guardrail_models` | `[model_name, version]` | artifact_hash, size_bytes, load_timestamp | Version mismatch |
|
||||
| `policy_config` | `[policy_name]` | version, rules_json, bert_thresholds | Admin push |
|
||||
| `audit_tokens` | `[session_hash, entity_type]` | token_value, created_at | Session expiry |
|
||||
|
||||
---
|
||||
|
||||
## 6. Interface Contracts (Selected)
|
||||
|
||||
### 6.1 REST Endpoints
|
||||
|
||||
| Method | Path | Auth | Request | Response | Notes |
|
||||
|--------|------|------|---------|----------|-------|
|
||||
| POST | `/api/analyze` | JWT | multipart DICOM | JSON + Grad-CAM PNG | Sync, ≤1.5 s target |
|
||||
| GET | `/api/consult/stream` | JWT + consent | SSE text stream | NDJSON chunks | Token streaming ≤200 ms TTFT |
|
||||
| POST | `/api/emr/push` | JWT | case_id | 202 Accepted | Outbox if offline |
|
||||
| POST | `/api/admin/models` | Admin | modelZip | 200 OK | K3s rolling restart |
|
||||
| GET | `/api/rag/citations?q=` | JWT | query string | JSON top-5 chunks | NFR-18 |
|
||||
|
||||
### 6.2 SSE Consult Stream Contract
|
||||
|
||||
```
|
||||
event: token
|
||||
data: {"text":"The","confidence":0.92}
|
||||
|
||||
event: citation
|
||||
data: {"source":"MOH guideline 2024 §3.2","page":12}
|
||||
|
||||
event: done
|
||||
data: {"tier":"tier_2","latency_ms":340}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Build vs Buy Matrix (Sprint 1-2 Specific)
|
||||
|
||||
| Component | Build | Buy | Decision | Rationale |
|
||||
|-----------|-------|-----|----------|-----------|
|
||||
| Frontend PWA | Build | — | **Build** | Custom DICOM viewer + Grad-CAM layer; thin client requirement |
|
||||
| FastAPI backend | Build | — | **Build** | Tight Decree 13 + NFR-16a middleware; in-house domain logic |
|
||||
| Triton inference | — | Deploy self-hosted | **Deploy** | Open-source stack; no SaaS |
|
||||
| Ontology (ladybugDB) | Build | — | **Build** | Embedded C++; SNOMED-CT mapping custom to MSK |
|
||||
| pgvector | — | Postgres extension | **Use** | Zero infra overhead |
|
||||
| Redis | — | OSS | **Use** | Scoped to 5 types; self-hosted |
|
||||
| MinIO | — | OSS | **Use** | S3-compatible; self-hosted |
|
||||
| Auth | Build (Keycloak) | Auth0/Okta SaaS | **Build** | NFR-16: Keycloak on-prem, no SaaS identity |
|
||||
| EMR integration | Build | Mirth Connect | **Build** | Thin HL7 wrapper FastAPI → EMR; keeps surface area small |
|
||||
| CI/CD | Jenkins in K3s | SaaS GitHub Actions | **Build** | Jenkins inside LAN; cloud GitLab via SSH |
|
||||
| Issue tracking | Self-hosted Jira | Atlassian Cloud | **Build (NFR-16a)** | Cloud VM, compensating controls |
|
||||
|
||||
---
|
||||
|
||||
## 8. Task Decomposition
|
||||
|
||||
### Phase 1: Foundation (Parallel)
|
||||
- [ ] **T1-A Infra: K3s bootstrap + network policy** (S)
|
||||
Deploy K3s on Dell PowerEdge; Calico network policies; NGINX + Keepalived VIP; TLS secret automation.
|
||||
|
||||
- [ ] **T1-B Database VM: Postgres + pgvector + MinIO + Redis** (S)
|
||||
Install Postgres 16 + pgvector extension; MinIO (4 disk RAID); Redis AOF+RDB; backup cron to MinIO.
|
||||
|
||||
- [ ] **T1-C CI/CD: Jenkins in K3s + GitLab on cloud VM** (M)
|
||||
Jenkins agents as K3s jobs; pre-push PII hook; GitLab RDB backup job to MinIO; IP-whitelist IAM.
|
||||
|
||||
- [ ] **T1-D Auth: Keycloak realm + RBAC** (S)
|
||||
Realm `vkist-msk`; roles: radiologist, senior_expert, admin; client `pwa`, `edge-svc`, `rag-svc`.
|
||||
|
||||
- [ ] **T1-E PWA shell: React + Zustand + PWA manifest** (S)
|
||||
Offline-capable shell; Dexie.js setup; Service Worker with cache-first for shell only.
|
||||
|
||||
### Phase 2: Core ML Pipeline
|
||||
- [ ] **T2-A Triton server + 3-step model ensemble** (L)
|
||||
Angle → Inflammation → Segmentation pipeline; gRPC server on port 8001; TensorRT engines.
|
||||
|
||||
- [ ] **T2-B FastAPI edge-inference-svc** (L)
|
||||
`/api/analyze` endpoint; DICOM ingest; pipeline orchestration; Grad-CAM overlay generation.
|
||||
|
||||
- [ ] **T2-C Circuit Breaker + consult_mode state machine** (M)
|
||||
pybreaker around Triton + EMR; consult_mode Redis keys; SSE status push to PWA.
|
||||
|
||||
- [ ] **T2-D Retry decorators (Triton + EMR)** (S)
|
||||
Exponential backoff; idempotency enforcement; outbox queue for EMR failures.
|
||||
|
||||
### Phase 3: NLP & Knowledge
|
||||
- [ ] **T3-A pgvector guideline ingestion pipeline** (M)
|
||||
Ingest MOH PDFs → chunk → embed (BERT/Writing-Alignment) → HNSW index; Postgres NOTIFY pub/sub.
|
||||
|
||||
- [ ] **T3-B ladybugDB ontology setup** (M)
|
||||
SNOMED-CT knee/hip subset; MSK entity relationships; C++ embedded bindings in FastAPI.
|
||||
|
||||
- [ ] **T3-C RAG service + RAG-Referee** (M)
|
||||
`/rag/query`; top-5 retrieval; BERT classifier to validate LLM citations; reject if threshold fail.
|
||||
|
||||
- [ ] **T3-D Browser WebLLM + Cloud MedGemma LLM endpoints** (L)
|
||||
Browser: WebLLM (GemmaE2B-Q4) loaded via Service Worker from intranet/GCP CDN; runs in separate WebWorker from CV pipeline.
|
||||
Cloud: MedGemma on GCP Vertex AI (NFR-16a governed); FastAPI wrapper with streaming + Decree 13 redaction middleware.
|
||||
Triton: EmbeddingGemma only (768-dim RAG embeddings), no LLM hosting.
|
||||
|
||||
- [ ] **T3-E Decree 13 scrubber (client + server)** (M)
|
||||
Client: Dexie.js pre-egress regex. Server: FastAPI middleware for NFR-16a redaction; role-hash tokens.
|
||||
|
||||
### Phase 4: Compliance & HITL
|
||||
- [ ] **T4-A Immutable audit log** (M)
|
||||
Append-only WAL writer (audit-svc); schema per NFR-17; every tier transition + consent + finalize event.
|
||||
|
||||
- [ ] **T4-B HITL signature gate** (S)
|
||||
`cases.finalized_at` + `signer_id` non-null constraint; digital signature capture (VKI token + timestamp).
|
||||
|
||||
- [ ] **T4-C Circular 46 PDF report generator** (M)
|
||||
WeasyPrint template; includes grade, Grad-CAM image, MOH citations, signer block, audit hash.
|
||||
|
||||
### Phase 5: Observability & Hardening
|
||||
- [ ] **T5-A Prometheus + Grafana dashboards** (M)
|
||||
Triton VRAM/utilization; inference latency p50/p99; Redis hit rate; circuit-breaker state; K3s node health.
|
||||
|
||||
- [ ] **T5-A BERT drift monitor** (M)
|
||||
Weekly batch job comparing current session embeddings vs. baseline; alert admin via Grafana if KL divergence > threshold.
|
||||
|
||||
- [ ] **T5-C Fallback chain integration test** (S)
|
||||
Simulate Triton down → verify Tier 3a consent flow + redaction; simulate CDN down → verify GCP CDN signed URL fallback.
|
||||
|
||||
---
|
||||
|
||||
## 9. Execution Plan
|
||||
|
||||
### Week 1-2: Phase 1
|
||||
Deliverables: K3s cluster up, DB VM ready, CI/CD pipeline green, PWA shell live.
|
||||
|
||||
### Week 3-4: Phase 2
|
||||
Deliverables: `/api/analyze` end-to-end; Grad-CAM overlay visible in PWA; circuit-breaker handles Triton failure.
|
||||
|
||||
### Week 5-6: Phase 3
|
||||
Deliverables: RAG queries return MOH citations; GemmaE2B/MedGemma streams Vietnamese explanations; RAG-Referee blocks unmapped LLM text.
|
||||
|
||||
### Week 7: Phase 4
|
||||
Deliverables: Audit log immutable; HITL signature enforces finalization; Circular 46 PDF exports.
|
||||
|
||||
### Week 8: Phase 5 + Sprint Review
|
||||
Deliverables: Dashboards, drift monitor, integration tests, demo-ready PWA.
|
||||
|
||||
---
|
||||
|
||||
## 10. References
|
||||
|
||||
- [Solution Architecture Spec](./SOLUTION_ARCHITECTURE_SPEC.md) — full pattern citations, trade-offs, NFR-16a design
|
||||
- [Sprint 1-2 Architecture Spec](../workspace/sprint_1_2/SPRINT_1_2_ARCHITECTURE_SPEC.md) — sprint-scoped container/component/deployment diagrams
|
||||
- [Agent Skills](../../AGENT_SKILL/) — coding convention, secrets/PHI safety, contract hygiene
|
||||
- [Codebase Structure](../workspace/sprint_1_2/CODEBASE/) — backend/frontend/infra/knowledge/ml layout
|
||||
587
proj_level_reading/ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md
Normal file
587
proj_level_reading/ARCHITECT/SOLUTION_ARCHITECTURE_SPEC.md
Normal file
@@ -0,0 +1,587 @@
|
||||
# Solution Architecture Specification
|
||||
|
||||
## 1. Introduction
|
||||
|
||||
This document outlines the cloud-native architecture for the real-time musculoskeletal pathology processing system, designed to meet the functional and non-functional requirements outlined in the project's requirement analysis. The architecture leverages cloud-native principles (microservices, containerization, dynamic orchestration) while adhering to strict on-premise/local-intranet constraints due to data sovereignty and air-gap requirements (NFR-16). A time-limited, PoC-scoped relaxation (NFR-16a) permits governed emergency cloud fallback under mandatory redaction and consent controls.
|
||||
|
||||
## 2. Key Requirements
|
||||
|
||||
### 2.1 Functional Requirements (FRs)
|
||||
Key functional requirements extracted from the FR database include:
|
||||
- **FR-4**: Parse clinical text on-device (Prescription Parser / WebML Module)
|
||||
- **FR-5**: Map spatial coordinates to visual models (3D Mapping / Visualization Engine)
|
||||
- **FR-6**: Render muscle depth cross-sections (Kinetic Overlay Module)
|
||||
- **FR-7**: Log kinetic progress pins (Progress Tracking Module)
|
||||
- **FR-8**: Demonstrate joint mechanics dynamically (Patient Education Module)
|
||||
- **FR-9**: Render haptic-assisted edge-snapping magnifier (DICOM Viewer / Annotation Module)
|
||||
- **FR-10**: Record asynchronous voice and canvas telemetry (Asynchronous Communication Engine)
|
||||
- **FR-11**: Display progressive disclosure sheets and native alerts (User Interface / Push Notifications)
|
||||
- **FR-12**: Synchronize hardware-adaptive musculoskeletal models (Patient Education Module / 3D Visualization Engine)
|
||||
- **FR-13**: Encrypt and deliver sanitized patient payloads (Patient Portal / Security Module)
|
||||
- **FR-23**: Automatically segment joint anatomical structures by AI (Phân vùng hình ảnh cấu trúc giải phẫu khớp gối bằng AI)
|
||||
- **FR-24**: Automatically measure synovium thickness (Đo độ dày màng hoạt dịch Tự động)
|
||||
- **FR-25**: Diagnose and grade synovitis level (Đánh giá và phân cấp mức độ viêm màng hoạt dịch)
|
||||
- **FR-26**: Suggest treatment plan based on knee inflammation level (Gợi ý phương án điều trị dựa trên mức độ viêm khớp gối)
|
||||
- **FR-27**: Suggest list of anti-inflammatory and pain relief drugs (Đề xuất danh mục thuốc kháng viêm và giảm đau theo ca bệnh)
|
||||
|
||||
### 2.2 Non-Functional Requirements (NFRs)
|
||||
Critical non-functional requirements include:
|
||||
- **NFR-1**: DICOM Collaborative Rendering Speed ≤ 3.0 seconds
|
||||
- **NFR-4**: Client Memory Footprint ≤ 150 MB RAM
|
||||
- **NFR-5**: Core Vision Inference Latency ≤ 1.5 seconds (local on-premise server)
|
||||
- **NFR-6**: Server-Side Edge Model Quantization ≤ 2 GB VRAM on target server nodes
|
||||
- **NFR-7**: Real-Time UI Screen Refresh (Token Streaming) ≤ 200 ms from inference begins
|
||||
- **NFR-8**: Local Network Fault Tolerance: 0% data corruption, active/remember user request during Wi-Fi disconnections
|
||||
- **NFR-9**: Localized System Availability ≥ 99.9% during official public sector operating windows, downtime ≤ 45 seconds per day
|
||||
- **NFR-10**: Automated Generative Safety Guardrails: <90% verification prohibited, 100% of LLM-generated patient text explanations must pass verification
|
||||
- **NFR-11**: Frontline Usability & Training Curve: onboarding training ≤ 45 minutes, error rate ≤ 1 config slip per week
|
||||
- **NFR-12**: Zero-Friction Explainability Integration: accessing baseline model confidence intervals or guideline justifications requires EXACTLY 0 extra clicks, displayed in primary medical viewport
|
||||
- **NFR-13**: Spatial Layer-Activation Mapping (Anti-Black-Box Mandate): vision stack must natively output spatial layer-activation maps (e.g., Grad-CAM overlays), display with zero extra clicks during automated screening
|
||||
- **NFR-14**: Legacy Local Hardware Compatibility: UI modules shall not require dedicated external client-side GPU or hardware neural accelerator, operate on Android 10+ with 3GB RAM
|
||||
- **NFR-15**: National EMR Compliance (Circular 46/2018/TT-BYT): must comply with Vietnamese MOH EMR regulations
|
||||
- **NFR-16**: Local Intranet Cloud & Air-Gapped Data Isolation (PRIMARY — permanent): platform tech stack shall NOT transmit diagnostic/identifiable clinical info across public internet during normal primary clinical workflows; external cloud processing prohibited for production operation; only execute & deployed on on-premise servers, local intranet, or isolated specialist machines
|
||||
- **NFR-16a**: Emergency Cloud Fallback with Redaction (PoC-SCOPE — time-limited, expires upon PoC user sign-off):
|
||||
|
||||
[1] Cloud LLM API (e.g., GCP Vertex AI MedGemma) may be invoked ONLY when browser-side WebLLM (GemmaE2B) is unavailable AND the user explicitly consents per-session via UI acknowledgment;
|
||||
|
||||
[2] before ANY bytes leave the hospital boundary, FastAPI Redaction Middleware MUST strip all Decree 13 PII fields — patient name, DOB, MRN, address, phone, insurance ID, facial geometry, and DICOM patient metadata — replacing them with role-hash tokens [BN:7f3a2c]; non-identifying clinical fields (age, sex, joint site, findings) are retained for inference utility;
|
||||
|
||||
[3] GCP project must be configured with Customer-Managed KMS keys (CSEK) enabling crypto-erasure on key revocation, 30-day Cloud Storage lifecycle deletion, and a 1-year project-level auto-deletion timer;
|
||||
|
||||
[4] consent token + redaction manifest written to on-prem immutable audit log BEFORE the egress call;
|
||||
|
||||
[5] redacted payloads MUST NEVER contain: any free-text field capable of re-identification via MOH or administrative records;
|
||||
|
||||
[6] NFR-16a is formally retired upon PoC user sign-off and NFR-16 is restored as the permanent production deployment constraint; NFR-16a relaxation does not apply to NFR-15 Circular 46 EMR compliance obligations
|
||||
- **NFR-17**: Cryptographic Accountability Logging: application layer shall not allow any user to alter/delete logs; every action where AI recommendation accepted/overridden saved immutably
|
||||
- **NFR-18**: MOH Guideline-Anchored RAG Pipeline: system shall not render open-ended clinical text summaries without clear traceable footnote citing official MOH protocols; use RAG tied to health guidelines
|
||||
- **NFR-19**: Human-in-the-Loop (HITL) Clinical Gatekeeping: database layer shall not allow automated ML/LLM diagnosis/report to transition to FINALIZED/ARCHIVED/PATIENT_ACCESSIBLE without authenticating digital signature from licensed human clinician
|
||||
|
||||
## 3. Architectural Goals and Principles
|
||||
|
||||
Guided by the above requirements and the solution_architect_corpus, the architecture adheres to the following principles:
|
||||
- **Low Latency**: Optimize for sub-second responses where critical (NFR-1,5,7).
|
||||
- **Edge Computing**: Leverage edge servers and client-side WASM for local inference to meet NFR-5 and NFR-16; NFR-16a emergency fallback activates only when all local tiers are confirmed unavailable.
|
||||
- **Resilience**: Implement fault tolerance, caching, and graceful degradation to handle network instability (NFR-8,9).
|
||||
- **Security & Compliance**: Ensure data sovereignty, air-gap, immutable audit logs, and HIPAA-equivalent safeguards (NFR-15,16,17). NFR-16a introduces a governed, redaction-mandatory cloud pathway during PoC only; all egress requires prior audit-log commit and explicit user consent.
|
||||
- **Explainability & Trust**: Provide zero-friction access to model confidence and spatial attributions (NFR-12,13).
|
||||
- **Human-in-the-Loop**: Enforce clinical oversight for AI-generated decisions (NFR-19).
|
||||
- **Scalability & Maintainability**: Use microservices, container orchestration, and observability for easy updates and scaling.
|
||||
|
||||
## 4. Proposed Architecture
|
||||
|
||||
### 4.1 High-Level Overview
|
||||
The system adopts a hybrid edge-cloud (on-premise) architecture with the following layers:
|
||||
1. **Edge Inference Layer**: Located on-premise at hospitals/clinics, runs lightweight ML models for real-time DICOM processing (NFR-5,6).
|
||||
2. **Micro-Services Layer**: Containerized services orchestrated by K3s (Kubernetes-certified, lightweight distribution designed for edge and resource-constrained production environments). K3s selected over alternatives (Docker Swarm, Nomad, ECS Fargate, Cloud Run) on the following criteria: NFR-16 requires on-premise deployment — eliminating ECS Fargate and Cloud Run (cloud-only platforms). Docker Swarm offers lowest PoC operational cost but is in maintenance mode with no viable scaling path; migration to production-grade orchestration requires complete rewrite of all deployment artifacts. Nomad is operationally viable but lacks ecosystem depth and has less documented migration path to multi-cluster federation. K3s is already production-grade at single-site scale, requires no migration to "production" — it IS the production platform. The scaling path to N hospitals is multi-cluster federation via K3s cluster-api, not platform replacement. All manifests and service definitions are forward-compatible with upstream Kubernetes.
|
||||
3. **Data Layer**: On-premise databases (PostgreSQL + pgvector for relational + vector search, Redis for caching, MinIO for S3-compatible object storage).
|
||||
3. **Embedding Layer**: EmbeddingGemma (768-dim) for RAG retrieval vectors; separate from BERT which is reserved for drift monitoring and RAG-Referee classification only.
|
||||
3. **Vector Layer (two-tier)**:
|
||||
- **Hot + Warm: pgvector** (Postgres disk-backed HNSW index, ~15K MOH guideline vectors plus session and case embeddings — serves both real-time RAG during consult and non-real-time analytical queries in a single store. NFR-18, NFR-12. NFR-16 compliant.)
|
||||
- **Cold: S3 Vectors** (AWS ap-southeast-1, billion-scale archival, cross-facility bootstrap source, disaster recovery rebuild origin — accessed under NFR-16a governance.)
|
||||
5. **Presentation Layer**: Progressive Web App (PWA) built with React, Web Workers, and service workers for offline capability and low memory footprint (NFR-4,14).
|
||||
6. **Observability & Security Layer**: Centralized logging (immutable append-only), monitoring, and audit trails (NFR-10,17).
|
||||
|
||||
### 4.2 Data Flow
|
||||
1. User captures DICOM image via PWA frontend.
|
||||
2. Image sent to edge inference service via secure local API (BFF pattern).
|
||||
3. Edge service runs preliminary detection/segmentation (e.g., synovium thickness) using quantized models (NFR-6).
|
||||
4. Results sent to microservice backbone for further analysis (e.g., suggesting treatment plans via RAG).
|
||||
5. RAG pipeline retrieves relevant guidelines from local vector database, generates explanation with citations (NFR-18).
|
||||
6. Final results (including spatial overlays) sent back to PWA for display with zero-friction explainability (NFR-12,13).
|
||||
7. User actions (e.g., accepting treatment suggestion) logged immutably (NFR-17) and require HITL approval for finalization (NFR-19).
|
||||
8. All data remains within local intranet; no egress to public internet under NFR-16 (normal operation). Under NFR-16a (PoC emergency fallback only): egress permitted exclusively of redacted, non-identifiable payloads with explicit user consent and prior audit-log commit.
|
||||
|
||||
## 5. Component Mapping to Patterns (with Citations)
|
||||
|
||||
### 5.1 Edge Computing, Model Distribution & Governed Cloud Fallback
|
||||
- **Pattern**: Bring-Your-Own-Compute (BYOC) Client-Side Inference + Intranet CDN Distribution + Emergency Cloud Fallback (NFR-16a)
|
||||
- **Source**: Cloud_Architecture_Patterns.md, line 244-247 (Asset Ingestion at edge nodes); Cloud_Design_pattern_Azure.md, line 19,84 (Cache-Aside, Read-Through); Designing_high_quality_distributed_Cloud_Native_applications_on_Azure_V1.1.md, line 211 (BFF)
|
||||
- **Application**:
|
||||
- **BYOC primary (weight distribution)**: GemmaE2B-Q4 (~1.3GB, 4-bit quantized) distributed in the following order:
|
||||
[1] Pre-bundled ~200MB model stub (shipped inside PWA install — covers immediate shell load, zero network dependency).
|
||||
[2] Hospital intranet CDN (MinIO/NAS at `hospital.local/models/`) — full weights lazy-loaded after shell is interactive; SHA-256 content verification; cache-first Service Worker. This is the preferred distribution path under NFR-16.
|
||||
[3] GCP CDN emergency fallback (asia-southeast1 = Singapore, nearest to Vietnam) — activated ONLY when hospital intranet CDN is confirmed unreachable. GCP backend bucket (`cdn-backend-models-vkist`) wraps a Cloud Storage bucket (`gs://vkist-models-{site_id}`) with Cloud CDN enabled; signed-URL access control (no `allUsers` public exposure); cache mode `CACHE_ALL_STATIC`, default TTL 24h, max TTL 7d (matches model version lifecycle). Service Worker receives a 307 redirect to signed CDN URL on intranet CDN failure; PWA caches the fetched weights locally via Cache Storage as normal. Because model weights contain zero patient data — they are static computational artifacts — the GCP CDN fetch event is committed to the on-prem audit log (case_hash, timestamp, consult_mode = "cdn_fallback") but does NOT trigger the full NFR-16a redaction pipeline. The redaction pipeline applies only to Tier 3a inference calls carrying clinical payloads, not to non-clinical model binary fetches.
|
||||
- **NFR-4 clarification**: NFR-4's 150MB ceiling covers the core application bundle (JS/TS code, UI assets, runtime state) at idle. Supra-150MB memory consumed by models, caches, and inference buffers is bounded separately under the WASM heap cap (1.5GB) and instrumented independently.
|
||||
- **Inference Fallback Chain** (LLM consult path only — CV pipeline is always on-prem Triton):
|
||||
- **Tier 0** → Browser WebLLM (GemmaE2B local in WASM, instant, zero network — preferred default when device has ≥3GB RAM and WebGPU available). Consult mode: `browser_local`. Triggered automatically on page load if model weights are cached and WebGPU is detected.
|
||||
- **Tier 1** → [REMOVED — Triton does NOT host LLM. Triton hosts CV models + EmbeddingGemma only.]
|
||||
- **Tier 2** → On-prem Triton (hospital GPU node — CV pipeline ONLY: angle → inflammation → segmentation + EmbeddingGemma for RAG embeddings). The LLM consult does NOT fall back to Triton. If browser WebLLM is unavailable, the next tier is cloud.
|
||||
- **Tier 3a** → GCP Vertex AI (MedGemma via REST API — **PoC ONLY under NFR-16a**). Activated when Tier 0 (browser WebLLM) is unavailable (device unsupported, model not cached, memory exceeded) AND user explicitly consents per-session. All payloads MUST pass Decree 13 PII redaction gate (see §5.4) before leaving hospital boundary. GCP project configured with CSEK, 30-day storage lifecycle, 1-year project deletion timer. Consult mode: `cloud_vertex`.
|
||||
- **Tier 3b** → Cloud LLM Arbiter (GCP Vertex AI MedGemma — **PoC ONLY under NFR-16a, BERT-triggered correction**). Activated when BERT drift monitor (UC-74821) detects semantic impasse, logical contradiction, or contextual hallucination during Socratic dialogue, OR when radiologist explicitly requests "second opinion." Same Vertex AI endpoint as Tier 3a, but with arbiter system prompt + retrieved MOH chunks injected. ALL calls require prior audit-log commit of drift signal + user consent + Decree 13 redaction. Consult mode: `cloud_vertex_arbiter`.
|
||||
- **Tier 3c** → MOH guideline template responses (rule-based, no generative model, always available, deterministic, cited — safety floor when all inference tiers fail). Consult mode: `templates`.
|
||||
- **Circuit Breaker**: PWA Feature Flag Manager (`consult_mode` state machine) attempts tiers sequentially: `browser_local` → `cloud_vertex` → `cloud_vertex_arbiter` → `templates`. Emits tier-transition events to on-prem immutable audit log (NFR-17). Tier 3a/3b transitions require consent token + redaction manifest committed *before* the HTTP call. CDN fallback events (model weight distribution only) log lighter audit entry: no redaction required, no consent required (weights carry no PII).
|
||||
- **GCP CDN configuration** (emergency distribution path only):
|
||||
- Cloud Storage bucket: `gs://vkist-models-{site_id}` — Uniform bucket-level access, no `allUsers` public exposure.
|
||||
- Signed-URL generation: FastAPI generates time-limited signed URLs (signed by project service account private key, CSEK-managed) with a 15-minute expiry; returned to PWA as a `307` redirect on intranet CDN failure.
|
||||
- Cloud CDN backend bucket: wraps the GCS bucket; `enable-cdn: true`, `cache-mode: CACHE_ALL_STATIC`, `default-ttl: 86400` (24h), `max-ttl: 604800` (7d = model version lifecycle). Edge caching means subsequent PWA fetches for the same model version are served from the nearest Google PoP, not the asia-southeast1 origin — no repeated hospital-to-GCP round trip needed.
|
||||
- Private origin authentication: CDN service account has `roles/storage.objectViewer` on the bucket; direct GCS access (bypassing CDN) is blocked by IAM — all model fetches flow through CDN.
|
||||
- Model version rotation: on drift monitor or admin update, a new bucket path (`models/v2/gemma-e2b-q4.bin`) is uploaded; CDN cache is invalidated via `gcloud compute backend-buckets invalidate-cdn-cache`; old path serves stale content only until TTL expires — zero in-flight breakage.
|
||||
- **Cache strategy**: Service Worker intercepts `/models/*` imports with cache-first strategy. Pre-bundled ~200MB model stub shipped inside PWA install; remaining weights lazy-loaded from intranet CDN after shell is interactive. Cache entries tagged with content-hash version header; corruption triggers re-download with exponential backoff (NFR-8).
|
||||
- **Pattern**: Gateway Aggregator
|
||||
- **Source**: Designing_high_quality_distributed_Cloud_Native_applications_on_Azure_V1.1.md, line 203, 1011
|
||||
- **Application**: API gateway aggregates multiple microservice calls (e.g., prediction + RAG + logging) into a single response to reduce client-side round trips.
|
||||
- **Pattern**: Backend for Frontend (BFF)
|
||||
- **Source**: Designing_high_quality_distributed_Cloud_Native_applications_on_Azure_V1.1.md, line 211
|
||||
- **Application**: Tailor API responses for different client types (e.g., lightweight payloads for low-end tablets per NFR-14; enriched payloads with Grad-CAM overlays for desktop workstations).
|
||||
|
||||
### 5.2 Caching & Performance
|
||||
- **Pattern**: Cache-Aside (Lazy Loading) + Version-Key Invalidation + Stale-While-Revalidate
|
||||
- **Source**: A_Cache_Handbook_for_SWE_Quang_Hoang.md, line 72-76 (Local vs Remote Cache), line 94-114 (Look-Aside workflow)
|
||||
- **Application**: Use Redis as a look-aside cache exclusively for the server-side hot paths that justify a remote cache. Every other data type is served from its primary store without a cache layer. Caching is not applied by default — each entry must pass a read-frequency vs. staleness-tolerance analysis.
|
||||
- **Redis cache-only data types** (server-side, local Redis per hospital):
|
||||
- **JWT session state** (TTL 1h, matches token expiry): validates active sessions without Postgres roundtrip; coalesced look-aside with single-flight mutex to prevent thundering herd on session-miss.
|
||||
- **MOH guideline chunks** (TTL 7d): version-key invalidation triggered by Postgres NOTIFY on guideline ingestion events; cache key = `guideline:{version}:{chunk_id}`, full key-space retired on version bump, no TTL races.
|
||||
- **DICOM metadata per session** (TTL 12h, per-session scope): avoids repeated Postgres lookups of DICOM headers during the 3-step ML pipeline; evicted after session completion.
|
||||
- **Circuit-breaker / consult_mode state** (TTL 2h, matches session): consult_mode enum (tier_1/tier_2/tier_3a/tier_3b) keyed by session_hash. Written atomically on tier transitions. Read by: FastAPI routing layer, PWA (via SSE status push), Prometheus metrics exporter. Justifies Redis over Postgres: sub-ms reads required, writes are frequent (per BERT token stream during Socratic dialogue), data is volatile (expires with session), and multiple FastAPI workers behind NGINX need shared visibility — in-process memory cannot cross worker boundaries.
|
||||
- **Rate-limit counters** (TTL 30s, sliding window): INCR on consult request entry. Self-clearing via EXPIRE. Prevents Triton GPU saturation under concurrent burst from multiple radiologists. Atomic Redis operation; Postgres row-lock alternative creates write amplification on hot path.
|
||||
- **Not cached in Redis** (served from primary store):
|
||||
- Audit log: append-only, NFR-17 immutability requirement — no intermediate cache layer.
|
||||
- Inference results: already held in FastAPI process memory during active session.
|
||||
- ladybugDB ontology: embedded C++ inside FastAPI — already an in-process structured cache.
|
||||
- BERT drift tensors: compute-bound, held in Triton GPU memory or local NVMe; not reusable across sessions and latency-sensitive to in-process serving.
|
||||
- EmbeddingGemma inference: small 768-dim model, served on-demand by Triton for RAG embedding extraction; outputs written directly to pgvector, not cached.
|
||||
- **Client-side preference caching (IndexedDB)**:
|
||||
- User preferences (language, UI density, contrast mode, default annotation tool) are persisted in IndexedDB via Dexie.js.
|
||||
- Characteristics:
|
||||
- write frequency ≈ 0 (changes only on explicit user action), read frequency = every session start, no sharing needed across devices.
|
||||
- Cache invalidation: manual only — user taps Settings → Save → IndexedDB write + background sync to Postgres.
|
||||
- No server roundtrip on read. IndexedDB survives browser cache clears (distinct storage domain), so preference data outlives the Service Worker model cache — appropriate for data that is intentionally persistent.
|
||||
- **Deployment**: Standalone Redis per hospital LAN (local-only, no cross-site routing). AOF enabled (everysec fsync) + periodic RDB snapshots every 15 min / 5 min / 1 min depending on write volume. Restart policy: unless-stopped. No Redis Sentinel in PoC/Pilot: Redis is a latency optimization, never a source of truth — total cache loss degrades one request cycle's latency, not data integrity. RDB snapshots backed up to site MinIO for disk-failure recovery.
|
||||
- **Cache-miss protocol**: Look-aside with request coalescing. On miss, FastAPI reads Postgres, writes Redis, returns response. On simultaneous misses for same key, first requester acquires a 5s lock in Redis (`lock:{key}`); other requesters wait up to 5s then fail-open to Postgres directly — no queue buildup.
|
||||
- **Cache-staleness protocol**: MOH guidelines use version-key invalidation (active, event-driven). Session state uses TTL (passive, matches JWT lifetime). DICOM metadata uses TTL with no revalidation (immutable per session). User preferences use IndexedDB with manual invalidation — no TTL needed.
|
||||
- **Failure contract**: Redis process crash → Docker Compose restarts within ~2s → AOF replay recovers to last checkpoint (<5s). During recovery window, all reads fall through to Postgres (source of truth) with ≤10ms additional latency per request. No user-facing failure. If AOF + RDB both lost (disk failure): cold-start miss spike, repopulated organically from Postgres within one request cycle per key.
|
||||
- **Pattern**: Read-Through/Write-Through (handled explicitly by application per corpus)
|
||||
- **Source**: Cloud_Design_pattern_Azure.md, line 19,84
|
||||
- **Application**: User preference writes flow through FastAPI → Postgres (durable) → IndexedDB (client-side, on next sync). Application logic manages consistency explicitly; no cache-driven inconsistency risk since IndexedDB and Postgres are both authoritative for their respective scopes (client vs. server).
|
||||
|
||||
### 5.3 Resilience & Fault Tolerance
|
||||
- **Pattern**: Circuit Breaker
|
||||
- **Source**: Cloud_Design_pattern_Azure.md, line 106-117
|
||||
- **Application**: Wrap external service calls (e.g., to knowledge base) with circuit breaker to prevent cascading failures during network glitches (NFR-8).
|
||||
- **Pattern**: Retry with Exponential Backoff (application-layer, not Ambassador sidecar)
|
||||
- **Source**: Designing_high_quality_distributed_Cloud_Native_applications_on_Azure_V1.1.md, line 222 (Ambassador Pattern — principle applied without sidecar in PoC)
|
||||
- **Rationale**: The Ambassador pattern abstracts retry, timeout, and mTLS into a sidecar proxy. For PoC/Docker Compose deployment, the operational cost of Envoy sidecars exceeds their benefit. The retry-with-backoff principle is applied in-application via Python decorators, scoped only to the two call boundaries where transient failures are demonstrated and recovery is safe.
|
||||
- **Retry-scoped calls** (all others use fail-fast, no retry):
|
||||
- **FastAPI → Triton gRPC** (ML inference): up to 3 attempts, exponential backoff 1s→2s→4s, total deadline 30s. Retry only on gRPC UNAVAILABLE (node restart). Not retried on INVALID_ARGUMENT or DEADLINE_EXCEEDED. Triton inference is contractually idempotent: identical DICOM input produces identical output; retry re-submits full pipeline, no partial state accumulation between steps. If all 3 attempts fail, circuit breaker opens and consult routes to Tier 3b (MOH templates).
|
||||
- **FastAPI → EMR HL7/FHIR** (report push): up to 3 attempts, exponential backoff 2s→4s→8s, total deadline 60s. Retry only on ConnectionError/TimeoutError. Not retried on EMR business-level rejections (validation failures). If all 3 attempts fail, report written to `emr_outbox` Postgres table with status PENDING; background worker retries every 5 min with jittered backoff. Radiologist sees: "Report signed and queued for EMR sync."
|
||||
- **Not retried** (fail-fast): Redis, Postgres (including pgvector), ladybugDB (in-process), MinIO/CDN (LAN stable). Retry on these creates amplification on already-healthy or already-failed infrastructure.
|
||||
- **Idempotency constraint**: The Triton gRPC interface contract requires that re-submission of identical input bytes produces identical output bytes. No Triton handler may accumulate partial state between pipeline steps that would change on re-entry. This is a design-time invariant enforced at interface review, not a runtime check.
|
||||
- **Production scaling path**: At N ≥ 20 hospitals or K3s deployment, migrate retry logic from in-process decorators to declarative Envoy sidecar configuration. Retry contracts remain semantically identical; deployment mechanism changes only.
|
||||
|
||||
### 5.4 Security, Compliance & Audit
|
||||
- **Pattern**: Immutable Audit Log (Append-Only Event Store)
|
||||
- **Source**: Cloud_Design_pattern_Azure.md, line 449 (Append-Only Event Store)
|
||||
- **Source**: GCP_Cloud_Archtiecture_Gudie.md, line 132 (unalterable, immutable audit log)
|
||||
- **Application**: Store all clinical decisions and AI interactions in an append-only log (e.g., using Apache Kafka or event-sourced database) to prevent tampering (NFR-17).
|
||||
- **Pattern**: Air-Gap Isolation
|
||||
- **Source**: GCP_Cloud_Archtiecture_Gudie.md, line 178,181 (logical air-gap by powering down MediaAgents or using scheduler)
|
||||
- **Application**: Enforce network policies to isolate healthcare workloads from public internet under NFR-16; use local-only container registries and internal service mesh. During PoC under NFR-16a, a governed egress pathway is permitted exclusively through the FastAPI Redaction Middleware carrying pre-egress audit-log commits and explicit user consent tokens — no direct client-to-cloud calls are ever permitted.
|
||||
- **Pattern**: Two-Person Rule (Multi-Party Authorization)
|
||||
- **Source**: google_infrastructure_whitepaper_fa.md, line 129,133,435
|
||||
- **Application**: Require dual clinician approval for high-risk AI-generated treatment plans (mapping to NFR-19 HITL gatekeeping).
|
||||
|
||||
### 5.5 Explainability & RAG
|
||||
- **Pattern**: Two-Tier Vector Architecture for Retrieval-Augmented Generation
|
||||
- **Source**: A_Cache_Handbook_for_SWE_Quang_Hoang.md (retrieval times, cache tiering) + Cloud_Design_pattern_Azure.md (Query Stack, line 371) + Cloud_Architecture_Patterns.md (hot/warm/cold data tiering)
|
||||
- **Application**: Two vector stores serve distinct access patterns. pgvector handles all in-service query loads; S3 Vectors handles archival and disaster recovery bootstrap. One less infrastructure tier than the three-tier design because pgvector covers both the hot real-time RAG path and the warm analytical path — the dataset size (~15K vectors) does not justify a separate in-memory ANN store at PoC scale.
|
||||
|
||||
**Tier 1 — HOT + WARM: pgvector (Postgres HNSW, single store for all live queries)**
|
||||
- What:
|
||||
• MOH guideline embeddings (~15K vectors, 768-dim) — live index for active consult RAG queries (NFR-18, NFR-12).
|
||||
Generated by EmbeddingGemma (768-dim), NOT BERT. BERT is reserved for drift monitoring and RAG-Referee classification.
|
||||
|
||||
• Session consult embeddings (GemmaE2B dialogue turns linked to session_hash)
|
||||
|
||||
• Finalized case embeddings (joined to EMR case_id in same transaction — NFR-19 HITL finalization writes both report and embedding atomically)
|
||||
|
||||
• Historical guideline version embeddings (prior MOH versions retained for audit, not current-query use)
|
||||
|
||||
• Drift-detection training embeddings (baseline corpus for BERT drift comparison — BERT 768-dim, separate from pgvector RAG store; stored in dedicated drift_embeddings table, not in the live HNSW query path)
|
||||
|
||||
- When: ALL live queries — real-time RAG during consult (NFR-18) AND non-real-time batch analytics, audit review, longitudinal research, drift detection.
|
||||
- Why pgvector for real-time RAG: At ~15K vectors with HNSW index, pgvector query latency is ~5–20ms on the Postgres warm buffer set. The entire active guideline index fits in Postgres shared_buffers (set to 4GB on the DB VM). This is within NFR-7's ≤200ms TTFT budget with room to spare. Adding a separate Qdrant process for this dataset size is operational overhead without measurable latency benefit.
|
||||
- Why pgvector for warm/analytical: You already run Postgres. Adding vector search is one migration, not a new service. The dataset scales to millions of vectors before HNSW degrades meaningfully. Transactional consistency with clinical records eliminates sync bugs — case embedding and EMR record are written in the same WAL, same backup, same recovery. Complex SQL filtering (WHERE joint_site = 'left_knee' AND grade = 2 AND session_date > '2025-01-01') is expressible in one query. Zero additional operational overhead.
|
||||
- Why NOT Qdrant at PoC scale: Qdrant's in-memory ANN advantage appears at millions of vectors or high QPS loads. At 15K vectors and <100 queries/minute (realistic for a single-hospital PoC with ~20 concurrent radiologists), pgvector HNSW in shared_buffers is indistinguishable in latency. Qdrant adds a container, a separate backup strategy, a separate gRPC endpoint, and a separate monitoring target — none of which earn their cost at this scale.
|
||||
- Phase 2 migration path: If the guideline corpus grows beyond ~100K vectors OR concurrent query volume exceeds ~500 QPS, introduce Qdrant as Tier 1 hot cache, promoted from pgvector on a scheduled basis. pgvector remains the authoritative warm store. This is an additive migration — no data model changes.
|
||||
- Placement: Existing Postgres VM. NFR-16 compliant. Already deployed.
|
||||
|
||||
**Tier 2 — COLD: S3 Vectors (AWS, archival + bootstrap source)**
|
||||
- What:
|
||||
• All historical guideline version embeddings (every MOH update, retained indefinitely for longitudinal audit)
|
||||
|
||||
• Billion-scale case embedding archive (long-term research corpus, written weekly in batch)
|
||||
|
||||
• Cross-facility replication source (single source of truth for guideline index across all hospital sites)
|
||||
|
||||
• Training corpus snapshots for next model iteration
|
||||
- When: Disaster recovery bootstrap (rebuilds pgvector Qdrant-promotion cache from S3 Vectors snapshot), research analysis (bulk similarity search across years of cases), cross-site aggregation. NOT real-time consult queries.
|
||||
- Why S3 Vectors: Sub-second latency for infrequent queries is acceptable for archival workloads. 11 9's durability exceeds any local disk or MinIO backup. Serverless — no infrastructure to manage at VKIST central. The 90% cost reduction vs specialized vector DBs matters at billion-vector scale.
|
||||
- NFR-16 compliance: Tier 2 is accessed ONLY under NFR-16a governance. Batch jobs and DR bootstrap events are committed to the on-prem audit log before the S3 Vectors API call.
|
||||
|
||||
**Data Lifecycle Between Tiers**:
|
||||
- Ingress: New embeddings are written to pgvector (Tier 1, immediate query availability) AND S3 Vectors (Tier 2, archival) at creation time — a single FastAPI handler fans out writes.
|
||||
- Qdrant promotion (Phase 2 only): A scheduled job reads hot guideline vectors from pgvector and promotes to Qdrant when corpus size or QPS thresholds are crossed. Not active in PoC.
|
||||
- Demotion: Weekly batch job removes session embeddings from pgvector's hot HNSW index (they are retained in a separate pgvector table for archival within Postgres, not deleted). S3 Vectors receives the full weekly batch.
|
||||
- Disaster recovery (Qdrant loss — Phase 2): Admin triggers reload from S3 Vectors snapshot → rebuilds Qdrant index. In PoC: if Postgres is lost, restore from WAL + S3 Vectors snapshot of guideline embeddings.
|
||||
- Guideline update: New MOH version → embeddings written to pgvector (new HNSW index version, old index retained), S3 Vectors (archived as `guideline_v2025.06`). Postgres NOTIFY invalidates Redis guideline cache.
|
||||
- **Pattern**: LLM Hallucination Arbiter (Cloud Correction Tier)
|
||||
- **Source**: UC-74821 (BERT Drift Monitor triggers arbiter escalation); NFR-16a (governed cloud fallback)
|
||||
- **Application**: When the BERT drift monitor (UC-74821) detects semantic impasse, logical contradiction, or contextual hallucination during the Socratic dialogue between UP5 and the on-prem GemmaE2B, the system escalates the dialogue transcript to a cloud-hosted LLM arbiter (GemmaE2B on GCP Vertex AI, NFR-16a governed). The cloud arbiter acts as an unbiased independent corrector: it reviews the local model's statements against the retrieved MOH guideline chunks (from pgvector via RAG), corrects factual errors, resolves contradictions, and issues a grounded, cited response. This prevents the radiologist from accepting a locally-generated hallucination when the on-prem drift monitor flags risk. Invocation conditions: (1) BERT drift score exceeds threshold, OR (2) radiologist explicitly requests "second opinion" during consult. All cloud arbiter calls: require prior audit-log commit + user consent + Decree 13 redaction, identical to NFR-16a Tier 3a inference path.
|
||||
|
||||
- **Pattern**: Citation Contestant Validation (RAG-Referee Extended)
|
||||
- **Source**: NFR-18 (MOH Guideline-Anchored RAG), NFR-10 (Generative Safety Guardrails)
|
||||
- **Application**: RAG-Referee is a BERT-based text classifier that performs multi-dimensional validation of every LLM-generated explanation before it reaches the radiologist. Validation axes: (1) **Attribution correctness**: is each cited MOH guideline chunk actually retrieved by the RAG query? (2) **Logical cohesion**: does the explanation's reasoning chain remain factually consistent with the cited sources? (3) **Factual contestant status**: does the LLM claim contradict any retrieved guideline? If validation fails on any axis → reject LLM output → fallback to deterministic MOH template (Tier 3b safety floor). This extends beyond simple "citation present" to "citation contested and coherent."
|
||||
|
||||
- **Pattern**: Two-Tier Embedding Architecture
|
||||
- **Source**: A_Cache_Handbook_for_SWE_Quang_Hoang.md (retrieval times) + Cloud_Design_pattern_Azure.md (Query Stack, line 371)
|
||||
- **Application**: Two distinct embedding models serve distinct paths. EmbeddingGemma (768-dim) generates RAG retrieval vectors for pgvector. BERT (BioClinicalBERT/PubMedBERT, 768-dim) is used EXCLUSIVELY for drift monitoring and RAG-Referee classification — never written to the pgvector retrieval index. Same vector dimensionality enables future cross-compatibility, but the models remain architecturally separate with different failure modes, retrain schedules, and query paths.
|
||||
|
||||
### 5.6 Human-in-the-Loop (HITL)
|
||||
- **Pattern**: Manual Approval Gate
|
||||
- **Source**: google_infrastructure_whitepaper_fa.md (Two-Person Rule, line 133)
|
||||
- **Application**: Before any AI-generated diagnosis/treatment plan is marked FINALIZED, require digital signature from licensed clinician (NFR-19).
|
||||
|
||||
### 5.7 Edge Guardrail Tier (LLM Behavior Control Without Heavy Server-Side Frameworks)
|
||||
- **Pattern**: Constrained Prompt Engineering + Edge BERT Detection + Session Termination to Cloud Mitigation
|
||||
- **Source**: NFR-10 (Automated Generative Safety Guardrails); NFR-16a (Governed Cloud Fallback)
|
||||
- **Application**: NeMo Guardrails and GuardrailsAI require substantial runtime scaffolding, rule engines, and WASM-compatible bindings that do not exist in the browser ecosystem at the model inference layer. The client-side cannot run a NeMo Rail or GuardrailsAI pipeline inside a WebWorker alongside WebLLM. Therefore, the edge guardrail tier relies on two lightweight mechanisms:
|
||||
1. **Robust prompt-based rules**: Hard system-prompt constraints injected at every consult turn. Boundaries: no diagnosis outside MOH protocol scope; mandatory citation for every clinical assertion; no free-text echoing of PII even if user inputs it; immediate refusal to follow instructions that attempt prompt injection or jailbreak; tone and scope locked to clinical decision-support.
|
||||
2. **BERT hallucination / mal-intention detection**: Each candidate LLM token stream (or per-turn summary) is evaluated by a local Transformers.js BERT classifier running in a dedicated `guardrail.worker.ts` WebWorker. The classifier scores for: (a) hallucination probability (factual grounding vs retrieved MOH context); (b) mal-intention / prompt-injection indicators (user-context leaking, instruction override attempts, jailbreak markers); (c) scope violation (claims outside synovitis/MSK MOH guidelines).
|
||||
- **Session termination and cloud mitigation**: If BERT edge guardrail returns a violation score above threshold on any axis → FastAPI immediately terminates the WebLLM session (`stream.close()`), logs the trigger event to immutable audit, and opens a new mitigated consult session toward Tier 3a (cloud Vertex AI MedGemma). Mitigation path: the same user query + retrieved MOH chunks (already computed) + edge-detected-offense log are forwarded under NFR-16a governance (consent token confirmed, Decree 13 redaction re-applied, audit-log commit before egress). The radiologist sees: "Edge model restricted — escalating to cloud consult (redacted)."
|
||||
- **WebWorker isolation**: `guardrail.worker.ts` runs in a separate WebWorker from `cv.worker.ts` (LiteRT) and `llm.worker.ts` (WebLLM). No shared WASM memory between workers. IndexedDB caches artifacts but is never injected into LLM context. Unload order on memory pressure: `llm.worker.ts` → `guardrail.worker.ts` → `cv.worker.ts`.
|
||||
- **Cloud model guardrail**: Cloud Vertex AI MedGemma (Tier 3a/3b) uses natively supported Vertex AI Model Garden safety filters (tuned for healthcare) plus the same server-side FastAPI BERT RAG-Referee gate. No additional client-side framework required.
|
||||
|
||||
### 5.8 Edge Data Hygiene: Anonymization, Redaction & Server-Side Ground-Check
|
||||
- **Pattern**: Edge-to-Server Redaction Pipeline with Ground-Truth Verification
|
||||
- **Source**: NFR-16a (Decree 13 PII redaction before egress); NFR-15 (Circular 46 EMR compliance)
|
||||
- **Client-side edge redaction** (browser, before any bytes leave the device):
|
||||
- Libraries: `OpenRedaction` (regex + ML-assisted PII detection in Vietnamese/English clinical text), `pii-filter` (fast deterministic replacement), `js-data-anonymizer` (consistent i-agent pseudonymization for cross-session reference integrity).
|
||||
- Scope: patient name, DOB, MRN, address, phone, insurance ID, national ID, facial geometry, DICOM patient metadata fields (PatientName, PatientID, PatientBirthDate, PatientAddress, etc.).
|
||||
- Replacement: Role-hash token [BN:7f3a2c] or type-preserving stub (e.g., `AGE: 45` retained, `NAME: <PATIENT>` replaced). Non-identifying clinical fields (age, sex, joint site, findings, measurement values) preserved for inference utility.
|
||||
- Trigger: every text field, DICOM header, metadata JSON object, and JSON-LD payload before network write.
|
||||
- **Server-side re-verification + refinement** (FastAPI, inside hospital LAN, before Vertex AI egress):
|
||||
- The FastAPI `presidio.redaction` middleware (Microsoft Presidio AnalyzerEngine + AnonymizerEngine) re-verifies the client redaction manifest and **actively refines/continues cleaning** any residual PII that the edge missed.
|
||||
- If Presidio successfully cleans the remaining PII → replace the payload with the refined version and continue to the selected inference tier.
|
||||
- If Presidio cannot fully clean the payload (unstructured residual PII it cannot resolve) → halt the pipeline, return a structured error to the client, and log `redaction_failure` event to the immutable audit log.
|
||||
- **Stable pseudonyms for analytics**: `js-data-anonymizer` generates stable i-agent pseudonyms (same patient → same token within a session and across sessions via `session_hash` binding). This enables longitudinal audit linkage without PHI leakage.
|
||||
|
||||
### 5.9 RAG as Essential Pipeline Step (Non-Optional)
|
||||
- **Pattern**: Mandatory RAG Pre-Processing for All LLM Consult Paths
|
||||
- **Source**: NFR-18 (100% LLM text cites MOH protocol); NFR-12 (zero-friction explainability)
|
||||
- **Application**: RAG retrieval is NOT an optional tool the LLM may choose to invoke. It is a mandatory pipeline step that executes before any LLM generation, for both edge (browser WebLLM) and cloud (Vertex AI MedGemma) tiers.
|
||||
- **Pipeline order**:
|
||||
1. User query arrives (post-redaction)
|
||||
2. `rag.query()` retrieves top-k MOH guideline chunks from pgvector HNSW
|
||||
3. Retrieved chunks + strict system prompt ("You are a clinical decision-support assistant. Answer ONLY using the provided MOH guidelines. Cite chunk IDs. Do not speculate.") → injected into LLM context
|
||||
4. LLM generates grounded, cited response
|
||||
5. `referee.validate()` (BERT RAG-Referee) checks citation correctness, logical cohesion, factual contestant status
|
||||
6. If any axis fails → reject → fallback to MOH template
|
||||
- **Rationale**: If RAG were merely a tool call, a compromised or under-performing edge LLM could choose to skip retrieval and generate ungrounded text, violating NFR-18. Mandatory RAG pre-processing closes this vector.
|
||||
|
||||
### 5.10 Tool-Calling Semantics for Edge and Cloud LLMs
|
||||
- **Pattern**: Function-Calling as Convenience Layer Over Mandatory RAG
|
||||
- **Source**: Gemma Functions (Google DeepMind) + Vertex AI Function Calling
|
||||
- **Application**:
|
||||
- **Edge LLM (GemmaE2B via WebLLM)**: Supports Gemma Functions. The RAG retrieval is exposed as a function tool (`retrieve_moh_guidelines(query, top_k)`). HOWEVER, this tool is pre-invoked by the host pipeline per §5.9 before the first LLM token is generated. The function-calling interface is available for subsequent turns of Socratic dialogue where the model may need additional retrieval mid-conversation (e.g., user asks "what about contraindications?" → mid-turn retrieval). The host pipeline enforces the mandatory RAG call on turn 0; subsequent per-turn retrieval is model-initiated via function calling.
|
||||
- **Cloud LLM (MedGemma on Vertex AI)**: Native Vertex AI Function Calling is enabled. Same semantics: server-side FastAPI injects top-k MOH chunks before first generation; model may request further retrieval via function call during multi-turn consult.
|
||||
- **Why not pure tool-calling**: Pure tool-calling delegates the decision to retrieve to the model. NFR-18 requires 100% citation; delegating retrieval to model choice creates a non-zero probability of unsupported generation. Mandatory pre-injection + optional per-turn tool call is the safer contract.
|
||||
|
||||
### 5.11 Output Filtering
|
||||
- **Pattern**: Multi-Axis Output Gate
|
||||
- **Server-side FastAPI**:
|
||||
- Regex + ladybugDB entity link validation: reject outputs containing unlinked named entities not present in the source context.
|
||||
- BERT RAG-Referee (§5.5 pattern extended): citation contestant validation runs on every LLM output before SSE streaming to PWA.
|
||||
- Refusal pattern detection: if output contains model-generated refusal of form "I cannot answer" without citing a specific MOH section → flag as incomplete grounding, fallback to template.
|
||||
- **Client-side PWA final gate** (before rendering in primary viewport):
|
||||
- PWA JavaScript post-processor strips any residual PII that escaped server-side checks (defense-in-depth).
|
||||
- Citation footnote auto-format: if RAG-Referee returned validated citations, render as superscript links to MOH chunk IDs.
|
||||
- **NFR-10 compliance**: <90% verification score → output is prohibited from display. The Circuit Breaker feeds back to the user: "Unable to provide verified explanation — please consult senior expert."
|
||||
|
||||
### 5.12 Logging, Retention & Model Telemetry
|
||||
- **Pattern**: Immutable Append-Only Audit + Model Behavior Telemetry
|
||||
- **Audit events** (append-only Postgres table, NFR-17):
|
||||
- `edge_guardrail_violation`: session_hash, violation_axis (hallucination|mal_intention|scope_breach), bert_score, user_query_hash, mitigation_target (cloud|template).
|
||||
- `rag_retrieval_event`: session_hash, query_hash, retrieved_chunk_ids, latency_ms.
|
||||
- `referee_decision`: session_hash, axis_results (attribution|cohesion|contestant), overall_pass, fallback_triggered.
|
||||
- **Audit events for this layer**: `redaction_edge_manifest` (client), `redaction_server_refine` (server-side cleaning pass), `redaction_failure` (server unable to clean → error), `egress_consent` + `egress_redact_manifest` (before cloud call, NFR-16a).
|
||||
- **Retention**: Audit events retained indefinitely (append-only, no UPDATE/DELETE). Raw LLM token streams retained for 90 days in MinIO (compressed, AES-256 at rest) for drift analysis. After 90 days, streams are deleted; only aggregated `referee_decision` and `rag_retrieval_event` summaries remain.
|
||||
- **BERT drift monitor telemetry**: edge BERT classifier score distributions exported to Prometheus; alert if false-positive rate on hallucination detection exceeds 5% or false-negative rate exceeds 0.1% (clinical safety threshold).
|
||||
- **Model weight distribution audit**: every GCP CDN signed-URL fetch event committed to audit log (case_hash, timestamp, consult_mode).
|
||||
|
||||
### 5.13 IndexedDB Schema for Guardrail State
|
||||
- **Pattern**: Client-Side State Persistence with Manual Invalidation
|
||||
- **Database**: `vkist_msk` (Dexie.js)
|
||||
- **Tables**:
|
||||
- `user_prefs`: existing — language, UI density, contrast.
|
||||
- `guardrail_models`: model artifact cache (WebLLM weights hash, BERT classifier version, manifest timestamps).
|
||||
- `policy_config`: server-pushed guardrail policy (prompt rule version, BERT score thresholds, redaction rule version) — updated on admin change, invalidated by version key.
|
||||
- `audit_tokens`: stable pseudonym tokens generated by `js-data-anonymizer` for cross-session reference (replaces Redis for this data type).
|
||||
- **Not cached in Redis** (consistent with §5.2 design):
|
||||
- Guardrail policy config and model artifacts are client state; invalidated only by explicit admin action or version mismatch. Server-side source of truth for policy version is Postgres `policy_versions` table.
|
||||
|
||||
## 6. Trade-Off Analysis
|
||||
|
||||
| Criterion | Choice | Trade-Off Justification |
|
||||
|--------------------|---------------------------------------------|---------------------------------------------------------------------------------------|
|
||||
| **Latency vs. Consistency** | Eventual consistency with read-through cache | Strong consistency would increase latency; cache-aside with short TTL meets NFR-1,5,7 while keeping data fresh enough for clinical use. |
|
||||
| **Edge vs. Centralized** | Hybrid edge for inference, central for analytics | Pure edge lacks resources for complex RAG; pure central violates NFR-5,16. Edge handles real-time inference at edge, heavier analytics centrally. |
|
||||
| **SQL vs. NoSQL** | Polyglot persistence (PostgreSQL + Redis + MinIO) | Relational for transactions and vector search (PostgreSQL + pgvector), caching (Redis), object storage (MinIO). Single DB type cannot optimally serve all needs — but pgvector collapses the vector-store decision into Postgres, avoiding a separate Qdrant service at PoC scale. |
|
||||
| **Monolith vs. Microservices** | Microservices with API Gateway | Increased operational overhead vs. independent scaling, fault isolation, and technology diversity (required for ML vs. web services). |
|
||||
| **Public Cloud vs. On-Prem** | On-premise Kubernetes (e.g., Rancher, K3s) for production; GCP Vertex AI permitted only under NFR-16a (PoC emergency fallback, redacted payloads only) | NFR-16 prohibits public cloud in normal operation. NFR-16a creates a governed, time-limited exception for PoC validation: cloud LLM may supplement on-prem tiers only after Decree 13 redaction and explicit user consent. On-premise provides control for production; NFR-16a provides a pragmatic safety net during PoC, with a mandatory retirement trigger upon PoC sign-off. |
|
||||
| **Sync vs. Async UI Updates** | Asynchronous token streaming (WebSockets) | Synchronous would block UI; async streaming meets NFR-7 (≤200ms TTFT) and keeps UI responsive. |
|
||||
|
||||
## 7. Conclusion
|
||||
|
||||
The proposed architecture satisfies all functional and non-functional requirements by combining cloud-native patterns with strict on-premise deployment to meet data sovereignty and air-gap mandates. Key innovations include edge inference for low-latency DICOM processing, immutable audit logs for accountability, RAG for explainable AI, and HITL gatekeeping for clinical safety. The architecture is scalable, maintainable, and aligns with the solution_architect_corpus patterns as detailed above. NFR-16a provides a time-limited, governed bridge for PoC validation only, with a mandatory retirement trigger upon PoC user sign-off — restoring NFR-16 as the permanent production constraint.
|
||||
|
||||
|
||||
## 8. Architecture Diagrams
|
||||
|
||||
### 8.1 Context Diagram (C4) - Sprint 1-2 Scope (FR-25 Synovitis Grading)
|
||||
```plantuml
|
||||
@startuml
|
||||
!include <C4/C4_Context>
|
||||
|
||||
title System Context Diagram - VKIST MSK Platform (Sprint 1-2)
|
||||
' Sprint 1-2 scope: FR-25 Synovitis Grading + Multi-Modal NLP Integration
|
||||
' Active Users: UP5, UP6 | Governance: UP1, UP4 | Future: UP7, UP8
|
||||
|
||||
' === ACTORS IN SPRINT SCOPE ===
|
||||
Person(radiologist, "Diagnostic Radiologist (UP5)", "FR-25: loads knee ultrasound, reviews AI grading (synovitis 0-3), confirms/overrides grade, finalizes & signs report, views GradCAM explanations, triggers circuit breaker, reviews RAG evidence")
|
||||
|
||||
' === GOVERNANCE / NFR ALIGNMENT (not in FR scope) ===
|
||||
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "NFR alignment: clinical validator, protocol governance, model threshold approval")
|
||||
Person_Ext(support_staff, "Support Staff (UP4)", "NFR alignment: patient registration, queue management")
|
||||
|
||||
' === FUTURE SPRINTS (not in current scope) ===
|
||||
Person_Ext(therapist, "Physical Therapist (UP6)", "Future sprint: prescription scanning, 3D joint guides, kinetic overlays")
|
||||
Person_Ext(ortho_surgeon, "Orthopedic Surgeon & Rheumatologist (UP7)", "Future sprint: treatment planning, aggregated dashboard")
|
||||
Person_Ext(patient_caregiver, "Patient & Family Caregiver (UP8)", "Future sprint: patient portal, self-monitoring, reminders")
|
||||
|
||||
Person(admin, "System Administrator", "Manages on-premise K3s deployment, model updates, observability dashboards, NGINX failover")
|
||||
|
||||
' === SYSTEM ===
|
||||
System(mpps, "VKIST MSK Processing System (FR-25)", "Sprint 1-2 scope:\n- Knee ultrasound AI analysis (angle → inflammation → segmentation)\n- Synovitis grading 0-3 with GradCAM explanations\n- Clinical safety: Conversational Circuit Breaker, BERT drift monitor, RAG-Referee\n- PDF report generation (Circular 46)\n- NLP: Decree 13 PII scrubbing, ladybugDB ontology, GemmaE2B/MedGemma explanations")
|
||||
|
||||
' === EXTERNAL SYSTEMS IN SCOPE ===
|
||||
System_Ext(pacs, "Hospital PACS / Ultrasound Machine", "Source of DICOM images (C-MOVE + direct device capture)")
|
||||
System_Ext(emr, "Hospital EMR/HIS", "Finalized report storage, prescription & lab sync (HL7/FHIR)")
|
||||
System_Ext(triton, "Triton Inference Server", "GPU node: 3-step ML pipeline (angle → inflammation → segmentation) + embedding extraction")
|
||||
System_Ext(knowledge, "Clinical Knowledge Systems", "ladybugDB (SNOMED-CT/LOINC graph), pgvector (MOH guideline vectors in Postgres HNSW), EmbeddingGemma (768-dim RAG embeddings), GemmaE2B/MedGemma (Vietnamese + clinical LLM: browser WebLLM local OR cloud Vertex AI)")
|
||||
|
||||
' === RELATIONSHIPS ===
|
||||
Rel(radiologist, mpps, "Loads scan, reviews AI grading, confirms/overrides grade, finalizes report, views explanations, engages circuit breaker dialog", "HTTPS")
|
||||
Rel(admin, mpps, "Deploys, monitors, updates models", "HTTPS/SSH")
|
||||
|
||||
Rel(senior_expert, mpps, "Validates clinical protocols, approves model thresholds", "HTTPS (Remote Governance)")
|
||||
Rel(support_staff, mpps, "Patient registration, case queue management", "HTTPS")
|
||||
|
||||
Rel(therapist, mpps, "Not in Sprint 1-2 scope — future FRs", "—")
|
||||
Rel(ortho_surgeon, mpps, "Not in Sprint 1-2 scope — future FRs", "—")
|
||||
Rel(patient_caregiver, mpps, "Not in Sprint 1-2 scope — future FRs", "—")
|
||||
|
||||
Rel(mpps, pacs, "Retrieves/imports DICOM images", "DICOM/C-MOVE + direct upload")
|
||||
Rel(mpps, emr, "Finalized reports, audit logs, ground-truth records", "HL7/FHIR")
|
||||
Rel(mpps, triton, "ML inference offload", "gRPC (Port 8001)")
|
||||
Rel(mpps, knowledge, "RAG queries, ontology traversal, LLM generation", "SQL (pgvector) + in-process C++")
|
||||
|
||||
note right of mpps
|
||||
**Sprint 1-2 Scope Boundary:**
|
||||
• FR-25: Synovitis Grading (UC-48376 → UC-47988 → UC-92006)
|
||||
• NLP: Decree 13 scrubbing, GemmaE2B/MedGemma explanations
|
||||
• Safety: Circuit Breaker, BERT drift, RAG-Referee
|
||||
• Audit: High-Trust Concur, Ground-Truth Commit
|
||||
• Reporting: Circular 46 PDF export
|
||||
|
||||
**Excluded (Future Sprints):**
|
||||
• UP6: Prescription parser, kinetic overlay, 3D mapping
|
||||
• UP7: Treatment planning, aggregated dashboards
|
||||
• UP8: Patient portal, self-monitoring, reminders
|
||||
end note
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 8.2 Container Diagram (C4) - Sprint 1-2 Implementation
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_System_Architecture_Containers_Sprint1_2"
|
||||
!include <C4/C4_Container>
|
||||
|
||||
title C4 Container Diagram - VKIST MSK Platform (Sprint 1-2)
|
||||
|
||||
Person(radiologist, "Image Radiologist (UP5)", "Performs primary diagnosis, view validation, and severity grading.")
|
||||
Person(therapist, "Physical Therapist (UP6)", "Observes scans, evaluates kinetic data, and structures physical rehabilitation plans.")
|
||||
|
||||
System_Boundary(hospital_lan, "Air-Gapped Hospital LAN Boundary (Max 10 Mbps)") {
|
||||
|
||||
Container(pwa, "React PWA Frontend Client", "React, TS, Zustand, LiteRT, MediaPipe", "Provides interactive visualization, runs local WebAssembly LiteRT/MediaPipe models for view-angle validation, and encrypts data.")
|
||||
|
||||
ContainerDb(indexeddb, "Local Browser Storage (IndexedDB)", "Dexie.js Sandbox", "Caches encrypted patient sessions, DICOM images, and compiled annotation layers locally to avoid redundant LAN fetches.")
|
||||
|
||||
Container(nginx, "Active-Passive Gateway (NGINX + Keepalived)", "Reverse Proxy", "Provides single Virtual IP (VIP) load balancing, SSL/TLS termination, and instant failover (<=2s switch).")
|
||||
|
||||
System_Boundary(backend_servers, "Application Server Cluster") {
|
||||
|
||||
Container(fastapi, "FastAPI Application Server", "Python, Uvicorn, SQLAlchemy", "Orchestrates API routers, role checks, and Socratic circuit-breaker state evaluations.")
|
||||
|
||||
System_Boundary(modal_iac, "Modal Infra-as-Code") {
|
||||
Container(triton, "Triton Inference Server", "NVIDIA Triton, ONNX, OpenVINO, TensorRT (deployed via Modal with FastAPI proxy)", "Executes heavy 3-step ML pipeline (detection, UNet3+ segmentation, scoring) and runs embedding extraction.")
|
||||
}
|
||||
|
||||
System_Boundary(observability, "Observability Stack (Prometheus + Grafana)") {
|
||||
Container(prometheus, "Prometheus Metrics Store", "Prometheus", "Scrapes server, API, and inference metrics; stores time-series performance statistics for dashboards and alerting.")
|
||||
Container(grafana, "Grafana Dashboard", "Grafana", "Displays local dashboards for request latency, Triton VRAM/GPU utilization, inference latency, cache hit rate, and server uptime.")
|
||||
}
|
||||
|
||||
System_Boundary(info_stack, "Transactional Information Stack") {
|
||||
ContainerDb(postgres, "Central EMR Database (PostgreSQL + PostGIS)", "PostgreSQL Core", "Stores spatial anatomical markers, structured EMR histories, user profiles, S3 object keys, checksums, and the immutable audit ledger.")
|
||||
Container(s3, "Clinical Image Object Store (MinIO S3-compatible)", "MinIO", "Persists raw DICOM-derived image payloads, inference inputs, segmentation overlays, Grad-CAM heatmaps, and finalized report blobs.")
|
||||
ContainerDb(redis, "In-Memory Cache", "Redis (Localhost Cluster)", "Manages shared session variables, real-time JWT token state, and connection rate parameters.")
|
||||
}
|
||||
|
||||
System_Boundary(knowledge_stack, "Clinical Knowledge Stack (GraphRAG)") {
|
||||
ContainerDb(ladybugDB, "Ontology Graph Database (ladybugDB)", "C++ Embedded DB", "Stores structured relationship links, anatomical entities, and treatment pathways (Embedded inside FastAPI).")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Rel(radiologist, pwa, "Interacts with MSK workspace, views overlays, and saves grades")
|
||||
Rel(therapist, pwa, "Views scan results, annotates clinical feedback, and records rehabilitation plans")
|
||||
|
||||
Rel(pwa, indexeddb, "Reads/Writes cached and encrypted local files (Dexie.js)", "Local I/O")
|
||||
Rel(pwa, nginx, "Sends HTTP requests and pre-validated coordinate arrays over SSL", "HTTPS (Port 443)")
|
||||
|
||||
Rel(nginx, fastapi, "Routes API requests & routes stream paths to active server node", "HTTP/1.1 (Port 8000)")
|
||||
Rel(fastapi, redis, "Verifies session state and retrieves connection tokens", "TCP (Port 6379)")
|
||||
Rel(fastapi, postgres, "Inserts metadata, object keys, checksums, and queries relational/spatial schemas; dense semantic vector lookups for guidelines text (pgvector HNSW)", "SQL/TCP (Port 5432)")
|
||||
Rel(fastapi, s3, "Reads/writes image and report blobs", "S3 API (Private Bucket)")
|
||||
|
||||
Rel(prometheus, nginx, "Scrapes gateway health and upstream timing metrics", "HTTP /metrics (Port 9113)")
|
||||
Rel(prometheus, fastapi, "Scrapes request latency, SSE throughput, error rate, and DB pool metrics", "HTTP /metrics (Port 8000)")
|
||||
Rel(prometheus, triton, "Scrapes model latency, GPU utilization, VRAM, and batching metrics", "HTTP /metrics (Port 8002)")
|
||||
Rel(grafana, prometheus, "Queries metrics for operational dashboards", "HTTP (Port 9090)")
|
||||
|
||||
Rel(fastapi, triton, "Offloads heavy tasks & embeds retrieval vectors", "gRPC (Port 8001)")
|
||||
Rel(fastapi, ladybugDB, "Queries clinical guidelines ontology and entity relationships", "In-Process C++ Bindings")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 8.3 Component Diagram (C4) - Edge Inference Service Internals
|
||||
```plantuml
|
||||
!define https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master
|
||||
!includeurl /C4_Component.puml
|
||||
|
||||
Container_Boundary(boundary, "Edge Inference Service") {
|
||||
Container(edge_inference, "Edge Inference Service", "FastAPI (Python)", "Handles image upload, runs ML models")
|
||||
Component(api_controller, "API Controller", "Python/FastAPI", "REST endpoint /api/analyze")
|
||||
Component(preprocessor, "Image Preprocessor", "Python/OpenCV", "CLAHE, resizing, normalization")
|
||||
Component(angle_model, "Angle Classification Model", "PyTorch (ConvNeXt/DenseNet/etc.)", "Predicts imaging plane")
|
||||
Component(inflammation_model, "Inflammation Detection Model", "PyTorch (EfficientNet-B0)", "Binary inflammation presence")
|
||||
Component(segmentation_model, "Segmentation Model", "PyTorch (UNet/DeepLabV3)", "Pixel-wise mask for anatomical structures")
|
||||
Component(measurement_engine, "Measurement Engine", "Python/NumPy", "Calculates synovium thickness in mm and effusion metrics")
|
||||
Component(severity_analyzer, "Severity Analyzer", "Python", "Determines synovitis level (0-3) and generates explanation")
|
||||
Component(report_generator, "Report Generator", "Python", "Builds final JSON response, optional PDF")
|
||||
}
|
||||
|
||||
Rel(api_controller, preprocessor, "Calls", "Sync")
|
||||
Rel(preprocessor, angle_model, "Calls", "Sync")
|
||||
Rel(angle_model, api_controller, "Returns angle class & confidence", "Sync")
|
||||
Rel(api_controller, inflammation_model, "Calls (if needed)", "Sync")
|
||||
Rel(inflammation_model, api_controller, "Returns inflammation flag & confidence", "Sync")
|
||||
Rel(api_controller, segmentation_model, "Calls (if inflammation)", "Sync")
|
||||
Rel(segmentation_model, api_controller, "Returns segmentation masks", "Sync")
|
||||
Rel(api_controller, measurement_engine, "Calls", "Sync")
|
||||
Rel(measurement_engine, api_controller, "Returns thickness & bbox", "Sync")
|
||||
Rel(api_controller, severity_analyzer, "Calls", "Sync")
|
||||
Rel(severity_analyzer, api_controller, "Returns severity level, description, color", "Sync")
|
||||
Rel(api_controller, report_generator, "Calls", "Sync")
|
||||
Rel(report_generator, api_controller, "Returns final JSON (with optional PDF bytes)", "Sync")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 8.4 Deployment Diagram (C4 Deployment View)
|
||||
```plantuml
|
||||
!define https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master
|
||||
!includeurl /C4_Deployment.puml
|
||||
|
||||
Deployment_Node(hw_node, "Hospital On-Premise Hardware (e.g., Dell PowerEdge)") {
|
||||
Deployment_Node(k8s_cluster, "Kubernetes Cluster (K3s)") {
|
||||
Deployment_Node(pod_inference, "Pod: edge-inference-svc") {
|
||||
Container(container_inf, "edge-inference-svc", "FastAPI container")
|
||||
}
|
||||
Deployment_Node(pod_rag, "Pod: rag-svc") {
|
||||
Container(container_rag, "rag-svc", "FastAPI container")
|
||||
}
|
||||
Deployment_Node(pod_audit, "Pod: audit-svc") {
|
||||
Container(container_audit, "audit-svc", "Node.js container")
|
||||
}
|
||||
Deployment_Node(pod_api_gw, "Pod: api-gateway") {
|
||||
Container(container_gw, "envoy", "Envoy proxy")
|
||||
}
|
||||
Deployment_Node(pod_auth, "Pod: auth-svc") {
|
||||
Container(container_auth, "keycloak", "Keycloak container")
|
||||
}
|
||||
Deployment_Node(pod_obs, "Pod: observability") {
|
||||
Container(container_prom, "prometheus", "Prometheus server")
|
||||
Container(container_graf, "grafana", "Grafana UI")
|
||||
Container(container_elk, "elk", "Elasticsearch, Logstash, Kibana")
|
||||
}
|
||||
}
|
||||
Deployment_Node(vm_db, "Database VM") {
|
||||
ContainerDb(db_primary, "PostgreSQL + pgvector", "Primary DB + Vector (HNSW)", "Stores spatial anatomical markers, structured EMR histories, user profiles, S3 object keys, checksums, immutable audit ledger, and pgvector HNSW index for MOH guideline embeddings and case embeddings.")
|
||||
ContainerDb(cache, "Redis", "Cache")
|
||||
ContainerDb(object_store, "MinIO", "Object storage")
|
||||
}
|
||||
Deployment_Node(aws_cloud, "AWS Cloud (ap-southeast-1, NFR-16a only)") {
|
||||
ContainerDb(s3_vectors, "S3 Vectors", "AWS Vector Bucket", "Cold archival storage for historical guideline versions and billion-scale case embedding archive. Disaster recovery bootstrap source. Accessed under NFR-16a governance with audit logging.")
|
||||
}
|
||||
}
|
||||
|
||||
Rel(db_primary, cache, "Replication", "TCP")
|
||||
Rel(db_primary, object_store, "Backup", "TCP")
|
||||
Rel(k8s_cluster, vm_db, "Accesses (SQL)", "TCP")
|
||||
Rel(k8s_cluster, aws_cloud, "Batch archival + DR bootstrap (NFR-16a)", "HTTPS (via AWS SDK)")
|
||||
Rel(hw_node, k8s_cluster, "Runs on", "Network")
|
||||
@enduml
|
||||
```
|
||||
|
||||
### 8.5 ML Workflow Flowchart
|
||||
```plantuml
|
||||
@startuml
|
||||
title ML Processing Workflow for Knee Ultrasound
|
||||
start
|
||||
:Upload DICOM image via PWA;
|
||||
:Apply CLAHE preprocessing;
|
||||
:Run Angle Classification Model;
|
||||
if (Angle == post-transverse?) then (yes)
|
||||
:Run Inflammation Detection Model;
|
||||
if (Inflammation detected?) then (yes)
|
||||
:Run Posterior Segmentation Model;
|
||||
:Generate Segmentation Masks;
|
||||
:Measure Synovium Thickness;
|
||||
:Calculate Effusion Metrics;
|
||||
:Analyze Severity (0-3);
|
||||
else (no)
|
||||
:Skip segmentation and measurement;
|
||||
:Set severity = 0;
|
||||
endif
|
||||
else (no)
|
||||
:Run Suprapatellar Segmentation Model;
|
||||
:Generate Segmentation Masks;
|
||||
:Measure Hoffa Fat Pad Thickness;
|
||||
:Analyze Severity (0-3);
|
||||
endif
|
||||
:Retrieve relevant guidelines from pgvector (RAG);
|
||||
:Generate explanation with citations;
|
||||
:Assemble final JSON response (including overlay images);
|
||||
@enduml
|
||||
```
|
||||
## References
|
||||
- [Solution Architect Corpus](./.kilocode/skills/solution_architect/assets/solution_architect_corpus/)
|
||||
- [Project FR Database](./proj_level_reading/Requirement_Analysis/FRs_Database/FR-Engineer-DB .csv)
|
||||
- [Project NFR Database](./proj_level_reading/Requirement_Analysis/Non_FRS_DB/Non_Functional_(NFR)_List_Result/Non-FR Engineering Result 376f910aea7580c3ae9afed6937701fd_all.csv)
|
||||
- [Existing Component Specs](./workspace/sprint_1_2/CODEBASE/*/spec/)
|
||||
- [Sprint 1-2 Architecture Spec](./workspace/sprint_1_2/SPRINT_1_2_ARCHITECTURE_SPEC.md) - Sprint-specific context, container, component, and deployment diagrams
|
||||
@@ -0,0 +1,26 @@
|
||||
# Stage 1: Build stage
|
||||
FROM python:3.12 AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install dependencies into a virtual environment to make copying easy
|
||||
RUN python -m venv /opt/venv
|
||||
ENV PATH="/opt/venv/bin:$PATH"
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Stage 2: Final Runtime stage
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the pre-compiled Python packages from the builder stage
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
COPY . .
|
||||
|
||||
# Ensure the virtual environment is used
|
||||
ENV PATH="/opt/venv/bin:$PATH"
|
||||
|
||||
EXPOSE 8000
|
||||
CMD ["python", "main.py"]
|
||||
@@ -0,0 +1,852 @@
|
||||
from fastapi import FastAPI, File, UploadFile, HTTPException, Query, Request, Response
|
||||
from pydantic import BaseModel
|
||||
from fastapi.responses import JSONResponse, FileResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
import os
|
||||
import uuid
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
import torchvision.models as models
|
||||
from torchvision import transforms
|
||||
import uvicorn
|
||||
from pathlib import Path
|
||||
import cv2
|
||||
import io
|
||||
import base64
|
||||
from datetime import datetime
|
||||
import re
|
||||
|
||||
# Import custom models
|
||||
import sys
|
||||
sys.path.append('.')
|
||||
from arch.efficientfeedback import EfficientFeedbackNetwork
|
||||
from arch.unet3plus_att import UNet3Plus_Attention
|
||||
from pdf_service import generate_medical_report
|
||||
|
||||
# Configuration
|
||||
UPLOAD_FOLDER = 'uploads'
|
||||
RESULTS_FOLDER = 'results'
|
||||
TEMPLATES_FOLDER = 'templates'
|
||||
|
||||
# for folder in [UPLOAD_FOLDER, RESULTS_FOLDER, TEMPLATES_FOLDER]:
|
||||
# os.makedirs(folder, exist_ok=True)
|
||||
os.makedirs(TEMPLATES_FOLDER, exist_ok=True)
|
||||
|
||||
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||
|
||||
# Classes
|
||||
ANGLE_CLASSES = ['med-lat', 'post-trans', 'sup-trans-flex', 'sup-up-long']
|
||||
SEGMENT_CLASSES_SUPRAPAT = {0: "background", 1: "effusion", 2: "fat", 3: "fat-pat", 4: "femur", 5: "synovium", 6: "tendon"}
|
||||
SEGMENT_CLASSES_POST = {0: "background", 1: "fat", 2: "tendon", 3: "muscle", 4: "femur", 5: "artery", 6: "baker's cyst"}
|
||||
|
||||
# Color map for Suprapat
|
||||
COLOR_MAP_SUP = {
|
||||
'background': {'color': [0, 0, 0], 'name': 'Nền'},
|
||||
'effusion': {'color': [255, 0, 0], 'name': 'Dịch khớp'},
|
||||
'fat': {'color': [255, 255, 0], 'name': 'Mỡ'},
|
||||
'fat-pat': {'color': [0, 255, 255], 'name': 'Mỡ Hoffa'},
|
||||
'femur': {'color': [0, 255, 0], 'name': 'Xương đùi'},
|
||||
'synovium': {'color': [255, 0, 255], 'name': 'Màng hoạt dịch'},
|
||||
'tendon': {'color': [0, 0, 255], 'name': 'Gân'}
|
||||
}
|
||||
|
||||
# Color map for Post-trans
|
||||
COLOR_MAP_POST = {
|
||||
'background': {'color': [0, 0, 0], 'name': 'Nền'},
|
||||
"baker's cyst": {'color': [255, 0, 0], 'name': "Baker's cyst"},
|
||||
'fat': {'color': [255, 255, 0], 'name': 'Mỡ'},
|
||||
'muscle': {'color': [0, 255, 255], 'name': 'Cơ bắp'},
|
||||
'femur': {'color': [0, 255, 0], 'name': 'Xương đùi'},
|
||||
'synovium': {'color': [255, 0, 255], 'name': 'Màng hoạt dịch'},
|
||||
'tendon': {'color': [0, 0, 255], 'name': 'Gân'}
|
||||
}
|
||||
|
||||
# Measurement configuration
|
||||
DEFAULT_MEASURE_IDS = [1, 5]
|
||||
PIXEL_TO_MM = 45.0 / 655.0
|
||||
|
||||
app = FastAPI(title="Medical Image Analysis API")
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# app.mount("/uploads", StaticFiles(directory=UPLOAD_FOLDER), name="uploads")
|
||||
# app.mount("/results", StaticFiles(directory=RESULTS_FOLDER), name="results")
|
||||
|
||||
# ============ MODEL LOADING ============
|
||||
|
||||
def load_angle_model(model_name):
|
||||
print(f"📄 Loading angle model: {model_name}")
|
||||
|
||||
if model_name == "convnext":
|
||||
model = models.convnext_tiny(weights=None)
|
||||
model.classifier[2] = nn.Linear(model.classifier[2].in_features, 4)
|
||||
checkpoint = torch.load(f"models/best_convnext_tiny.pth", map_location=device, weights_only=False)
|
||||
checkpoint = {k.replace('model.', ''): v for k, v in checkpoint.items()}
|
||||
model.load_state_dict(checkpoint)
|
||||
elif model_name == "densenet":
|
||||
model = models.densenet121(weights=None)
|
||||
model.classifier = nn.Linear(model.classifier.in_features, 4)
|
||||
checkpoint = torch.load(f"models/best_densenet.pth", map_location=device, weights_only=False)
|
||||
checkpoint = {k.replace('model.', ''): v for k, v in checkpoint.items()}
|
||||
model.load_state_dict(checkpoint)
|
||||
elif model_name == "resnet50":
|
||||
model = models.resnet50(weights=None)
|
||||
model.fc = nn.Linear(model.fc.in_features, 4)
|
||||
checkpoint = torch.load(f"models/best_resnet50.pth", map_location=device, weights_only=False)
|
||||
checkpoint = {k.replace('model.', ''): v for k, v in checkpoint.items()}
|
||||
model.load_state_dict(checkpoint)
|
||||
elif model_name == "efficientnet_b2":
|
||||
model = models.efficientnet_b2(weights=None)
|
||||
model.classifier[1] = nn.Linear(model.classifier[1].in_features, 4)
|
||||
checkpoint = torch.load(f"models/best_efficientnet_b2.pth", map_location=device, weights_only=False)
|
||||
checkpoint = {k.replace('model.', ''): v for k, v in checkpoint.items()}
|
||||
model.load_state_dict(checkpoint)
|
||||
elif model_name == "swin":
|
||||
model = models.swin_v2_s(weights=None)
|
||||
model.head = nn.Linear(model.head.in_features, 4)
|
||||
checkpoint = torch.load(f"models/best_swin_v2_s.pth", map_location=device, weights_only=False)
|
||||
checkpoint = {k.replace('model.', ''): v for k, v in checkpoint.items()}
|
||||
model.load_state_dict(checkpoint)
|
||||
else:
|
||||
raise ValueError(f"Unknown angle model: {model_name}")
|
||||
|
||||
print(f"✅ Loaded: {model_name}")
|
||||
return model.to(device).eval()
|
||||
|
||||
def load_inflammation_model():
|
||||
print("📄 Loading inflammation model")
|
||||
model = models.efficientnet_b0(weights=None)
|
||||
model.classifier[1] = nn.Linear(model.classifier[1].in_features, 2)
|
||||
model.load_state_dict(torch.load("models/efficientnet_b0_ultrasound_2_class.pth", map_location=device, weights_only=False))
|
||||
print("✅ Loaded inflammation model")
|
||||
return model.to(device).eval()
|
||||
|
||||
def load_segmentation_model_sup(model_name):
|
||||
print(f"📄 Loading segmentation model SUP: {model_name}")
|
||||
|
||||
if model_name == "deeplabv3":
|
||||
model = models.segmentation.deeplabv3_resnet50(weights=None)
|
||||
in_ch = model.classifier[-1].in_channels
|
||||
model.classifier = nn.Sequential(
|
||||
model.classifier[0],
|
||||
nn.Dropout(0.3),
|
||||
nn.Conv2d(in_ch, 7, kernel_size=1)
|
||||
)
|
||||
model.load_state_dict(torch.load("models/best_model_Deeplav3.pth", map_location=device, weights_only=False), strict=False)
|
||||
elif model_name == "unet_resnet101":
|
||||
try:
|
||||
from segmentation_models_pytorch import Unet
|
||||
model = Unet(encoder_name="resnet101", encoder_weights=None, classes=7)
|
||||
model.load_state_dict(torch.load("models/unet_resnet101.pth", map_location=device, weights_only=False))
|
||||
except ImportError:
|
||||
raise ValueError("segmentation_models_pytorch not installed")
|
||||
elif model_name == "efficientfeedback":
|
||||
model = EfficientFeedbackNetwork(in_channels=3, num_class=7)
|
||||
model.load_state_dict(torch.load("models/efficientfeedback.pth", map_location=device, weights_only=False))
|
||||
elif model_name == "unet3plus":
|
||||
model = UNet3Plus_Attention(in_channels=3, num_classes=7)
|
||||
model.load_state_dict(torch.load("models/unet3plus_att.pth", map_location=device, weights_only=False))
|
||||
else:
|
||||
raise ValueError(f"Unknown segmentation model: {model_name}")
|
||||
|
||||
print(f"✅ Loaded SUP: {model_name}")
|
||||
return model.to(device).eval()
|
||||
|
||||
def load_segmentation_model_post(model_name):
|
||||
print(f"📄 Loading segmentation model POST: {model_name}")
|
||||
|
||||
if model_name == "deeplabv3_resnet101":
|
||||
model = models.segmentation.deeplabv3_resnet101(weights=None)
|
||||
in_ch = model.classifier[-1].in_channels
|
||||
model.classifier = nn.Sequential(
|
||||
model.classifier[0],
|
||||
nn.Dropout(0.3),
|
||||
nn.Conv2d(in_ch, 7, kernel_size=1)
|
||||
)
|
||||
model.load_state_dict(torch.load("models/best_model_deeplabv3_resnet101_seed_16.pth", map_location=device, weights_only=False), strict=False)
|
||||
else:
|
||||
raise ValueError(f"Unknown post segmentation model: {model_name}")
|
||||
|
||||
print(f"✅ Loaded POST: {model_name}")
|
||||
return model.to(device).eval()
|
||||
|
||||
# Transforms
|
||||
angle_transform = transforms.Compose([
|
||||
transforms.Resize((224, 224)),
|
||||
transforms.ToTensor(),
|
||||
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
])
|
||||
|
||||
inflammation_transform = transforms.Compose([
|
||||
transforms.Resize((224, 224)),
|
||||
transforms.ToTensor(),
|
||||
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
])
|
||||
|
||||
segmentation_transform = transforms.Compose([
|
||||
transforms.Resize((512, 512)),
|
||||
transforms.ToTensor()
|
||||
])
|
||||
|
||||
# ============ PREDICTION ============
|
||||
|
||||
@torch.no_grad()
|
||||
def predict_angle(model, image_pil):
|
||||
img_tensor = angle_transform(image_pil).unsqueeze(0).to(device)
|
||||
output = model(img_tensor)
|
||||
probs = torch.softmax(output, dim=1)
|
||||
pred_class = torch.argmax(probs, dim=1).item()
|
||||
confidence = probs[0][pred_class].item()
|
||||
return ANGLE_CLASSES[pred_class], round(confidence * 100, 2)
|
||||
|
||||
@torch.no_grad()
|
||||
def predict_inflammation(model, image_pil):
|
||||
img_tensor = inflammation_transform(image_pil).unsqueeze(0).to(device)
|
||||
output = model(img_tensor)
|
||||
probs = torch.softmax(output, dim=1)
|
||||
pred_class = torch.argmax(probs, dim=1).item()
|
||||
confidence = probs[0][pred_class].item()
|
||||
is_inflammation = (pred_class == 1)
|
||||
return is_inflammation, round(confidence * 100, 2)
|
||||
|
||||
@torch.no_grad()
|
||||
def segment_image(model, image_pil, model_type, angle_type):
|
||||
original_size = image_pil.size
|
||||
img_tensor = segmentation_transform(image_pil).unsqueeze(0).to(device)
|
||||
|
||||
if model_type.startswith("deeplabv3"):
|
||||
outputs = model(img_tensor)['out']
|
||||
else:
|
||||
outputs = model(img_tensor)
|
||||
|
||||
upsampled = F.interpolate(outputs, size=original_size[::-1], mode='bilinear', align_corners=False)
|
||||
preds = upsampled.argmax(dim=1)[0].cpu().numpy()
|
||||
|
||||
if angle_type == 'sup' and model_type in ["unet3plus", "efficientfeedback"]:
|
||||
remap = {0: 0, 1: 2, 2: 6, 3: 1, 4: 4, 5: 5, 6: 3}
|
||||
preds = np.vectorize(remap.get)(preds)
|
||||
|
||||
class_map = SEGMENT_CLASSES_SUPRAPAT if angle_type == 'sup' else SEGMENT_CLASSES_POST
|
||||
|
||||
masks = {}
|
||||
for class_id, class_name in class_map.items():
|
||||
mask = (preds == class_id).astype(np.uint8)
|
||||
masks[class_name] = mask
|
||||
|
||||
return preds, masks
|
||||
|
||||
def get_mask_bounding_box(mask, dist_percent=0.01):
|
||||
"""
|
||||
Duyệt toàn bộ vùng được mask, loại bỏ nhiễu và trả về khung bao (Bounding Box).
|
||||
Áp dụng quy tắc kết hợp:
|
||||
1. Giữ lại khối có diện tích lớn nhất (vùng trung tâm).
|
||||
2. Giữ lại các khối phụ nếu thỏa mãn một trong hai điều kiện:
|
||||
- Diện tích >= 1/5 diện tích khối lớn nhất.
|
||||
- Khoảng cách tới khối lớn nhất <= dist_percent * chiều rộng ảnh.
|
||||
"""
|
||||
if mask is None or np.sum(mask) == 0:
|
||||
return None
|
||||
|
||||
# 1. Chuyển sang uint8
|
||||
mask_uint8 = mask.astype(np.uint8)
|
||||
if np.max(mask_uint8) == 1:
|
||||
mask_uint8 *= 255
|
||||
|
||||
# Lấy chiều rộng ảnh để tính ngưỡng khoảng cách theo %
|
||||
img_width = mask_uint8.shape[1]
|
||||
dist_threshold = img_width * dist_percent
|
||||
|
||||
# 2. Làm sạch mask cơ bản (Morphological Opening)
|
||||
kernel = np.ones((5, 5), np.uint8)
|
||||
clean_mask = cv2.morphologyEx(mask_uint8, cv2.MORPH_OPEN, kernel)
|
||||
|
||||
# 3. Tìm các đường bao (các khối tách rời)
|
||||
contours, _ = cv2.findContours(clean_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||
|
||||
if not contours:
|
||||
return None
|
||||
|
||||
# 4. Tính diện tích từng khối và tìm khối lớn nhất
|
||||
contour_info = []
|
||||
for cnt in contours:
|
||||
contour_info.append({'cnt': cnt, 'area': cv2.contourArea(cnt)})
|
||||
|
||||
# Sắp xếp theo diện tích giảm dần
|
||||
contour_info.sort(key=lambda x: x['area'], reverse=True)
|
||||
main_block = contour_info[0]
|
||||
max_area = main_block['area']
|
||||
|
||||
if max_area < 50:
|
||||
return None
|
||||
|
||||
# 5. Chuẩn bị để tính khoảng cách (Distance Transform)
|
||||
main_mask = np.zeros_like(mask_uint8)
|
||||
cv2.drawContours(main_mask, [main_block['cnt']], -1, 255, -1)
|
||||
# dist_map chứa khoảng cách từ mỗi điểm tới biên gần nhất của khối chính
|
||||
dist_map = cv2.distanceTransform(255 - main_mask, cv2.DIST_L2, 3)
|
||||
|
||||
# 6. Lọc các khối
|
||||
significant_contours = [main_block['cnt']]
|
||||
area_threshold = max_area / 4.0
|
||||
|
||||
for i in range(1, len(contour_info)):
|
||||
other = contour_info[i]
|
||||
|
||||
# Tạo mask cho khối đang xét để lấy giá trị khoảng cách
|
||||
other_mask = np.zeros_like(mask_uint8)
|
||||
cv2.drawContours(other_mask, [other['cnt']], -1, 255, -1)
|
||||
|
||||
# Khoảng cách nhỏ nhất từ khối này tới khối chính
|
||||
min_dist = np.min(dist_map[other_mask > 0])
|
||||
|
||||
# Điều kiện giữ lại: (Diện tích đủ lớn) HOẶC (Ở gần khối chính theo %)
|
||||
if other['area'] >= area_threshold or min_dist <= dist_threshold:
|
||||
significant_contours.append(other['cnt'])
|
||||
|
||||
# 7. Tính toán bounding box bao quanh tất cả các vùng được chọn
|
||||
all_points = np.concatenate(significant_contours)
|
||||
x, y, w, h = cv2.boundingRect(all_points)
|
||||
|
||||
return x, y, w, h
|
||||
|
||||
def find_max_continuous_segment(col_array):
|
||||
padded = np.concatenate(([0], col_array, [0]))
|
||||
diffs = np.diff(padded)
|
||||
|
||||
starts = np.where(diffs == 1)[0]
|
||||
ends = np.where(diffs == -1)[0]
|
||||
|
||||
if len(starts) == 0:
|
||||
return 0, -1, -1
|
||||
|
||||
lengths = ends - starts
|
||||
max_idx = np.argmax(lengths)
|
||||
max_len = lengths[max_idx]
|
||||
|
||||
return max_len, starts[max_idx], ends[max_idx]
|
||||
|
||||
def measure_thickness_new(masks, image_size, measure_ids=None):
|
||||
if measure_ids is None:
|
||||
measure_ids = DEFAULT_MEASURE_IDS
|
||||
|
||||
width, height = image_size
|
||||
|
||||
mask_all_labels = np.zeros((height, width), dtype=np.uint8)
|
||||
mask_measure = np.zeros((height, width), dtype=np.uint8)
|
||||
|
||||
has_any_label = False
|
||||
|
||||
if 'fat-pat' in masks:
|
||||
class_map = SEGMENT_CLASSES_SUPRAPAT
|
||||
else:
|
||||
class_map = SEGMENT_CLASSES_POST
|
||||
|
||||
for class_id, class_name in class_map.items():
|
||||
if class_name not in masks or class_name == 'background':
|
||||
continue
|
||||
mask = masks[class_name]
|
||||
if np.sum(mask) > 0:
|
||||
has_any_label = True
|
||||
mask_all_labels = np.logical_or(mask_all_labels, mask).astype(np.uint8)
|
||||
|
||||
if class_id in measure_ids:
|
||||
mask_measure = np.logical_or(mask_measure, mask).astype(np.uint8)
|
||||
|
||||
if not has_any_label or np.sum(mask_measure) == 0:
|
||||
return None
|
||||
|
||||
# Đóng khung toàn bộ vùng được mask (xương, màng, dịch, mỡ...)
|
||||
bbox_all = get_mask_bounding_box(mask_all_labels)
|
||||
if bbox_all is None:
|
||||
return None
|
||||
|
||||
x_all, y_all, w_all, h_all = bbox_all
|
||||
|
||||
# Từ khung này, xác định vùng quét là 1/3 ở giữa theo chiều ngang
|
||||
roi_start = x_all + (w_all // 3)
|
||||
roi_end = x_all + (2 * w_all // 3)
|
||||
|
||||
roi_strip = mask_measure[:, roi_start:roi_end]
|
||||
|
||||
global_max_len_px = 0
|
||||
best_x_rel = 0
|
||||
best_y_start = 0
|
||||
best_y_end = 0
|
||||
|
||||
for x in range(roi_strip.shape[1]):
|
||||
col = roi_strip[:, x]
|
||||
if not np.any(col):
|
||||
continue
|
||||
|
||||
length, y_s, y_e = find_max_continuous_segment(col)
|
||||
|
||||
if length > global_max_len_px:
|
||||
global_max_len_px = length
|
||||
best_x_rel = x
|
||||
best_y_start = y_s
|
||||
best_y_end = y_e
|
||||
|
||||
if global_max_len_px == 0:
|
||||
return None
|
||||
|
||||
thickness_mm = global_max_len_px * PIXEL_TO_MM
|
||||
real_x = roi_start + best_x_rel
|
||||
|
||||
print(f"📏 Measurement: {thickness_mm:.2f}mm ({global_max_len_px}px) at x={real_x}")
|
||||
|
||||
return {
|
||||
'thickness_px': int(global_max_len_px),
|
||||
'thickness_mm': float(round(thickness_mm, 2)),
|
||||
'x': int(real_x),
|
||||
'y_start': int(best_y_start),
|
||||
'y_end': int(best_y_end),
|
||||
'roi_start': int(roi_start),
|
||||
'roi_end': int(roi_end),
|
||||
'bbox': {'x': int(x_all), 'y': int(y_all), 'w': int(w_all), 'h': int(h_all)}
|
||||
}
|
||||
|
||||
def analyze_inflammation_severity(masks, image_size):
|
||||
if not masks:
|
||||
return None
|
||||
|
||||
width, height = image_size
|
||||
total_pixels = width * height
|
||||
|
||||
effusion_mask = masks.get('effusion', np.zeros((height, width), dtype=np.uint8))
|
||||
effusion_pixels = int(np.sum(effusion_mask))
|
||||
effusion_ratio = (effusion_pixels / total_pixels) * 100
|
||||
|
||||
effusion_thickness = 0
|
||||
if effusion_pixels > 0:
|
||||
rows_with_effusion = np.any(effusion_mask > 0, axis=1)
|
||||
if np.any(rows_with_effusion):
|
||||
effusion_thickness = int(np.sum(rows_with_effusion))
|
||||
|
||||
synovium_mask = masks.get('synovium', np.zeros((height, width), dtype=np.uint8))
|
||||
synovium_pixels = int(np.sum(synovium_mask))
|
||||
synovium_ratio = (synovium_pixels / total_pixels) * 100
|
||||
|
||||
effusion_score = min(effusion_thickness / height * 100, 100)
|
||||
synovium_score = synovium_ratio
|
||||
combined_score = (effusion_score * 0.6 + synovium_score * 0.4)
|
||||
|
||||
if combined_score > 15:
|
||||
level, severity, color = 3, "Nặng", "#dc3545"
|
||||
description = f"Dịch khớp dày ({effusion_thickness}px), màng hoạt dịch tăng sinh rõ"
|
||||
elif combined_score >= 8:
|
||||
level, severity, color = 2, "Trung bình", "#fd7e14"
|
||||
description = f"Dịch khớp trung bình ({effusion_thickness}px), màng hoạt dịch tăng sinh vừa"
|
||||
elif combined_score >= 3:
|
||||
level, severity, color = 1, "Nhẹ", "#ffc107"
|
||||
description = f"Dịch khớp mỏng ({effusion_thickness}px), màng hoạt dịch tăng sinh nhẹ"
|
||||
else:
|
||||
level, severity, color = 0, "Rất nhẹ", "#28a745"
|
||||
description = "Lượng dịch và màng hoạt dịch trong giới hạn bình thường"
|
||||
|
||||
return {
|
||||
'level': int(level),
|
||||
'severity': severity,
|
||||
'color': color,
|
||||
'description': description,
|
||||
'effusion': {'pixels': effusion_pixels, 'ratio': float(round(effusion_ratio, 2)), 'thickness': effusion_thickness},
|
||||
'synovium': {'pixels': synovium_pixels, 'ratio': float(round(synovium_ratio, 2))},
|
||||
'combined_score': float(round(combined_score, 2))
|
||||
}
|
||||
|
||||
def create_segmentation_overlay(image_pil, masks, measurement=None, angle_type='sup'):
|
||||
if masks is None:
|
||||
return image_pil
|
||||
|
||||
color_map = COLOR_MAP_SUP if angle_type == 'sup' else COLOR_MAP_POST
|
||||
|
||||
img_array = np.array(image_pil)
|
||||
overlay = img_array.copy()
|
||||
|
||||
for class_name, mask in masks.items():
|
||||
if class_name in color_map and np.sum(mask) > 0:
|
||||
color = color_map[class_name]['color']
|
||||
for i in range(3):
|
||||
overlay[:, :, i] = np.where(mask > 0,
|
||||
(overlay[:, :, i] * 0.6 + color[i] * 0.4).astype(np.uint8),
|
||||
overlay[:, :, i])
|
||||
|
||||
overlay_pil = Image.fromarray(overlay)
|
||||
draw = ImageDraw.Draw(overlay_pil)
|
||||
|
||||
for class_name in ['effusion', 'synovium']:
|
||||
mask = masks.get(class_name)
|
||||
if mask is not None and np.sum(mask) > 0:
|
||||
mask_uint8 = (mask * 255).astype(np.uint8)
|
||||
contours, _ = cv2.findContours(mask_uint8, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||
|
||||
for contour in contours:
|
||||
points = contour.reshape(-1, 2).tolist()
|
||||
if len(points) > 2:
|
||||
points = [(int(p[0]), int(p[1])) for p in points]
|
||||
draw.line(points + [points[0]], fill=(255, 255, 255), width=3)
|
||||
|
||||
if measurement and angle_type == 'sup':
|
||||
x = measurement['x']
|
||||
y_start = measurement['y_start']
|
||||
y_end = measurement['y_end']
|
||||
thickness_mm = measurement['thickness_mm']
|
||||
roi_start = measurement['roi_start']
|
||||
roi_end = measurement['roi_end']
|
||||
bbox = measurement['bbox']
|
||||
|
||||
draw.rectangle(
|
||||
[bbox['x'], bbox['y'], bbox['x'] + bbox['w'], bbox['y'] + bbox['h']],
|
||||
outline=(0, 255, 0), # Chuyển sang xanh lá cho dễ nhìn
|
||||
width=3 # Tăng độ dày khung
|
||||
)
|
||||
|
||||
h = image_pil.size[1]
|
||||
draw.line([(roi_start, 0), (roi_start, h)], fill=(0, 255, 255), width=2)
|
||||
draw.line([(roi_end, 0), (roi_end, h)], fill=(0, 255, 255), width=2)
|
||||
|
||||
draw.line([(x, y_start), (x, y_end)], fill=(255, 0, 0), width=4)
|
||||
|
||||
radius = 4
|
||||
draw.ellipse([x-radius, y_start-radius, x+radius, y_start+radius],
|
||||
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
|
||||
draw.ellipse([x-radius, y_end-radius, x+radius, y_end+radius],
|
||||
fill=(0, 255, 0), outline=(255, 255, 255), width=2)
|
||||
|
||||
text = f"{thickness_mm:.2f} mm"
|
||||
|
||||
try:
|
||||
from PIL import ImageFont
|
||||
font = ImageFont.load_default()
|
||||
bbox_text = draw.textbbox((0, 0), text, font=font)
|
||||
text_w = bbox_text[2] - bbox_text[0]
|
||||
text_h = bbox_text[3] - bbox_text[1]
|
||||
except:
|
||||
text_w, text_h = 100, 20
|
||||
|
||||
text_x = x + 8
|
||||
text_y = y_start - text_h - 8
|
||||
|
||||
draw.rectangle(
|
||||
[text_x - 2, text_y - 2, text_x + text_w + 2, text_y + text_h + 2],
|
||||
fill=(0, 0, 0)
|
||||
)
|
||||
draw.text((text_x, text_y), text, fill=(255, 0, 0))
|
||||
|
||||
return overlay_pil
|
||||
|
||||
def apply_clahe(image_pil):
|
||||
"""Áp dụng thuật toán CLAHE để tăng độ tương phản. Phục vụ cả hiển thị và làm đầu vào AI."""
|
||||
# Chuyển từ PIL sang OpenCV (numpy array)
|
||||
img_array = np.array(image_pil)
|
||||
# Chuyển sang thang độ xám (Gray) để xử lý CLAHE
|
||||
gray = cv2.cvtColor(img_array, cv2.COLOR_RGB2GRAY)
|
||||
|
||||
# Tạo đối tượng CLAHE
|
||||
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
|
||||
enhanced_gray = clahe.apply(gray)
|
||||
|
||||
# Chuyển ngược lại sang RGB (3 kênh) để tương thích với các models
|
||||
enhanced_rgb = cv2.cvtColor(enhanced_gray, cv2.COLOR_GRAY2RGB)
|
||||
return Image.fromarray(enhanced_rgb)
|
||||
|
||||
# Mount thư mục static (CSS, JS)
|
||||
app.mount("/css", StaticFiles(directory="templates/css"), name="css")
|
||||
app.mount("/js", StaticFiles(directory="templates/js"), name="js")
|
||||
|
||||
@app.get("/")
|
||||
async def read_index():
|
||||
html_file = Path(TEMPLATES_FOLDER) / "index.html"
|
||||
if html_file.exists():
|
||||
return FileResponse(html_file)
|
||||
return JSONResponse({"error": "Template not found"})
|
||||
|
||||
@app.post("/api/analyze")
|
||||
async def analyze_image(
|
||||
image: UploadFile = File(...),
|
||||
angle_model: str = Query("convnext"),
|
||||
inflammation_model: str = Query("efficientnet_b0"),
|
||||
segment_model_sup: str = Query("deeplabv3"),
|
||||
segment_model_post: str = Query("deeplabv3_resnet101")
|
||||
):
|
||||
try:
|
||||
print(f"\n{'='*60}")
|
||||
print(f"📊 NEW REQUEST")
|
||||
print(f"Models: angle={angle_model}, inflam={inflammation_model}")
|
||||
print(f" seg_sup={segment_model_sup}, seg_post={segment_model_post}")
|
||||
print(f"{'='*60}")
|
||||
|
||||
contents = await image.read()
|
||||
image_pil = Image.open(io.BytesIO(contents)).convert('RGB')
|
||||
|
||||
# Tạo ảnh tăng cường độ tương phản (Enhanced) cho mục đích hiển thị
|
||||
enhanced_pil = apply_clahe(image_pil)
|
||||
buffered_en = io.BytesIO()
|
||||
enhanced_pil.save(buffered_en, format="PNG")
|
||||
enhanced_str = base64.b64encode(buffered_en.getvalue()).decode()
|
||||
|
||||
result = {
|
||||
'success': True,
|
||||
'filename': image.filename,
|
||||
'images': {
|
||||
'enhanced': f"data:image/png;base64,{enhanced_str}"
|
||||
},
|
||||
'models_used': {
|
||||
'angle': angle_model,
|
||||
'inflammation': inflammation_model,
|
||||
'segmentation_sup': segment_model_sup,
|
||||
'segmentation_post': segment_model_post
|
||||
}
|
||||
}
|
||||
|
||||
angle_clf = load_angle_model(angle_model)
|
||||
angle, angle_conf = predict_angle(angle_clf, image_pil)
|
||||
result['angle'] = {'class': angle, 'confidence': angle_conf}
|
||||
print(f"✅ Angle: {angle} ({angle_conf}%)")
|
||||
|
||||
if 'post-trans' in angle.lower():
|
||||
print(f"🔍 POST-TRANS pipeline")
|
||||
|
||||
inflam_model = load_inflammation_model()
|
||||
has_inflammation, inflam_conf = predict_inflammation(inflam_model, image_pil)
|
||||
result['inflammation'] = {'detected': has_inflammation, 'confidence': inflam_conf}
|
||||
print(f"✅ Inflammation: {has_inflammation} ({inflam_conf}%)")
|
||||
|
||||
if has_inflammation:
|
||||
seg_model = load_segmentation_model_post(segment_model_post)
|
||||
preds, masks = segment_image(seg_model, image_pil, segment_model_post, 'post')
|
||||
|
||||
if masks:
|
||||
segmented_img = create_segmentation_overlay(image_pil, masks, None, 'post')
|
||||
# Convert to base64
|
||||
buffered = io.BytesIO()
|
||||
segmented_img.save(buffered, format="PNG")
|
||||
img_str = base64.b64encode(buffered.getvalue()).decode()
|
||||
|
||||
result['images']['segmented'] = f"data:image/png;base64,{img_str}"
|
||||
|
||||
detected_classes = [k for k, v in masks.items() if np.sum(v) > 0]
|
||||
color_legend = []
|
||||
for class_name in detected_classes:
|
||||
if class_name in COLOR_MAP_POST:
|
||||
color_legend.append({
|
||||
'name': COLOR_MAP_POST[class_name]['name'],
|
||||
'color': COLOR_MAP_POST[class_name]['color'],
|
||||
'key': class_name
|
||||
})
|
||||
|
||||
result['segmentation'] = {
|
||||
'performed': True,
|
||||
'classes_detected': detected_classes,
|
||||
'color_legend': color_legend,
|
||||
'angle_type': 'post'
|
||||
}
|
||||
|
||||
print(f"✅ Segmentation POST completed")
|
||||
else:
|
||||
print(f"ℹ️ No inflammation detected - skipping segmentation POST")
|
||||
|
||||
elif 'sup-up-long' in angle.lower():
|
||||
print(f"🔍 SUPRAPAT pipeline")
|
||||
|
||||
inflam_model = load_inflammation_model()
|
||||
has_inflammation, inflam_conf = predict_inflammation(inflam_model, image_pil)
|
||||
result['inflammation'] = {'detected': has_inflammation, 'confidence': inflam_conf}
|
||||
print(f"✅ Inflammation: {has_inflammation} ({inflam_conf}%)")
|
||||
|
||||
if has_inflammation:
|
||||
seg_model = load_segmentation_model_sup(segment_model_sup)
|
||||
preds, masks = segment_image(seg_model, image_pil, segment_model_sup, 'sup')
|
||||
|
||||
if masks:
|
||||
measurement = measure_thickness_new(masks, image_pil.size)
|
||||
|
||||
segmented_img = create_segmentation_overlay(image_pil, masks, measurement, 'sup')
|
||||
# Convert to base64
|
||||
buffered = io.BytesIO()
|
||||
segmented_img.save(buffered, format="PNG")
|
||||
img_str = base64.b64encode(buffered.getvalue()).decode()
|
||||
|
||||
result['images']['segmented'] = f"data:image/png;base64,{img_str}"
|
||||
|
||||
if measurement:
|
||||
result['measurement'] = {
|
||||
'thickness_mm': measurement['thickness_mm'],
|
||||
'thickness_px': measurement['thickness_px'],
|
||||
'location_x': measurement['x'],
|
||||
'y_start': measurement['y_start'],
|
||||
'y_end': measurement['y_end']
|
||||
}
|
||||
print(f"✅ Measurement: {measurement['thickness_mm']:.2f}mm at x={measurement['x']}")
|
||||
|
||||
detected_classes = [k for k, v in masks.items() if np.sum(v) > 0]
|
||||
color_legend = []
|
||||
for class_name in detected_classes:
|
||||
if class_name in COLOR_MAP_SUP:
|
||||
color_legend.append({
|
||||
'name': COLOR_MAP_SUP[class_name]['name'],
|
||||
'color': COLOR_MAP_SUP[class_name]['color'],
|
||||
'key': class_name
|
||||
})
|
||||
|
||||
result['segmentation'] = {
|
||||
'performed': True,
|
||||
'classes_detected': detected_classes,
|
||||
'color_legend': color_legend,
|
||||
'angle_type': 'sup'
|
||||
}
|
||||
|
||||
severity = analyze_inflammation_severity(masks, image_pil.size)
|
||||
if severity:
|
||||
result['severity'] = severity
|
||||
print(f"✅ Severity: {severity['severity']}")
|
||||
|
||||
print(f"✅ Segmentation SUP completed")
|
||||
else:
|
||||
print(f"ℹ️ No inflammation detected - skipping segmentation SUP")
|
||||
|
||||
else:
|
||||
print(f"ℹ️ Other angle - only angle classification")
|
||||
|
||||
print(f"{'='*60}\n")
|
||||
return JSONResponse(result)
|
||||
|
||||
except Exception as e:
|
||||
import traceback
|
||||
print(f"❌ Error: {e}")
|
||||
print(traceback.format_exc())
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@app.get("/api/health")
|
||||
async def health_check():
|
||||
return JSONResponse({'status': 'healthy'})
|
||||
|
||||
def sanitize_name(name):
|
||||
# Loại bỏ ký tự không hợp lệ cho folder trên Windows
|
||||
if not name: return "unknown"
|
||||
# Thay thế ký tự lạ bằng dấu gạch dưới
|
||||
clean = re.sub(r'[\\/*?:"<>|]', "_", name)
|
||||
# Loại bỏ dấu cách thừa
|
||||
clean = clean.strip().replace(" ", "_")
|
||||
return clean
|
||||
|
||||
class SaveDataRequest(BaseModel):
|
||||
patient_info: dict
|
||||
analysis_result: dict
|
||||
images: dict
|
||||
|
||||
@app.post("/api/save")
|
||||
async def save_patient_data(data: SaveDataRequest):
|
||||
try:
|
||||
p = data.patient_info
|
||||
res = data.analysis_result
|
||||
imgs = data.images
|
||||
|
||||
# 1. Tạo thư mục theo mã bệnh nhân (nhóm chính)
|
||||
patient_id = sanitize_name(p.get('id', 'unknown'))
|
||||
patient_name = sanitize_name(p.get('name', 'no_name'))
|
||||
|
||||
# Thư mục chính của bệnh nhân
|
||||
patient_folder = f"{patient_id}_{patient_name}"
|
||||
|
||||
# Thư mục con theo thời gian hiện tại
|
||||
timestamp_folder = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
# Tổng hợp đường dẫn: patients/ID_Name/TIMESTAMP/
|
||||
target_dir = os.path.join("patients", patient_folder, timestamp_folder)
|
||||
os.makedirs(target_dir, exist_ok=True)
|
||||
|
||||
# 2. Lưu info.txt
|
||||
info_path = os.path.join(target_dir, "info.txt")
|
||||
with open(info_path, "w", encoding="utf-8") as f:
|
||||
f.write(f"--- THÔNG TIN BỆNH NHÂN ---\n")
|
||||
f.write(f"Mã BN: {patient_id}\n")
|
||||
f.write(f"Họ tên: {patient_name}\n")
|
||||
f.write(f"Giới tính: {p.get('gender')}\n")
|
||||
f.write(f"Tuổi: {p.get('age')}\n")
|
||||
f.write(f"Chẩn đoán BS: {p.get('diagnosis')}\n\n")
|
||||
|
||||
f.write(f"--- KẾT QUẢ PHÂN TÍCH AI ---\n")
|
||||
f.write(f"Góc chụp: {res.get('angle', {}).get('class')} ({res.get('angle', {}).get('confidence')}%)\n")
|
||||
|
||||
if 'inflammation' in res:
|
||||
infl = res['inflammation']
|
||||
f.write(f"Viêm nhiễm: {'Có' if infl['detected'] else 'Không'} ({infl['confidence']}%)\n")
|
||||
|
||||
if 'measurement' in res:
|
||||
m = res['measurement']
|
||||
f.write(f"Độ dày màng: {m['thickness_mm']} mm ({m['thickness_px']} px)\n")
|
||||
f.write(f"Vị trí x: {m['location_x']}\n")
|
||||
|
||||
if 'severity' in res:
|
||||
s = res['severity']
|
||||
f.write(f"Mức độ: {s['severity']}\n")
|
||||
f.write(f"Mô tả: {s['description']}\n")
|
||||
|
||||
# 3. Lưu ảnh
|
||||
def save_base64_img(b64_str, filename):
|
||||
if not b64_str: return
|
||||
# Remove header if present
|
||||
if "," in b64_str:
|
||||
b64_str = b64_str.split(",")[1]
|
||||
|
||||
img_data = base64.b64decode(b64_str)
|
||||
with open(os.path.join(target_dir, filename), "wb") as f:
|
||||
f.write(img_data)
|
||||
|
||||
save_base64_img(imgs.get('original'), "original.png")
|
||||
save_base64_img(imgs.get('segmented'), "segmented.png")
|
||||
|
||||
# 4. Tự động lưu PDF báo cáo
|
||||
try:
|
||||
pdf_bytes = generate_medical_report(p, res, imgs)
|
||||
pdf_path = os.path.join(target_dir, "report.pdf")
|
||||
with open(pdf_path, "wb") as f:
|
||||
f.write(bytes(pdf_bytes))
|
||||
print(f"📄 Report PDF saved to: {pdf_path}")
|
||||
except Exception as pdf_err:
|
||||
print(f"⚠️ Warning: Could not auto-save PDF: {pdf_err}")
|
||||
|
||||
print(f"✅ Data saved for patient: {patient_id}")
|
||||
return {"success": True, "folder": target_dir}
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Save Error: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@app.post("/api/export-pdf")
|
||||
async def export_patient_pdf(data: SaveDataRequest):
|
||||
try:
|
||||
pdf_bytes = generate_medical_report(
|
||||
data.patient_info,
|
||||
data.analysis_result,
|
||||
data.images
|
||||
)
|
||||
|
||||
filename = f"Phieu_Kham_{sanitize_name(data.patient_info.get('id', 'unknown'))}.pdf"
|
||||
|
||||
return Response(
|
||||
content=bytes(pdf_bytes),
|
||||
media_type="application/pdf",
|
||||
headers={"Content-Disposition": f"attachment; filename={filename}"}
|
||||
)
|
||||
except Exception as e:
|
||||
import traceback
|
||||
print(f"❌ PDF Export Error: {e}")
|
||||
print(traceback.format_exc())
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
if __name__ == '__main__':
|
||||
print("Medical Image Analysis Server")
|
||||
print(f"URL: http://127.0.0.1:8000")
|
||||
print(f"Device: {device}")
|
||||
uvicorn.run(app, host="127.0.0.1", port=8000)
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
@@ -0,0 +1,247 @@
|
||||
from fpdf import FPDF
|
||||
import io
|
||||
import base64
|
||||
import os
|
||||
from datetime import datetime
|
||||
from PIL import Image
|
||||
|
||||
class MedicalReportPDF(FPDF):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.main_font = 'helvetica' # Default fallback
|
||||
self.set_margins(10, 10, 10)
|
||||
|
||||
def header(self):
|
||||
# Logo support
|
||||
logo_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'assets', 'logo.png')
|
||||
show_logo = False
|
||||
|
||||
if os.path.exists(logo_path):
|
||||
logo_stream = get_clean_image_stream(logo_path)
|
||||
if logo_stream:
|
||||
try:
|
||||
self.image(logo_stream, 10, 8, 25, type='PNG')
|
||||
self.set_x(40)
|
||||
show_logo = True
|
||||
except:
|
||||
pass
|
||||
|
||||
# Header with Unicode support
|
||||
try:
|
||||
self.set_font(self.main_font, 'B', 16)
|
||||
title = 'TRUNG TÂM CHẨN ĐOÁN HÌNH ẢNH VKIST'
|
||||
if show_logo:
|
||||
self.cell(0, 10, title, 0, 1, 'L')
|
||||
self.set_x(40)
|
||||
self.set_font(self.main_font, '', 10)
|
||||
self.cell(0, 5, 'Địa chỉ: Khu Công nghệ cao Hòa Lạc, Thạch Thất, Hà Nội', 0, 1, 'L')
|
||||
else:
|
||||
self.cell(0, 10, title, 0, 1, 'C')
|
||||
self.set_font(self.main_font, '', 10)
|
||||
self.cell(0, 10, 'Địa chỉ: Khu Công nghệ cao Hòa Lạc, Thạch Thất, Hà Nội', 0, 1, 'C')
|
||||
self.ln(10)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def footer(self):
|
||||
self.set_y(-15)
|
||||
try:
|
||||
self.set_font(self.main_font, 'I', 8)
|
||||
self.cell(0, 10, f'Trang {self.page_no()}/{{nb}}', 0, 0, 'C')
|
||||
except Exception:
|
||||
self.set_font('helvetica', 'I', 8)
|
||||
self.cell(0, 10, f'Page {self.page_no()}/{{nb}}', 0, 0, 'C')
|
||||
|
||||
def get_clean_image_stream(image_source):
|
||||
"""
|
||||
Mở ảnh (file path hoặc base64), loại bỏ interlacing và trả về BytesIO stream.
|
||||
Giúp tránh lỗi 'Interlacing not supported' trong FPDF2.
|
||||
"""
|
||||
if not image_source:
|
||||
return None
|
||||
|
||||
try:
|
||||
if isinstance(image_source, str) and image_source.startswith('data:image'):
|
||||
# Xử lý base64
|
||||
header, content = image_source.split(',') if ',' in image_source else (None, image_source)
|
||||
img_data = base64.b64decode(content)
|
||||
img_io = io.BytesIO(img_data)
|
||||
elif isinstance(image_source, str) and os.path.exists(image_source):
|
||||
# Xử lý file path
|
||||
img_io = image_source
|
||||
else:
|
||||
return None
|
||||
|
||||
with Image.open(img_io) as img:
|
||||
# Chuyển đổi sang RGB nếu cần và lưu lại không có interlacing
|
||||
if img.mode in ('RGBA', 'P'):
|
||||
img = img.convert('RGB')
|
||||
|
||||
output = io.BytesIO()
|
||||
img.save(output, format='PNG', optimize=False, interlaced=False)
|
||||
output.seek(0)
|
||||
return output
|
||||
except Exception as e:
|
||||
print(f"⚠️ Lỗi xử lý ảnh: {e}")
|
||||
return None
|
||||
|
||||
def generate_medical_report(patient_info, analysis_result, images_base64):
|
||||
pdf = MedicalReportPDF()
|
||||
|
||||
# Sử dụng đường dẫn tuyệt đối để ổn định
|
||||
base_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
font_dir = os.path.join(base_dir, 'assets', 'fonts')
|
||||
|
||||
# 1. Đăng ký Font nội bộ (Arial đã sao chép)
|
||||
try:
|
||||
pdf.add_font('arial_local', '', os.path.join(font_dir, 'arial.ttf'))
|
||||
pdf.add_font('arial_local', 'B', os.path.join(font_dir, 'arialbd.ttf'))
|
||||
pdf.add_font('arial_local', 'I', os.path.join(font_dir, 'ariali.ttf'))
|
||||
pdf.main_font = 'arial_local'
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not load local Arial fonts: {e}")
|
||||
# Dự phòng cuối cùng
|
||||
pdf.main_font = 'helvetica'
|
||||
|
||||
font_main = pdf.main_font
|
||||
pdf.set_auto_page_break(auto=True, margin=15)
|
||||
pdf.add_page()
|
||||
pdf.alias_nb_pages()
|
||||
|
||||
# Tiêu đề
|
||||
pdf.set_font(font_main, 'B', 14)
|
||||
pdf.cell(0, 10, 'PHIẾU KẾT QUẢ SIÊU ÂM KHỚP GỐI', 0, 1, 'C')
|
||||
pdf.ln(5)
|
||||
|
||||
# Thông tin bệnh nhân
|
||||
pdf.set_x(10)
|
||||
pdf.set_font(font_main, 'B', 11)
|
||||
pdf.cell(0, 8, 'I. THÔNG TIN BỆNH NHÂN', 0, 1, 'L')
|
||||
pdf.set_font(font_main, '', 11)
|
||||
|
||||
# Độ rộng cột an toàn (Tổng 180mm < 190mm khả dụng cho A4)
|
||||
col1 = 95
|
||||
col2 = 85
|
||||
|
||||
pdf.cell(col1, 8, f"Họ tên: {patient_info.get('name', 'N/A')}", 0, 0)
|
||||
pdf.cell(col2, 8, f"Mã BN: {patient_info.get('id', 'N/A')}", 0, 1)
|
||||
|
||||
pdf.cell(col1, 8, f"Giới tính: {patient_info.get('gender', 'N/A')}", 0, 0)
|
||||
pdf.cell(col2, 8, f"Tuổi: {patient_info.get('age', 'N/A')}", 0, 1)
|
||||
|
||||
pdf.ln(5)
|
||||
pdf.line(10, pdf.get_y(), 200, pdf.get_y())
|
||||
pdf.ln(5)
|
||||
|
||||
# Images Section
|
||||
pdf.set_x(10)
|
||||
pdf.set_font(font_main, 'B', 11)
|
||||
pdf.cell(0, 10, 'II. HÌNH ẢNH SIÊU ÂM', 0, 1, 'L')
|
||||
|
||||
y_before_images = pdf.get_y()
|
||||
img_w = 90
|
||||
margin = 5
|
||||
|
||||
# Process Images
|
||||
has_orig = images_base64.get('original')
|
||||
has_seg = images_base64.get('segmented')
|
||||
|
||||
max_img_h = 0
|
||||
|
||||
if has_orig:
|
||||
try:
|
||||
orig_stream = get_clean_image_stream(has_orig)
|
||||
if orig_stream:
|
||||
with Image.open(orig_stream) as img:
|
||||
w, h = img.size
|
||||
img_h = (img_w * h) / w
|
||||
max_img_h = max(max_img_h, img_h)
|
||||
|
||||
orig_stream.seek(0)
|
||||
pdf.image(orig_stream, x=10, y=y_before_images, w=img_w, type='PNG')
|
||||
|
||||
# Label
|
||||
pdf.set_xy(10, y_before_images + img_h + 2)
|
||||
pdf.set_font(font_main, 'I', 9)
|
||||
pdf.cell(img_w, 5, 'Hình 1: Ảnh gốc / Tăng cường', 0, 0, 'C')
|
||||
except Exception as e:
|
||||
print(f"Error processing original image: {e}")
|
||||
|
||||
if has_seg:
|
||||
try:
|
||||
seg_stream = get_clean_image_stream(has_seg)
|
||||
if seg_stream:
|
||||
with Image.open(seg_stream) as img:
|
||||
w, h = img.size
|
||||
img_h = (img_w * h) / w
|
||||
max_img_h = max(max_img_h, img_h)
|
||||
|
||||
seg_stream.seek(0)
|
||||
pdf.image(seg_stream, x=110, y=y_before_images, w=img_w, type='PNG')
|
||||
|
||||
# Label
|
||||
pdf.set_xy(110, y_before_images + img_h + 2)
|
||||
pdf.set_font(font_main, 'I', 9)
|
||||
pdf.cell(img_w, 5, 'Hình 2: Ảnh phân đoạn AI', 0, 1, 'C')
|
||||
except Exception as e:
|
||||
print(f"Error processing segmented image: {e}")
|
||||
|
||||
# Reset Y to after images
|
||||
pdf.set_y(y_before_images + max_img_h + 10)
|
||||
pdf.set_x(10)
|
||||
|
||||
# AI Results
|
||||
pdf.set_font(font_main, 'B', 11)
|
||||
pdf.cell(0, 10, 'III. KẾT QUẢ PHÂN TÍCH TỰ ĐỘNG (AI)', 0, 1, 'L')
|
||||
pdf.set_font(font_main, '', 10)
|
||||
|
||||
angle = analysis_result.get('angle', {})
|
||||
pdf.multi_cell(185, 6, f"• Góc chụp dự đoán: {angle.get('class', 'N/A')} (Độ tin cậy: {angle.get('confidence', 'N/A')}%)")
|
||||
|
||||
if 'inflammation' in analysis_result:
|
||||
infl = analysis_result['inflammation']
|
||||
status = "Có khả năng viêm / Theo dõi viêm" if infl.get('detected') else "Không thấy dấu hiệu viêm rõ rệt"
|
||||
pdf.set_x(10)
|
||||
pdf.multi_cell(185, 6, f"• Tình trạng viêm: {status} (Độ tin cậy: {infl.get('confidence', 'N/A')}%)")
|
||||
|
||||
if 'measurement' in analysis_result:
|
||||
m = analysis_result['measurement']
|
||||
pdf.set_x(10)
|
||||
pdf.multi_cell(185, 6, f"• Đo đạc: Độ dày dịch & màng hoạt dịch đạt mức {m.get('thickness_mm', 'N/A')} mm")
|
||||
|
||||
if 'severity' in analysis_result:
|
||||
s = analysis_result['severity']
|
||||
pdf.set_x(10)
|
||||
pdf.multi_cell(185, 6, f"• Mức độ viêm: {s.get('severity', 'N/A')}")
|
||||
pdf.set_x(10)
|
||||
pdf.set_font(font_main, 'I', 10)
|
||||
pdf.multi_cell(185, 6, f" Chi tiết: {s.get('description', 'N/A')}")
|
||||
pdf.set_font(font_main, '', 10)
|
||||
|
||||
pdf.ln(5)
|
||||
pdf.set_x(10)
|
||||
|
||||
# Doctor Diagnosis
|
||||
pdf.set_font(font_main, 'B', 11)
|
||||
pdf.cell(0, 10, 'IV. CHẨN ĐOÁN VÀ KẾT LUẬN CỦA BÁC SĨ', 0, 1, 'L')
|
||||
pdf.set_font(font_main, '', 11)
|
||||
|
||||
diagnosis = patient_info.get('diagnosis', 'Ghi chú chẩn đoán trống.')
|
||||
pdf.set_x(10)
|
||||
pdf.multi_cell(185, 7, diagnosis)
|
||||
|
||||
pdf.ln(15)
|
||||
|
||||
# Signature
|
||||
current_date = datetime.now()
|
||||
date_str = f"Ngày {current_date.day} tháng {current_date.month} năm {current_date.year}"
|
||||
|
||||
pdf.set_font(font_main, 'I', 11)
|
||||
pdf.cell(0, 8, date_str, 0, 1, 'R')
|
||||
pdf.set_font(font_main, 'B', 11)
|
||||
pdf.cell(0, 8, 'BÁC SĨ CHẨN ĐOÁN', 0, 1, 'R')
|
||||
pdf.ln(15)
|
||||
pdf.set_font(font_main, '', 10)
|
||||
pdf.cell(0, 8, '(Ký và ghi rõ họ tên)', 0, 1, 'R')
|
||||
|
||||
return pdf.output()
|
||||
@@ -0,0 +1,681 @@
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
min-height: 100vh;
|
||||
padding: 20px;
|
||||
}
|
||||
.container {
|
||||
max-width: 1600px;
|
||||
margin: 0 auto;
|
||||
background: white;
|
||||
border-radius: 20px;
|
||||
box-shadow: 0 20px 40px rgba(0,0,0,0.1);
|
||||
overflow: hidden;
|
||||
}
|
||||
.header {
|
||||
background: linear-gradient(135deg, #2c3e50 0%, #3498db 100%);
|
||||
color: white;
|
||||
padding: 30px;
|
||||
text-align: center;
|
||||
}
|
||||
.header h1 { font-size: 2rem; margin-bottom: 10px; }
|
||||
.main-content {
|
||||
display: grid;
|
||||
grid-template-columns: 300px 1fr 1fr;
|
||||
gap: 25px;
|
||||
padding: 30px;
|
||||
min-height: 600px;
|
||||
}
|
||||
|
||||
.models-panel {
|
||||
background: #f8f9fa;
|
||||
padding: 20px;
|
||||
border-radius: 15px;
|
||||
height: fit-content;
|
||||
}
|
||||
.models-panel h3 {
|
||||
margin-bottom: 15px;
|
||||
color: #2c3e50;
|
||||
font-size: 1.1rem;
|
||||
}
|
||||
.model-section {
|
||||
background: white;
|
||||
padding: 15px;
|
||||
margin-bottom: 15px;
|
||||
border-radius: 10px;
|
||||
border-left: 4px solid #3498db;
|
||||
}
|
||||
.model-section h4 {
|
||||
font-size: 0.9rem;
|
||||
color: #2c3e50;
|
||||
margin-bottom: 10px;
|
||||
font-weight: 600;
|
||||
}
|
||||
.model-section select {
|
||||
width: 100%;
|
||||
padding: 8px;
|
||||
border: 2px solid #e0e0e0;
|
||||
border-radius: 6px;
|
||||
font-size: 0.85rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
.model-section select:focus {
|
||||
outline: none;
|
||||
border-color: #3498db;
|
||||
}
|
||||
.model-section small {
|
||||
display: block;
|
||||
margin-top: 6px;
|
||||
color: #7f8c8d;
|
||||
font-size: 0.75rem;
|
||||
line-height: 1.3;
|
||||
}
|
||||
|
||||
.upload-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 20px;
|
||||
}
|
||||
.panel-title {
|
||||
font-size: 1.3rem;
|
||||
color: #2c3e50;
|
||||
font-weight: 600;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
.upload-area {
|
||||
border: 3px dashed #bdc3c7;
|
||||
border-radius: 15px;
|
||||
padding: 40px 20px;
|
||||
text-align: center;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-height: 250px;
|
||||
}
|
||||
.upload-area:hover {
|
||||
border-color: #3498db;
|
||||
background: #f8f9fa;
|
||||
}
|
||||
.upload-icon { font-size: 3rem; margin-bottom: 15px; }
|
||||
.upload-btn {
|
||||
background: linear-gradient(135deg, #3498db, #2980b9);
|
||||
color: white;
|
||||
padding: 12px 30px;
|
||||
border: none;
|
||||
border-radius: 25px;
|
||||
font-size: 1rem;
|
||||
cursor: pointer;
|
||||
font-weight: 600;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
.upload-btn:hover {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 5px 15px rgba(52, 152, 219, 0.4);
|
||||
}
|
||||
.image-preview-box {
|
||||
background: #f8f9fa;
|
||||
border-radius: 15px;
|
||||
padding: 15px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
.image-preview-box h3 {
|
||||
margin-bottom: 12px;
|
||||
color: #2c3e50;
|
||||
font-size: 1.05rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
.preview-image {
|
||||
width: 100%;
|
||||
border-radius: 10px;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.results-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 20px;
|
||||
}
|
||||
|
||||
.angle-result-card {
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
padding: 20px;
|
||||
border-radius: 15px;
|
||||
text-align: center;
|
||||
box-shadow: 0 4px 15px rgba(102, 126, 234, 0.3);
|
||||
}
|
||||
.angle-result-card h3 {
|
||||
font-size: 0.9rem;
|
||||
margin-bottom: 10px;
|
||||
opacity: 0.9;
|
||||
}
|
||||
.angle-value {
|
||||
font-size: 2rem;
|
||||
font-weight: 900;
|
||||
margin: 10px 0;
|
||||
text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
|
||||
}
|
||||
.angle-confidence {
|
||||
font-size: 0.85rem;
|
||||
opacity: 0.8;
|
||||
}
|
||||
|
||||
.result-image-box {
|
||||
background: #f8f9fa;
|
||||
border-radius: 15px;
|
||||
padding: 15px;
|
||||
}
|
||||
.result-image-box h3 {
|
||||
margin-bottom: 12px;
|
||||
color: #2c3e50;
|
||||
font-size: 1.05rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
.result-image-box img {
|
||||
width: 100%;
|
||||
border-radius: 10px;
|
||||
}
|
||||
|
||||
.color-legend {
|
||||
background: white;
|
||||
padding: 12px;
|
||||
border-radius: 8px;
|
||||
margin-top: 12px;
|
||||
}
|
||||
.color-legend h4 {
|
||||
font-size: 0.9rem;
|
||||
color: #2c3e50;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.legend-items {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 8px;
|
||||
}
|
||||
.legend-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
.legend-color {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border-radius: 4px;
|
||||
border: 1px solid #ddd;
|
||||
}
|
||||
.legend-highlight {
|
||||
border: 2px solid #fff;
|
||||
box-shadow: 0 0 0 2px #333;
|
||||
}
|
||||
|
||||
.results-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 12px;
|
||||
}
|
||||
.result-card {
|
||||
background: #f8f9fa;
|
||||
border-radius: 12px;
|
||||
padding: 15px;
|
||||
border-left: 5px solid #3498db;
|
||||
}
|
||||
.result-card.full-width {
|
||||
grid-column: 1 / -1;
|
||||
}
|
||||
.result-card h4 {
|
||||
margin-bottom: 10px;
|
||||
color: #2c3e50;
|
||||
font-size: 1rem;
|
||||
}
|
||||
.result-value {
|
||||
font-size: 1.1rem;
|
||||
font-weight: 600;
|
||||
color: #2c3e50;
|
||||
}
|
||||
|
||||
/* Patient Form Styles */
|
||||
.patient-panel {
|
||||
background: #f8f9fa;
|
||||
padding: 20px;
|
||||
border-radius: 15px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
.form-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 15px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
.full-width {
|
||||
grid-column: 1 / -1;
|
||||
}
|
||||
.form-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 5px;
|
||||
}
|
||||
.form-group label {
|
||||
font-size: 0.85rem;
|
||||
font-weight: 600;
|
||||
color: #34495e;
|
||||
}
|
||||
.form-group label span {
|
||||
color: #e74c3c;
|
||||
}
|
||||
.form-group input, .form-group select, .form-group textarea {
|
||||
padding: 10px;
|
||||
border: 2px solid #e0e0e0;
|
||||
border-radius: 8px;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
.form-group input:focus, .form-group select:focus, .form-group textarea:focus {
|
||||
outline: none;
|
||||
border-color: #3498db;
|
||||
}
|
||||
.save-section {
|
||||
margin-top: 25px;
|
||||
padding-top: 20px;
|
||||
border-top: 2px solid #eee;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
gap: 15px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.save-btn {
|
||||
background: linear-gradient(135deg, #2ecc71, #27ae60);
|
||||
color: white;
|
||||
padding: 14px 40px;
|
||||
border: none;
|
||||
border-radius: 30px;
|
||||
font-size: 1.1rem;
|
||||
font-weight: 700;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
.save-btn:hover:not(:disabled) {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 5px 15px rgba(46, 204, 113, 0.4);
|
||||
}
|
||||
.save-btn:disabled {
|
||||
background: #bdc3c7;
|
||||
cursor: not-allowed;
|
||||
opacity: 0.7;
|
||||
}
|
||||
.save-btn .icon { font-size: 1.2rem; }
|
||||
|
||||
.export-btn {
|
||||
background: linear-gradient(135deg, #3498db, #2980b9);
|
||||
color: white;
|
||||
padding: 14px 40px;
|
||||
border: none;
|
||||
border-radius: 30px;
|
||||
font-size: 1.1rem;
|
||||
font-weight: 700;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
.export-btn:hover:not(:disabled) {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 5px 15px rgba(52, 152, 219, 0.4);
|
||||
}
|
||||
.export-btn:disabled {
|
||||
background: #bdc3c7;
|
||||
cursor: not-allowed;
|
||||
opacity: 0.7;
|
||||
}
|
||||
.export-btn .icon { font-size: 1.2rem; }
|
||||
.confidence {
|
||||
font-size: 0.85rem;
|
||||
color: #7f8c8d;
|
||||
margin-top: 5px;
|
||||
}
|
||||
.badge {
|
||||
display: inline-block;
|
||||
padding: 6px 14px;
|
||||
border-radius: 15px;
|
||||
font-weight: 600;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
.badge-success { background: #d4edda; color: #155724; }
|
||||
.badge-danger { background: #f8d7da; color: #721c24; }
|
||||
.severity-card { border-left-color: #e74c3c; }
|
||||
.severity-badge {
|
||||
padding: 8px 16px;
|
||||
border-radius: 20px;
|
||||
color: white;
|
||||
font-weight: 700;
|
||||
text-transform: uppercase;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
.severity-details {
|
||||
margin-top: 10px;
|
||||
font-size: 0.9rem;
|
||||
color: #555;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.severity-stats {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 10px;
|
||||
margin-top: 10px;
|
||||
}
|
||||
.stat-box {
|
||||
background: white;
|
||||
padding: 10px;
|
||||
border-radius: 8px;
|
||||
text-align: center;
|
||||
}
|
||||
.stat-label {
|
||||
font-size: 0.75rem;
|
||||
color: #7f8c8d;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.stat-value {
|
||||
font-size: 0.95rem;
|
||||
font-weight: 700;
|
||||
color: #2c3e50;
|
||||
}
|
||||
.loading {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
display: none;
|
||||
}
|
||||
.spinner {
|
||||
border: 4px solid #f3f3f3;
|
||||
border-top: 4px solid #3498db;
|
||||
border-radius: 50%;
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
animation: spin 1s linear infinite;
|
||||
margin: 0 auto 20px;
|
||||
}
|
||||
@keyframes spin {
|
||||
0% { transform: rotate(0deg); }
|
||||
100% { transform: rotate(360deg); }
|
||||
}
|
||||
.error {
|
||||
background: #f8d7da;
|
||||
color: #721c24;
|
||||
padding: 15px;
|
||||
border-radius: 10px;
|
||||
margin: 15px 0;
|
||||
display: none;
|
||||
}
|
||||
#fileInput { display: none; }
|
||||
.no-results-message {
|
||||
text-align: center;
|
||||
color: #7f8c8d;
|
||||
padding: 60px 20px;
|
||||
font-size: 1rem;
|
||||
}
|
||||
.info-badge {
|
||||
background: #e3f2fd;
|
||||
color: #1976d2;
|
||||
padding: 8px 12px;
|
||||
border-radius: 8px;
|
||||
font-size: 0.8rem;
|
||||
margin-top: 8px;
|
||||
display: block;
|
||||
line-height: 1.4;
|
||||
}
|
||||
.measurement-main {
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
padding: 20px;
|
||||
border-radius: 12px;
|
||||
text-align: center;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.measurement-value {
|
||||
font-size: 2.5rem;
|
||||
font-weight: 900;
|
||||
margin: 10px 0;
|
||||
text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
|
||||
}
|
||||
.measurement-label {
|
||||
font-size: 0.9rem;
|
||||
opacity: 0.9;
|
||||
}
|
||||
.measurement-details {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 10px;
|
||||
}
|
||||
.detail-box {
|
||||
background: white;
|
||||
padding: 12px;
|
||||
border-radius: 8px;
|
||||
text-align: center;
|
||||
}
|
||||
.detail-label {
|
||||
font-size: 0.75rem;
|
||||
color: #7f8c8d;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.detail-value {
|
||||
font-size: 1.1rem;
|
||||
font-weight: 700;
|
||||
color: #2c3e50;
|
||||
}
|
||||
|
||||
/* Modal Styles - Expanded */
|
||||
.modal-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.7);
|
||||
backdrop-filter: blur(12px);
|
||||
display: none;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
z-index: 1000;
|
||||
opacity: 0;
|
||||
transition: opacity 0.3s ease;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.modal-overlay.active {
|
||||
display: flex;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.modal-content {
|
||||
background: white;
|
||||
width: 100%;
|
||||
max-width: 1000px;
|
||||
max-height: 95vh;
|
||||
border-radius: 24px;
|
||||
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.3);
|
||||
overflow-y: auto;
|
||||
transform: scale(0.9) translateY(30px);
|
||||
transition: all 0.5s cubic-bezier(0.34, 1.56, 0.64, 1);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.modal-overlay.active .modal-content {
|
||||
transform: scale(1) translateY(0);
|
||||
}
|
||||
|
||||
.modal-header {
|
||||
background: linear-gradient(135deg, #2c3e50 0%, #3498db 100%);
|
||||
color: white;
|
||||
padding: 20px 30px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 10;
|
||||
}
|
||||
|
||||
.modal-header h2 {
|
||||
font-size: 1.4rem;
|
||||
font-weight: 800;
|
||||
}
|
||||
|
||||
.close-modal {
|
||||
background: rgba(255, 255, 255, 0.2);
|
||||
border: none;
|
||||
color: white;
|
||||
font-size: 1.5rem;
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
border-radius: 50%;
|
||||
cursor: pointer;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.close-modal:hover {
|
||||
background: rgba(255, 255, 255, 0.3);
|
||||
transform: rotate(90deg);
|
||||
}
|
||||
|
||||
.modal-body {
|
||||
padding: 30px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 25px;
|
||||
}
|
||||
|
||||
/* Image Grid in Modal */
|
||||
.modal-images-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 20px;
|
||||
}
|
||||
|
||||
.modal-img-box {
|
||||
background: #f1f3f5;
|
||||
padding: 10px;
|
||||
border-radius: 12px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.img-label {
|
||||
display: block;
|
||||
font-size: 0.85rem;
|
||||
font-weight: 700;
|
||||
color: #495057;
|
||||
margin-bottom: 8px;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.modal-img-box img {
|
||||
width: 100%;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 4px 10px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
/* Details Grid in Modal */
|
||||
.modal-details-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
|
||||
gap: 15px;
|
||||
}
|
||||
|
||||
.modal-detail-card {
|
||||
background: #f8f9fa;
|
||||
padding: 15px;
|
||||
border-radius: 16px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
text-align: center;
|
||||
border: 1px solid #e9ecef;
|
||||
}
|
||||
|
||||
.modal-detail-card.highlight {
|
||||
background: linear-gradient(135deg, #f0f7ff 0%, #e3f2fd 100%);
|
||||
border-color: #bbdefb;
|
||||
}
|
||||
|
||||
.modal-detail-card .label {
|
||||
font-size: 0.8rem;
|
||||
color: #6c757d;
|
||||
font-weight: 600;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
|
||||
.modal-detail-card .value {
|
||||
font-size: 1.25rem;
|
||||
font-weight: 800;
|
||||
color: #2c3e50;
|
||||
}
|
||||
|
||||
.modal-legend {
|
||||
background: white;
|
||||
padding: 20px;
|
||||
border-radius: 16px;
|
||||
border: 1px solid #dee2e6;
|
||||
}
|
||||
|
||||
.modal-legend h4 {
|
||||
margin-bottom: 12px;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
.modal-advice {
|
||||
background: #fff9db;
|
||||
padding: 20px;
|
||||
border-radius: 16px;
|
||||
border-left: 5px solid #fcc419;
|
||||
color: #856404;
|
||||
font-size: 1rem;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.modal-footer {
|
||||
padding: 0 30px 40px;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.modal-btn-primary {
|
||||
background: linear-gradient(135deg, #228be6, #1c7ed6);
|
||||
color: white;
|
||||
border: none;
|
||||
padding: 16px 60px;
|
||||
border-radius: 35px;
|
||||
font-size: 1.2rem;
|
||||
font-weight: 800;
|
||||
cursor: pointer;
|
||||
box-shadow: 0 10px 25px -5px rgba(34, 139, 230, 0.4);
|
||||
transition: all 0.3s;
|
||||
}
|
||||
|
||||
.modal-btn-primary:hover {
|
||||
transform: translateY(-4px);
|
||||
box-shadow: 0 15px 30px -5px rgba(34, 139, 230, 0.5);
|
||||
}
|
||||
|
||||
/* Responsive Modal */
|
||||
@media (max-width: 768px) {
|
||||
.modal-images-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
.modal-content {
|
||||
max-height: 100vh;
|
||||
border-radius: 0;
|
||||
}
|
||||
.modal-overlay {
|
||||
padding: 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,288 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="vi">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Phân tích ảnh siêu âm khớp gối</title>
|
||||
|
||||
<!-- Link CSS -->
|
||||
<link rel="stylesheet" href="css/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h1>🏥 PHÂN TÍCH ẢNH SIÊU ÂM KHỚP GỐI</h1>
|
||||
<p>Hệ thống phân tích tự động với AI - Pipeline: Góc → Viêm → Phân đoạn → Đo đạc</p>
|
||||
</div>
|
||||
|
||||
<div class="main-content">
|
||||
<div class="models-panel">
|
||||
<h3>⚙️ Chọn mô hình</h3>
|
||||
|
||||
<div class="model-section">
|
||||
<h4>1️⃣ Phân loại góc chụp</h4>
|
||||
<select id="angleModel">
|
||||
<option value="convnext">ConvNeXt-Tiny (100%)</option>
|
||||
<option value="densenet">DenseNet121 (98%)</option>
|
||||
<option value="resnet50">ResNet50 (98%)</option>
|
||||
<option value="efficientnet_b2">EfficientNet-B2 (97.78%)</option>
|
||||
<option value="swin">Swin Transformer (97.78%)</option>
|
||||
</select>
|
||||
<small>Luôn chạy đầu tiên cho mọi ảnh</small>
|
||||
</div>
|
||||
|
||||
<div class="model-section">
|
||||
<h4>2️⃣ Phát hiện viêm</h4>
|
||||
<select id="inflammationModel">
|
||||
<option value="efficientnet_b0">EfficientNet-B0</option>
|
||||
</select>
|
||||
<small>Chỉ với góc post-trans và suprapat-long</small>
|
||||
</div>
|
||||
|
||||
<div class="model-section">
|
||||
<h4>3️⃣ Phân đoạn Suprapat</h4>
|
||||
<select id="segmentModelSup">
|
||||
<option value="deeplabv3">DeepLabV3-ResNet50 (91.67%)</option>
|
||||
<option value="efficientfeedback">EfficientFeedback (86%)</option>
|
||||
<option value="unet3plus">UNet3+ Attention (80%)</option>
|
||||
<option value="unet_resnet101">UNet-ResNet101 (73.62%)</option>
|
||||
</select>
|
||||
<small>Dùng cho góc suprapat-long</small>
|
||||
</div>
|
||||
|
||||
<div class="model-section">
|
||||
<h4>4️⃣ Phân đoạn Post-trans</h4>
|
||||
<select id="segmentModelPost">
|
||||
<option value="deeplabv3_resnet101">DeepLabV3-ResNet101</option>
|
||||
</select>
|
||||
<small>Dùng cho góc post-trans</small>
|
||||
</div>
|
||||
|
||||
<div class="info-badge">
|
||||
💡 Đo độ dày (chỉ SUP): Quét vùng 1/3 giữa, tìm đoạn liên tục dài nhất
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="upload-panel">
|
||||
<div id="uploadSection">
|
||||
<h2 class="panel-title">📤 Tải ảnh lên</h2>
|
||||
|
||||
<div class="upload-area" id="uploadArea">
|
||||
<div class="upload-icon">📷</div>
|
||||
<div style="margin-bottom: 15px; color: #7f8c8d;">
|
||||
Kéo thả ảnh vào đây hoặc
|
||||
</div>
|
||||
<button class="upload-btn" onclick="document.getElementById('fileInput').click()">
|
||||
Chọn ảnh
|
||||
</button>
|
||||
<input type="file" id="fileInput" accept="image/*">
|
||||
</div>
|
||||
|
||||
<div class="loading" id="loading">
|
||||
<div class="spinner"></div>
|
||||
<p>Đang phân tích ảnh...</p>
|
||||
</div>
|
||||
|
||||
<div class="error" id="error"></div>
|
||||
</div>
|
||||
|
||||
<div id="originalImageContainer" style="display: none;">
|
||||
<h2 class="panel-title">📷 Ảnh gốc</h2>
|
||||
<div class="image-preview-box">
|
||||
<img id="originalImage" class="preview-image">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="results-panel">
|
||||
<h2 class="panel-title">📊 Kết quả phân tích</h2>
|
||||
|
||||
<div id="resultsContent">
|
||||
<!-- Patient Information Form -->
|
||||
<div class="patient-panel" id="patientInfoPanel" style="display: none;">
|
||||
<h3 class="panel-title" style="font-size: 1.1rem;">👤 Thông tin bệnh nhân</h3>
|
||||
<div class="form-grid">
|
||||
<div class="form-group">
|
||||
<label for="patientName">Họ và tên <span>*</span></label>
|
||||
<input type="text" id="patientName" placeholder="Nhập tên bệnh nhân...">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="patientId">Mã bệnh nhân <span>*</span></label>
|
||||
<input type="text" id="patientId" placeholder="VD: BN001...">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="patientGender">Giới tính</label>
|
||||
<select id="patientGender">
|
||||
<option value="Nam">Nam</option>
|
||||
<option value="Nữ">Nữ</option>
|
||||
<option value="Khác">Khác</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="patientAge">Tuổi</label>
|
||||
<input type="number" id="patientAge" placeholder="Nhập tuổi...">
|
||||
</div>
|
||||
<div class="form-group full-width">
|
||||
<label for="doctorDiagnosis">Chẩn đoán của bác sĩ</label>
|
||||
<textarea id="doctorDiagnosis" rows="3" placeholder="Nhập chẩn đoán hoặc ghi chú thêm..."></textarea>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="save-section">
|
||||
<button id="saveDataBtn" class="save-btn" disabled>
|
||||
<span class="icon">💾</span> Lưu dữ liệu
|
||||
</button>
|
||||
<button id="exportPdfBtn" class="export-btn" disabled>
|
||||
<span class="icon">📄</span> Xuất phiếu khám (PDF)
|
||||
</button>
|
||||
<div id="saveStatus" style="margin-top: 10px; font-size: 0.85rem;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="angleResultCard" style="display: none;">
|
||||
<div class="angle-result-card">
|
||||
<h3>🎯 GÓC CHỤP</h3>
|
||||
<div class="angle-value" id="angleValue">-</div>
|
||||
<div class="angle-confidence" id="angleConfidenceText">-</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="segmentedImageContainer" style="display: none;">
|
||||
<div class="result-image-box">
|
||||
<h3>🎨 Ảnh phân đoạn</h3>
|
||||
<img id="segmentedImage">
|
||||
|
||||
<div class="color-legend" id="colorLegend" style="display: none;">
|
||||
<h4>📋 Chú thích màu sắc:</h4>
|
||||
<div class="legend-items" id="legendItems"></div>
|
||||
<div style="margin-top: 8px; padding-top: 8px; border-top: 1px solid #e0e0e0; font-size: 0.8rem; color: #666;">
|
||||
⚠️ <strong>Viền trắng dày</strong>: Vùng viêm (dịch khớp & màng hoạt dịch)<br>
|
||||
<span id="measurementNote" style="display: none;">📏 <strong>Đường đỏ</strong>: Đo độ dày tại vùng 1/3 giữa</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="resultsGrid" style="display: none;">
|
||||
<div class="results-grid">
|
||||
<div class="result-card full-width" id="inflammationCard" style="display: none;">
|
||||
<h4>🔬 Tình trạng viêm</h4>
|
||||
<div id="inflammationResult"></div>
|
||||
<div class="confidence" id="inflammationConfidence">-</div>
|
||||
</div>
|
||||
|
||||
<div class="result-card full-width" id="measurementCard" style="display: none; border-left-color: #2ecc71; padding: 0; overflow: hidden;">
|
||||
<div style="padding: 15px;">
|
||||
<h4>📏 Đo độ dày (Dịch + Màng hoạt dịch)</h4>
|
||||
</div>
|
||||
|
||||
<div class="measurement-main">
|
||||
<div class="measurement-label">Độ dày tối đa</div>
|
||||
<div class="measurement-value" id="thicknessMm">- mm</div>
|
||||
<div class="measurement-label">
|
||||
(<span id="thicknessPx">-</span> pixels)
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div style="padding: 0 15px 15px 15px;">
|
||||
<div class="measurement-details">
|
||||
<div class="detail-box">
|
||||
<div class="detail-label">Vị trí X</div>
|
||||
<div class="detail-value" id="measurementLocationX">-</div>
|
||||
</div>
|
||||
<div class="detail-box">
|
||||
<div class="detail-label">Phạm vi Y</div>
|
||||
<div class="detail-value" id="measurementRangeY">-</div>
|
||||
</div>
|
||||
</div>
|
||||
<div style="margin-top: 8px; font-size: 0.75rem; color: #7f8c8d; text-align: center;">
|
||||
ℹ️ Đo tại vùng 1/3 giữa bounding box - Tìm đoạn liên tục dài nhất
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="result-card severity-card full-width" id="severityCard" style="display: none;">
|
||||
<h4>📈 Mức độ viêm</h4>
|
||||
<div>
|
||||
<span class="severity-badge" id="severityBadge">-</span>
|
||||
</div>
|
||||
<div class="severity-details" id="severityDescription">-</div>
|
||||
<div class="severity-stats">
|
||||
<div class="stat-box">
|
||||
<div class="stat-label">Dịch khớp (Effusion)</div>
|
||||
<div class="stat-value" id="effusionStat">-</div>
|
||||
</div>
|
||||
<div class="stat-box">
|
||||
<div class="stat-label">Màng hoạt dịch (Synovium)</div>
|
||||
<div class="stat-value" id="synoviumStat">-</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="noResults" class="no-results-message">
|
||||
Chưa có kết quả phân tích
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Popup Kết quả chẩn đoán -->
|
||||
<div id="resultModal" class="modal-overlay">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h2>🩺 KẾT QUẢ CHẨN ĐOÁN AI</h2>
|
||||
<button class="close-modal" id="closeModal">×</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div class="modal-images-grid">
|
||||
<div class="modal-img-box">
|
||||
<span class="img-label">Ảnh tăng cường</span>
|
||||
<img id="modalImgEnhanced" src="" alt="Enhanced">
|
||||
</div>
|
||||
<div class="modal-img-box" id="modalImgSegmentedBox">
|
||||
<span class="img-label">Ảnh phân đoạn</span>
|
||||
<img id="modalImgSegmented" src="" alt="Segmented">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="modal-details-grid">
|
||||
<div class="modal-detail-card">
|
||||
<span class="label">📐 Góc chụp</span>
|
||||
<span class="value" id="modalAngle">-</span>
|
||||
</div>
|
||||
<div class="modal-detail-card" id="modalInflammationRow">
|
||||
<span class="label">🔥 Tình trạng</span>
|
||||
<span class="value" id="modalInflammation">-</span>
|
||||
</div>
|
||||
<div class="modal-detail-card highlight" id="modalMeasurementRow" style="display: none;">
|
||||
<span class="label">📏 Độ dày</span>
|
||||
<span class="value" id="modalThickness">- mm</span>
|
||||
</div>
|
||||
<div class="modal-detail-card highlight" id="modalSeverityRow" style="display: none;">
|
||||
<span class="label">📈 Mức độ</span>
|
||||
<span class="severity-badge" id="modalSeverity">-</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="modalLegendContainer" class="modal-legend" style="display: none;">
|
||||
<h4>📋 Chú thích màu sắc:</h4>
|
||||
<div id="modalLegendItems" class="legend-items"></div>
|
||||
</div>
|
||||
|
||||
<div class="modal-advice">
|
||||
<p id="modalAdviceText">Hệ thống AI đã hoàn tất phân tích. Vui lòng đối chiếu hình ảnh và kết quả đo đạc phía trên.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button class="modal-btn-primary" id="modalViewDetail">Xem chi tiết</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Link JavaScript -->
|
||||
<script src="js/script.js?v=1.1"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,440 @@
|
||||
const API_BASE = 'http://127.0.0.1:8000';
|
||||
const uploadArea = document.getElementById('uploadArea');
|
||||
const fileInput = document.getElementById('fileInput');
|
||||
let currentResult = null;
|
||||
let originalImageBase64 = null;
|
||||
|
||||
const ANGLE_NAMES = {
|
||||
'med-lat': 'Med-Lat Long',
|
||||
'post-trans': 'Post Trans',
|
||||
'sup-trans-flex': 'Suprapat Trans Flex',
|
||||
'sup-up-long': 'Suprapat Up Long'
|
||||
};
|
||||
|
||||
uploadArea.addEventListener('dragover', (e) => {
|
||||
e.preventDefault();
|
||||
uploadArea.style.borderColor = '#3498db';
|
||||
uploadArea.style.background = '#f8f9fa';
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('dragleave', () => {
|
||||
uploadArea.style.borderColor = '#bdc3c7';
|
||||
uploadArea.style.background = 'transparent';
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('drop', (e) => {
|
||||
e.preventDefault();
|
||||
uploadArea.style.borderColor = '#bdc3c7';
|
||||
uploadArea.style.background = 'transparent';
|
||||
const files = e.dataTransfer.files;
|
||||
if (files.length > 0) handleFile(files[0]);
|
||||
});
|
||||
|
||||
fileInput.addEventListener('change', (e) => {
|
||||
if (e.target.files.length > 0) handleFile(e.target.files[0]);
|
||||
});
|
||||
|
||||
function handleFile(file) {
|
||||
if (!file.type.startsWith('image/')) {
|
||||
showError('Vui lòng chọn file ảnh hợp lệ');
|
||||
return;
|
||||
}
|
||||
|
||||
const reader = new FileReader();
|
||||
reader.onload = (e) => {
|
||||
originalImageBase64 = e.target.result;
|
||||
document.getElementById('originalImage').src = originalImageBase64;
|
||||
document.getElementById('originalImageContainer').style.display = 'block';
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
|
||||
uploadAndAnalyze(file);
|
||||
}
|
||||
|
||||
async function uploadAndAnalyze(file) {
|
||||
showLoading(true);
|
||||
hideError();
|
||||
hideResults();
|
||||
|
||||
const angleModelValue = document.getElementById('angleModel').value;
|
||||
const inflammationModelValue = document.getElementById('inflammationModel').value;
|
||||
const segmentModelSupValue = document.getElementById('segmentModelSup').value;
|
||||
const segmentModelPostValue = document.getElementById('segmentModelPost').value;
|
||||
|
||||
console.log('🎯 Selected models:', {
|
||||
angle: angleModelValue,
|
||||
inflammation: inflammationModelValue,
|
||||
seg_sup: segmentModelSupValue,
|
||||
seg_post: segmentModelPostValue
|
||||
});
|
||||
|
||||
const formData = new FormData();
|
||||
formData.append('image', file);
|
||||
|
||||
try {
|
||||
const url = `${API_BASE}/api/analyze?angle_model=${angleModelValue}&inflammation_model=${inflammationModelValue}&segment_model_sup=${segmentModelSupValue}&segment_model_post=${segmentModelPostValue}`;
|
||||
console.log('🌐 Request URL:', url);
|
||||
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
body: formData
|
||||
});
|
||||
|
||||
if (!response.ok) throw new Error('Lỗi phân tích');
|
||||
|
||||
const result = await response.json();
|
||||
console.log('✅ TRẢ VỀ TỪ API:', result);
|
||||
|
||||
if (result.success) {
|
||||
currentResult = result;
|
||||
displayResults(result);
|
||||
}
|
||||
else showError('Phân tích thất bại');
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Error:', error);
|
||||
showError(`Lỗi: ${error.message}`);
|
||||
} finally {
|
||||
showLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
function displayResults(result) {
|
||||
try {
|
||||
console.log('--- ĐANG HIỂN THỊ KẾT QUẢ ---');
|
||||
document.getElementById('noResults').style.display = 'none';
|
||||
|
||||
// Cập nhật ảnh gốc thành ảnh đã tăng cường tương phản (CLAHE)
|
||||
if (result.images && result.images.enhanced) {
|
||||
document.getElementById('originalImage').src = result.images.enhanced;
|
||||
}
|
||||
|
||||
document.getElementById('angleResultCard').style.display = 'block';
|
||||
document.getElementById('angleValue').textContent = ANGLE_NAMES[result.angle.class] || result.angle.class;
|
||||
document.getElementById('angleConfidenceText').textContent = `Độ tin cậy: ${result.angle.confidence}%`;
|
||||
|
||||
if (result.inflammation) {
|
||||
console.log('✅ Hiển thị Inflammation');
|
||||
document.getElementById('resultsGrid').style.display = 'block';
|
||||
document.getElementById('inflammationCard').style.display = 'block';
|
||||
|
||||
const inflDiv = document.getElementById('inflammationResult');
|
||||
if (result.inflammation.detected) {
|
||||
inflDiv.innerHTML = '<span class="badge badge-danger">CÓ KHẢ NĂNG VIÊM / THEO DÕI VIÊM</span>';
|
||||
} else {
|
||||
inflDiv.innerHTML = '<span class="badge badge-success">KHÔNG VIÊM</span>';
|
||||
}
|
||||
document.getElementById('inflammationConfidence').textContent = `Độ tin cậy: ${result.inflammation.confidence}%`;
|
||||
}
|
||||
|
||||
if (result.segmentation && result.segmentation.performed) {
|
||||
console.log('✅ Hiển thị Segmentation & Overlay');
|
||||
document.getElementById('segmentedImageContainer').style.display = 'block';
|
||||
document.getElementById('segmentedImage').src = result.images.segmented;
|
||||
|
||||
if (result.segmentation.color_legend) {
|
||||
displayColorLegend(result.segmentation.color_legend, result.segmentation.angle_type);
|
||||
}
|
||||
|
||||
if (result.segmentation.angle_type === 'sup') {
|
||||
document.getElementById('measurementNote').style.display = 'inline';
|
||||
} else {
|
||||
document.getElementById('measurementNote').style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
if (result.measurement) {
|
||||
console.log('✅ Hiển thị Đo đạc (Measurement)');
|
||||
document.getElementById('measurementCard').style.display = 'block';
|
||||
|
||||
document.getElementById('thicknessMm').textContent = `${result.measurement.thickness_mm} mm`;
|
||||
document.getElementById('thicknessPx').textContent = `${result.measurement.thickness_px}`;
|
||||
document.getElementById('measurementLocationX').textContent = result.measurement.location_x;
|
||||
document.getElementById('measurementRangeY').textContent =
|
||||
`${result.measurement.y_start} - ${result.measurement.y_end}`;
|
||||
}
|
||||
|
||||
if (result.severity) {
|
||||
console.log('✅ Hiển thị Mức độ (Severity)');
|
||||
document.getElementById('severityCard').style.display = 'block';
|
||||
const badge = document.getElementById('severityBadge');
|
||||
badge.textContent = result.severity.severity;
|
||||
badge.style.background = result.severity.color;
|
||||
document.getElementById('severityDescription').textContent = result.severity.description;
|
||||
document.getElementById('effusionStat').textContent =
|
||||
`${result.severity.effusion.ratio}% (${result.severity.effusion.thickness}px)`;
|
||||
document.getElementById('synoviumStat').textContent = `${result.severity.synovium.ratio}%`;
|
||||
}
|
||||
|
||||
// Hiện bảng nhập liệu bệnh nhân
|
||||
document.getElementById('patientInfoPanel').style.display = 'block';
|
||||
updateSaveButtonState();
|
||||
|
||||
// HIỂN THỊ POPUP KẾT QUẢ
|
||||
showResultPopup(result);
|
||||
|
||||
} catch (err) {
|
||||
console.error('❌ LỖI TRONG displayResults:', err);
|
||||
showError(`Lỗi hiển thị: ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Kiểm tra tính hợp lệ của form
|
||||
function updateSaveButtonState() {
|
||||
const name = document.getElementById('patientName').value.trim();
|
||||
const id = document.getElementById('patientId').value.trim();
|
||||
const btn = document.getElementById('saveDataBtn');
|
||||
const exportBtn = document.getElementById('exportPdfBtn');
|
||||
|
||||
const isValid = !!(currentResult && name && id);
|
||||
btn.disabled = !isValid;
|
||||
exportBtn.disabled = !isValid;
|
||||
}
|
||||
|
||||
// Lắng nghe thay đổi trên form
|
||||
['patientName', 'patientId', 'patientGender', 'patientAge', 'doctorDiagnosis'].forEach(id => {
|
||||
document.getElementById(id).addEventListener('input', updateSaveButtonState);
|
||||
});
|
||||
|
||||
// Hàm lưu dữ liệu
|
||||
document.getElementById('saveDataBtn').addEventListener('click', async () => {
|
||||
if (!currentResult) return;
|
||||
|
||||
const saveBtn = document.getElementById('saveDataBtn');
|
||||
const statusDiv = document.getElementById('saveStatus');
|
||||
|
||||
const payload = {
|
||||
patient_info: {
|
||||
name: document.getElementById('patientName').value,
|
||||
id: document.getElementById('patientId').value,
|
||||
gender: document.getElementById('patientGender').value,
|
||||
age: document.getElementById('patientAge').value,
|
||||
diagnosis: document.getElementById('doctorDiagnosis').value
|
||||
},
|
||||
analysis_result: currentResult,
|
||||
images: {
|
||||
original: originalImageBase64,
|
||||
segmented: currentResult.images.segmented
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
saveBtn.disabled = true;
|
||||
statusDiv.innerHTML = '<span style="color: blue;">⌛ Đang lưu...</span>';
|
||||
|
||||
const response = await fetch(`${API_BASE}/api/save`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload)
|
||||
});
|
||||
|
||||
const resData = await response.json();
|
||||
if (resData.success) {
|
||||
statusDiv.innerHTML = `<span style="color: green;">✅ Đã lưu vào thư mục: <strong>${resData.folder}</strong></span>`;
|
||||
} else {
|
||||
throw new Error(resData.detail || 'Lỗi không xác định');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('❌ Save error:', error);
|
||||
statusDiv.innerHTML = `<span style="color: red;">❌ Lỗi: ${error.message}</span>`;
|
||||
} finally {
|
||||
saveBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
document.getElementById('exportPdfBtn').addEventListener('click', async () => {
|
||||
if (!currentResult) return;
|
||||
|
||||
const exportBtn = document.getElementById('exportPdfBtn');
|
||||
const statusDiv = document.getElementById('saveStatus');
|
||||
|
||||
const payload = {
|
||||
patient_info: {
|
||||
name: document.getElementById('patientName').value,
|
||||
id: document.getElementById('patientId').value,
|
||||
gender: document.getElementById('patientGender').value,
|
||||
age: document.getElementById('patientAge').value,
|
||||
diagnosis: document.getElementById('doctorDiagnosis').value
|
||||
},
|
||||
analysis_result: currentResult,
|
||||
images: {
|
||||
original: originalImageBase64,
|
||||
segmented: currentResult.images.segmented
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
exportBtn.disabled = true;
|
||||
statusDiv.innerHTML = '<span style="color: blue;">⌛ Đang khởi tạo PDF...</span>';
|
||||
|
||||
const response = await fetch(`${API_BASE}/api/export-pdf`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload)
|
||||
});
|
||||
|
||||
if (!response.ok) throw new Error('Lỗi từ server khi tạo PDF');
|
||||
|
||||
const blob = await response.blob();
|
||||
const url = window.URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `Phieu_Kham_${payload.patient_info.id || 'BN'}.pdf`;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
window.URL.revokeObjectURL(url);
|
||||
document.body.removeChild(a);
|
||||
|
||||
statusDiv.innerHTML = `<span style="color: green;">✅ Đã xuất file PDF!</span>`;
|
||||
} catch (error) {
|
||||
console.error('❌ Export error:', error);
|
||||
statusDiv.innerHTML = `<span style="color: red;">❌ Lỗi: ${error.message}</span>`;
|
||||
} finally {
|
||||
exportBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
function displayColorLegend(colorLegend, angleType) {
|
||||
const legendContainer = document.getElementById('legendItems');
|
||||
legendContainer.innerHTML = '';
|
||||
|
||||
colorLegend.forEach(item => {
|
||||
const isHighlight = item.key === 'effusion' || item.key === 'synovium';
|
||||
const legendItem = document.createElement('div');
|
||||
legendItem.className = 'legend-item';
|
||||
legendItem.innerHTML = `
|
||||
<div class="legend-color ${isHighlight ? 'legend-highlight' : ''}"
|
||||
style="background-color: rgb(${item.color.join(',')})"></div>
|
||||
<span>${item.name}</span>
|
||||
`;
|
||||
legendContainer.appendChild(legendItem);
|
||||
});
|
||||
|
||||
document.getElementById('colorLegend').style.display = 'block';
|
||||
}
|
||||
|
||||
function showLoading(show) {
|
||||
document.getElementById('loading').style.display = show ? 'block' : 'none';
|
||||
}
|
||||
|
||||
function showError(msg) {
|
||||
const errorDiv = document.getElementById('error');
|
||||
errorDiv.textContent = msg;
|
||||
errorDiv.style.display = 'block';
|
||||
}
|
||||
|
||||
function hideError() {
|
||||
document.getElementById('error').style.display = 'none';
|
||||
}
|
||||
|
||||
function hideResults() {
|
||||
document.getElementById('angleResultCard').style.display = 'none';
|
||||
document.getElementById('resultsGrid').style.display = 'none';
|
||||
document.getElementById('segmentedImageContainer').style.display = 'none';
|
||||
document.getElementById('inflammationCard').style.display = 'none';
|
||||
document.getElementById('measurementCard').style.display = 'none';
|
||||
document.getElementById('severityCard').style.display = 'none';
|
||||
document.getElementById('noResults').style.display = 'block';
|
||||
}
|
||||
|
||||
function showResultPopup(result) {
|
||||
const modal = document.getElementById('resultModal');
|
||||
|
||||
// 1. Hình ảnh
|
||||
if (result.images) {
|
||||
document.getElementById('modalImgEnhanced').src = result.images.enhanced || '';
|
||||
const segImg = document.getElementById('modalImgSegmented');
|
||||
const segBox = document.getElementById('modalImgSegmentedBox');
|
||||
|
||||
if (result.images.segmented) {
|
||||
segImg.src = result.images.segmented;
|
||||
segBox.style.display = 'block';
|
||||
} else {
|
||||
segBox.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Chi tiết kết quả
|
||||
document.getElementById('modalAngle').textContent = ANGLE_NAMES[result.angle.class] || result.angle.class;
|
||||
|
||||
const inflEl = document.getElementById('modalInflammation');
|
||||
if (result.inflammation) {
|
||||
document.getElementById('modalInflammationRow').style.display = 'flex';
|
||||
if (result.inflammation.detected) {
|
||||
inflEl.innerHTML = '<span class="badge badge-danger">CÓ VIÊM/THEO DÕI</span>';
|
||||
} else {
|
||||
inflEl.innerHTML = '<span class="badge badge-success">KHÔNG VIÊM</span>';
|
||||
}
|
||||
} else {
|
||||
document.getElementById('modalInflammationRow').style.display = 'none';
|
||||
}
|
||||
|
||||
if (result.measurement) {
|
||||
document.getElementById('modalMeasurementRow').style.display = 'flex';
|
||||
document.getElementById('modalThickness').textContent = `${result.measurement.thickness_mm} mm`;
|
||||
} else {
|
||||
document.getElementById('modalMeasurementRow').style.display = 'none';
|
||||
}
|
||||
|
||||
if (result.severity) {
|
||||
document.getElementById('modalSeverityRow').style.display = 'flex';
|
||||
const badge = document.getElementById('modalSeverity');
|
||||
badge.textContent = result.severity.severity;
|
||||
badge.style.background = result.severity.color;
|
||||
} else {
|
||||
document.getElementById('modalSeverityRow').style.display = 'none';
|
||||
}
|
||||
|
||||
// 3. Chú thích màu sắc (Legend) trong modal
|
||||
const legendContainer = document.getElementById('modalLegendContainer');
|
||||
if (result.segmentation && result.segmentation.performed && result.segmentation.color_legend) {
|
||||
legendContainer.style.display = 'block';
|
||||
renderModalLegend(result.segmentation.color_legend);
|
||||
} else {
|
||||
legendContainer.style.display = 'none';
|
||||
}
|
||||
|
||||
// Hiển thị modal
|
||||
modal.classList.add('active');
|
||||
}
|
||||
|
||||
function renderModalLegend(colorLegend) {
|
||||
const itemsContainer = document.getElementById('modalLegendItems');
|
||||
itemsContainer.innerHTML = '';
|
||||
|
||||
colorLegend.forEach(item => {
|
||||
const isHighlight = item.key === 'effusion' || item.key === 'synovium';
|
||||
const legendItem = document.createElement('div');
|
||||
legendItem.className = 'legend-item';
|
||||
legendItem.innerHTML = `
|
||||
<div class="legend-color ${isHighlight ? 'legend-highlight' : ''}"
|
||||
style="background-color: rgb(${item.color.join(',')})"></div>
|
||||
<span>${item.name}</span>
|
||||
`;
|
||||
itemsContainer.appendChild(legendItem);
|
||||
});
|
||||
}
|
||||
|
||||
function closeResultPopup() {
|
||||
document.getElementById('resultModal').classList.remove('active');
|
||||
}
|
||||
|
||||
// Event Listeners for Modal
|
||||
document.getElementById('closeModal').addEventListener('click', closeResultPopup);
|
||||
document.getElementById('modalViewDetail').addEventListener('click', closeResultPopup);
|
||||
|
||||
// Click outside to close
|
||||
document.getElementById('resultModal').addEventListener('click', (e) => {
|
||||
if (e.target.id === 'resultModal') closeResultPopup();
|
||||
});
|
||||
|
||||
// Health check
|
||||
window.addEventListener('load', async () => {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/api/health`);
|
||||
const health = await response.json();
|
||||
console.log('✅ Server ready:', health);
|
||||
} catch (error) {
|
||||
showError('Không thể kết nối server. Vui lòng khởi động FastAPI backend.');
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,16 @@
|
||||
--extra-index-url https://download.pytorch.org/whl/cpu
|
||||
|
||||
fastapi==0.135.1
|
||||
numpy==2.2.6
|
||||
opencv-python==4.13.0.92
|
||||
pillow==12.1.1
|
||||
python-multipart==0.0.22
|
||||
segmentation-models-pytorch==0.5.0
|
||||
timm==1.0.25
|
||||
torch==2.5.1
|
||||
torchvision==0.20.1
|
||||
torchaudio==2.5.1
|
||||
torchinfo==1.8.0
|
||||
tqdm==4.67.3
|
||||
uvicorn==0.41.0
|
||||
fpdf2==2.8.7
|
||||
@@ -0,0 +1,82 @@
|
||||
# VKIST Ultrasound
|
||||
|
||||
## Introduction
|
||||
**VKIST Ultrasound** is an application designed to support the diagnosis of **knee arthritis** using **knee ultrasound images**.
|
||||
The system processes ultrasound images and assists clinicians in identifying potential signs of arthritis, providing a supportive tool for medical analysis and research.
|
||||
|
||||
---
|
||||
|
||||
## Yêu cầu hệ thống
|
||||
|
||||
### 1. YÊU CẦU VỀ MÁY TÍNH
|
||||
* **Hệ điều hành:** Windows 10/11 (64-bit) hoặc Ubuntu 20.04/22.04.
|
||||
* **CPU:** Tối thiểu 4 nhân (khuyến nghị Intel Core i5 thế hệ 10 trở lên hoặc tương đương).
|
||||
* **RAM:** Tối thiểu 16GB.
|
||||
* **GPU:** NVIDIA GPU hỗ trợ CUDA (Kiến trúc Pascal trở lên, ví dụ: GTX 10-series, RTX series).
|
||||
* **VRAM:** Khuyến nghị 8GB trở lên để tối ưu tốc độ phân vùng (segmentation).
|
||||
|
||||
### 2. CÀI ĐẶT PHẦN MỀM HỖ TRỢ
|
||||
* **Quản lý môi trường:** [Anaconda3](https://www.anaconda.com/download) hoặc Miniconda.
|
||||
* **Đồ họa:** NVIDIA Driver tương thích với CUDA 12.4.
|
||||
* **CUDA Toolkit:** Phiên bản 12.4.
|
||||
* **Trình biên dịch C++:**
|
||||
* **Windows:** [Microsoft Visual C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/).
|
||||
* **Ubuntu:** Cài đặt qua lệnh `sudo apt install build-essential`.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### 1. Clone the Repository
|
||||
```bash
|
||||
git clone https://github.com/itvkist/vkist-ultrasound.git
|
||||
cd vkist-ultrasound
|
||||
```
|
||||
|
||||
### 2. Create Environment and Install Dependencies
|
||||
```bash
|
||||
conda create -n vkist-ultrasound python=3.10 -y
|
||||
conda activate vkist-ultrasound
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Download Model Weights
|
||||
|
||||
The weights of the models can be found in the following link:
|
||||
|
||||
```
|
||||
https://drive.google.com/drive/folders/1lBkplP-5uv6V2wR1CJ2COaGy1SnZxxJl
|
||||
```
|
||||
|
||||
After downloading the link, unzip and copy the files into the `./models` folder
|
||||
|
||||
### Run the Application
|
||||
|
||||
Start the application with:
|
||||
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
The application will be available at:
|
||||
|
||||
```
|
||||
http://localhost:8000
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- Make sure your GPU drivers are compatible with the CUDA version installed.
|
||||
- If GPU support is not required, the PyTorch CPU version can also be used.
|
||||
|
||||
## Rules Section
|
||||
|
||||
### Naming Convention Exception for LEGACY Code
|
||||
- Legacy code (deprecated or intended for migration) may be referenced or used in the codebase only when accompanied by a conventional comment indicating its legacy status (e.g., `# LEGACY: to be refactored` or `# LEGACY: deprecated`).
|
||||
- Such legacy code should not be extended for new features; it is intended for read/reference only or as a stepping stone toward modernization.
|
||||
- When introducing new modules or files, follow the standard naming convention (e.g., snake_case for Python files, PascalCase for classes, etc.). Legacy exceptions are exempt only when explicitly marked.
|
||||
|
||||
### Visualization Guidelines
|
||||
- Prefer text-based diagram descriptions using PlantUML or Mermaid syntax for version control and diffability.
|
||||
- If a diagram must be drawn by hand, use raster image formats such as .png or .jpg.
|
||||
- Avoid binary formats like PDF, PPT, or other proprietary formats for diagrams in the repository.
|
||||
@@ -0,0 +1,212 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Architecture Review: VKIST Ultrasound Codebase</title>
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
primary: '#2563eb',
|
||||
secondary: '#64748b'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.min.js"></script>
|
||||
<script>
|
||||
mermaid.initialize({
|
||||
startOnLoad: true,
|
||||
theme: 'neutral',
|
||||
});
|
||||
</script>
|
||||
</head>
|
||||
<body class="bg-gray-50 text-gray-800 antialiased p-6 md:p-12">
|
||||
<main class="max-w-5xl mx-auto">
|
||||
|
||||
<header class="mb-8 border-b border-gray-200 pb-6">
|
||||
<h1 class="text-3xl font-extrabold text-gray-900 mb-2">Architecture Review: VKIST Ultrasound Codebase</h1>
|
||||
<p class="text-lg text-gray-600">Identified deepening opportunities to improve testability and AI-navigability.</p>
|
||||
</header>
|
||||
|
||||
<article class="bg-white rounded-xl shadow-sm border border-gray-200 p-6 md:p-8 mb-8 transition-shadow hover:shadow-md">
|
||||
|
||||
<div class="flex flex-col md:flex-row md:items-center justify-between gap-4 mb-6">
|
||||
<h2 class="text-2xl font-bold text-gray-900">Candidate: Refactor app.py into modular services</h2>
|
||||
<div>
|
||||
<span class="px-3 py-1 rounded-full text-xs font-semibold uppercase tracking-wider bg-red-100 text-red-800">
|
||||
Priority: Strong
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="space-y-4 mb-6">
|
||||
<p class="text-sm bg-gray-50 p-3 rounded-lg border border-gray-100 text-gray-700">
|
||||
<strong class="text-gray-900">Files Affected:</strong> <code class="text-blue-600">app.py</code>
|
||||
<span class="text-gray-400 mx-2">→</span>
|
||||
<strong class="text-gray-900">Proposed:</strong>
|
||||
<code class="bg-white px-1.5 py-0.5 rounded border text-xs">inference_pipeline.py</code>,
|
||||
<code class="bg-white px-1.5 py-0.5 rounded border text-xs">model_loader.py</code>,
|
||||
<code class="bg-white px-1.5 py-0.5 rounded border text-xs">preprocessor.py</code>,
|
||||
<code class="bg-white px-1.5 py-0.5 rounded border text-xs">postprocessor.py</code>
|
||||
</p>
|
||||
|
||||
<div>
|
||||
<h3 class="text-sm font-semibold uppercase tracking-wider text-gray-500 mb-1">The Problem</h3>
|
||||
<p class="text-gray-700 leading-relaxed">
|
||||
The main application file (<code class="bg-gray-100 px-1 rounded text-sm text-red-600">app.py</code>) is acting as a "god object". It manages web framework setup, model loading, inference pipelines, preprocessing, postprocessing, utilities, and API endpoints simultaneously. This creates **low architectural depth** (the interface is nearly as complex as its implementation), **poor testability** (isolated pipeline units cannot be targeted), and **low locality** (unrelated logic must be parsed to fix individual features).
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<h3 class="text-sm font-semibold uppercase tracking-wider text-gray-500 mb-1">Proposed Solution</h3>
|
||||
<p class="text-gray-700 mb-2">Split operational concerns into highly focused, cleanly encapsulated modules:</p>
|
||||
<ul class="space-y-2 text-gray-700 pl-4 list-disc">
|
||||
<li><strong><code class="text-gray-900">ModelLoader</code>:</strong> Handles asynchronous loading and persistent caching of PyTorch weights (angle, inflammation, segmentation models).</li>
|
||||
<li><strong><code class="text-gray-900">Preprocessor</code>:</strong> Isolates structural image transformations and CLAHE enhancement parameters.</li>
|
||||
<li><strong><code class="text-gray-900">InferencePipeline</code>:</strong> Orchestrates execution flows: angle classification <span class="text-gray-400">→</span> inflammation detection <span class="text-gray-400">→</span> conditional segmentation <span class="text-gray-400">→</span> sizing/severity metrics.</li>
|
||||
<li><strong><code class="text-gray-900">Postprocessor</code>:</strong> Houses structural thickness evaluations, severity metrics calculations, and visual overlay generators.</li>
|
||||
<li><strong><code class="text-gray-900">APIHandler</code>:</strong> A thin, declarative FastAPI endpoint wrapper delegating exclusively to the primary <code class="text-sm">InferencePipeline</code>.</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<h3 class="text-sm font-semibold uppercase tracking-wider text-gray-500 mb-1">Expected Architecture Benefits</h3>
|
||||
<ul class="grid grid-cols-1 md:grid-cols-2 gap-3 text-gray-700">
|
||||
<li class="bg-blue-50/50 p-3 rounded-lg border border-blue-100">
|
||||
<strong class="text-blue-900 block mb-0.5">Architectural Depth</strong> High complexity is hidden beneath simple interfaces (e.g., <code class="text-xs">analyze_image(img) -> Result</code>).
|
||||
</li>
|
||||
<li class="bg-blue-50/50 p-3 rounded-lg border border-blue-100">
|
||||
<strong class="text-blue-900 block mb-0.5">High Code Locality</strong> Bugs in sizing algorithms map cleanly to the postprocessor without risking model caching logic.
|
||||
</li>
|
||||
<li class="bg-blue-50/50 p-3 rounded-lg border border-blue-100">
|
||||
<strong class="text-blue-900 block mb-0.5">Simplified Testing</strong> Individual pipeline states and weights can be unit-tested seamlessly using mock behaviors.
|
||||
</li>
|
||||
<li class="bg-blue-50/50 p-3 rounded-lg border border-blue-100">
|
||||
<strong class="text-blue-900 block mb-0.5">AI and Dev Navigation</strong> Highly categorized file trees accelerate context gathering for language models and onboarding engineers.
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 xl:grid-cols-2 gap-6 mt-8">
|
||||
<div class="border border-gray-200 rounded-xl p-4 bg-gray-50">
|
||||
<h4 class="text-base font-bold text-gray-800 mb-3 flex items-center gap-2">
|
||||
<span class="w-2.5 h-2.5 rounded-full bg-red-500"></span> Before (Shallow Architecture)
|
||||
</h4>
|
||||
<div class="mermaid bg-white p-4 rounded-lg border border-gray-100 shadow-inner overflow-x-auto">
|
||||
graph TD
|
||||
A[FastAPI Endpoint] --> B[load_angle_model]
|
||||
A --> C[load_inflammation_model]
|
||||
A --> D[load_segmentation_model_sup]
|
||||
A --> E[load_segmentation_model_post]
|
||||
A --> F[predict_angle]
|
||||
A --> G[predict_inflammation]
|
||||
A --> H[segment_image]
|
||||
A --> I[measure_thickness_new]
|
||||
A --> J[analyze_inflammation_severity]
|
||||
A --> K[create_segmentation_overlay]
|
||||
A --> L[apply_clahe]
|
||||
A --> M[save_patient_data]
|
||||
|
||||
style A fill:#f87171,stroke:#ef4444,stroke-width:2px,color:#fff
|
||||
style B fill:#fee2e2,stroke:#f87171
|
||||
style C fill:#fee2e2,stroke:#f87171
|
||||
style D fill:#fee2e2,stroke:#f87171
|
||||
style E fill:#fee2e2,stroke:#f87171
|
||||
style F fill:#fee2e2,stroke:#f87171
|
||||
style G fill:#fee2e2,stroke:#f87171
|
||||
style H fill:#fee2e2,stroke:#f87171
|
||||
style I fill:#fee2e2,stroke:#f87171
|
||||
style J fill:#fee2e2,stroke:#f87171
|
||||
style K fill:#fee2e2,stroke:#f87171
|
||||
style L fill:#fee2e2,stroke:#f87171
|
||||
style M fill:#fee2e2,stroke:#f87171
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="border border-gray-200 rounded-xl p-4 bg-gray-50">
|
||||
<h4 class="text-base font-bold text-gray-800 mb-3 flex items-center gap-2">
|
||||
<span class="w-2.5 h-2.5 rounded-full bg-green-500"></span> After (Deepened Architecture)
|
||||
</h4>
|
||||
<div class="mermaid bg-white p-4 rounded-lg border border-gray-100 shadow-inner overflow-x-auto">
|
||||
graph TD
|
||||
subgraph API [API Gateway Router]
|
||||
A[FastAPI Endpoint] --> B[InferencePipeline.analyze]
|
||||
end
|
||||
|
||||
subgraph InferencePipeline [Core Workflow Orchestrator]
|
||||
B --> C[ModelLoader.get_angle_model]
|
||||
B --> D[Preprocess]
|
||||
B --> E[predict_angle]
|
||||
E --> F{Inflammation?}
|
||||
F -->|Yes| G[ModelLoader.get_inflammation_model]
|
||||
G --> H[predict_inflammation]
|
||||
H --> I{Inflammation Detected?}
|
||||
I -->|Yes| J[ModelLoader.get_seg_model_sup/post]
|
||||
J --> K[segment_image]
|
||||
K --> L[measure_thickness]
|
||||
K --> M[analyze_severity]
|
||||
L --> N[Result: Measurement]
|
||||
M --> O[Result: Severity]
|
||||
K --> P[Result: Segmentation Masks]
|
||||
E -->|No| Q[Result: Angle Only]
|
||||
F -->|No| Q
|
||||
end
|
||||
|
||||
subgraph ModelLoader [Model Assets Lifecycle]
|
||||
C --> C1[Load Angle Model]
|
||||
G --> G1[Load Inflammation Model]
|
||||
J --> J1[Load Segmentation Model]
|
||||
end
|
||||
|
||||
subgraph Preprocessor [Image Engineering]
|
||||
D --> D1[Transforms]
|
||||
D --> D2[CLAHE]
|
||||
end
|
||||
|
||||
subgraph Postprocessor [Metrics & Artifact Generation]
|
||||
L --> L1[Thickness Calc]
|
||||
M --> M1[Severity Scoring]
|
||||
end
|
||||
|
||||
style A fill:#3b82f6,stroke:#2563eb,stroke-width:2px,color:#fff
|
||||
style B fill:#dbeafe,stroke:#60a5fa
|
||||
style C fill:#dbeafe,stroke:#60a5fa
|
||||
style D fill:#dbeafe,stroke:#60a5fa
|
||||
style E fill:#dbeafe,stroke:#60a5fa
|
||||
style F fill:#dbeafe,stroke:#60a5fa
|
||||
style G fill:#dbeafe,stroke:#60a5fa
|
||||
style H fill:#dbeafe,stroke:#60a5fa
|
||||
style I fill:#dbeafe,stroke:#60a5fa
|
||||
style J fill:#dbeafe,stroke:#60a5fa
|
||||
style K fill:#dbeafe,stroke:#60a5fa
|
||||
style L fill:#dbeafe,stroke:#60a5fa
|
||||
style M fill:#dbeafe,stroke:#60a5fa
|
||||
style N fill:#e0f2fe,stroke:#0284c7
|
||||
style O fill:#e0f2fe,stroke:#0284c7
|
||||
style P fill:#e0f2fe,stroke:#0284c7
|
||||
style Q fill:#e0f2fe,stroke:#0284c7
|
||||
style C1 fill:#f3e8ff,stroke:#a855f7
|
||||
style G1 fill:#f3e8ff,stroke:#a855f7
|
||||
style J1 fill:#f3e8ff,stroke:#a855f7
|
||||
style D1 fill:#fef3c7,stroke:#d97706
|
||||
style D2 fill:#fef3c7,stroke:#d97706
|
||||
style L1 fill:#ecfccb,stroke:#65a30d
|
||||
style M1 fill:#ecfccb,stroke:#65a30d
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</article>
|
||||
|
||||
<footer class="text-center text-sm text-gray-500 mt-12">
|
||||
<p>Total candidates identified: <span class="font-semibold text-gray-700">1</span></p>
|
||||
</footer>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -7,16 +7,17 @@ Orchestrates API routers, role checks, Socratic circuit-breaker state evaluation
|
||||
Core Backend Team
|
||||
|
||||
## Boundary
|
||||
FastAPI server, API routers, authentication middleware, circuit breaker engine, report generator, RAG coordinator, ledger logger, and connections to Postgres, S3, Redis, Triton, Qdrant, ladybugDB.
|
||||
FastAPI server, API routers, authentication middleware, circuit breaker engine, report generator, RAG coordinator, RAG-Referee (BERT), ledger logger, and connections to Postgres + pgvector, S3 (MinIO), Redis, Triton, ladybugDB.
|
||||
|
||||
## Internal Design
|
||||
- Built with FastAPI (Python) and Uvicorn for async HTTP server.
|
||||
- Authentication middleware validates JWT tokens and enforces RBAC (roles: RO_RADIOLOGIST, RO_THERAPIST).
|
||||
- Socratic circuit-breaker engine monitors interaction telemetry (hover duration, decision time, override magnitude) and triggers safety dialogs.
|
||||
- Clinical Report Engine uses ReportLab to generate bilingual PDF reports per Circular 46/2018/TT-BYT.
|
||||
- RAG Coordinator orchestrates Retrieval-Augmented Generation: dense vector lookup in Qdrant, graph traversal in ladybugDB, prompt enrichment, LLM generation on Triton (PhoGPT/MedGemma), and hallucination guarding.
|
||||
- RAG Coordinator orchestrates Retrieval-Augmented Generation: dense vector lookup in pgvector (PostgreSQL HNSW), graph traversal in ladybugDB, mandatory pre-generation retrieval, prompt enrichment, LLM generation on browser WebLLM (GemmaE2B) or cloud Vertex AI (MedGemma via NFR-16a), and hallucination guarding via BERT RAG-Referee.
|
||||
- NLP Scrubber (Microsoft Presidio): re-verifies client edge redaction, refines residual PII, and returns error if unresolvable.
|
||||
- Ledger Logger appends immutable, cryptographically chained audit logs to Postgres via triggers preventing UPDATE/DELETE.
|
||||
- Connections: Postgres (via SQLAlchemy), S3 (via boto3), Redis (via redis-py), Triton (via gRPC), Qdrant (via gRPC/HTTP), ladybugDB (via in-process C++ bindings).
|
||||
- Connections: Postgres + pgvector (via SQLAlchemy), S3 (via boto3), Redis (via redis-py), Triton (via gRPC — CV + EmbeddingGemma only), ladybugDB (via in-process C++ bindings).
|
||||
- Model weights loaded at startup from internal registry; cached in memory.
|
||||
- API endpoints layered: public clinical (sessions, analysis, reports, feedback) and internal/local safety (explanations, safety, drift, RAG, activations, annotations, ground-truth, escalation, morphology, telemetry).
|
||||
|
||||
@@ -11,10 +11,10 @@ Qdrant vector database instances, ladybugDB graph database instances, embedding
|
||||
|
||||
## Internal Design
|
||||
- Hybrid knowledge architecture: vector similarity search + graph traversal
|
||||
- Qdrant: stores guideline section embeddings (BioClinicalBERT, PubMedBERT) with payload metadata
|
||||
- pgVec: stores guideline section embeddings (BioClinicalBERT, PubMedBERT) with payload metadata
|
||||
- ladybugDB: stores ontology concepts (SNOMED-CT, LOINC, RadLex) and relational axioms
|
||||
- EmbeddingGemma: generates 768-dimension vectors for text chunks
|
||||
- PhoGPT/MedGPT: LLM for answer generation with constrained decoding
|
||||
- GemmaE2B/MedGemma: LLM for answer generation with constrained decoding
|
||||
- Retrieval pipeline: hybrid search (vector + BM25) → graph expansion → reranking
|
||||
- Grounding module: verifies LLM outputs against source guidelines with citation extraction
|
||||
- Arbitration engine: resolves conflicting evidence using belief propagation
|
||||
Submodule workspace/sprint_1_2/Design_Material/LEGACY/VKIST_ML/codebase-vkist-ultrasound-legacy deleted from 8b2758928a
570
workspace/sprint_1_2/SPRINT_1_2_ARCHITECTURE_SPEC.md
Normal file
570
workspace/sprint_1_2/SPRINT_1_2_ARCHITECTURE_SPEC.md
Normal file
@@ -0,0 +1,570 @@
|
||||
# Sprint 1-2 Architecture Specification
|
||||
|
||||
**Sprint:** Sprint 1 (June 15 – June 26) + Sprint 2 (June 29 – July 10)
|
||||
**Theme:** "The Fast PoC Baseline" + "Multi-Modal & NLP Integration"
|
||||
**Parent Document:** [SOLUTION_ARCHITECTURE_SPEC.md](../SOLUTION_ARCHITECTURE_SPEC.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Sprint Scope & Goals
|
||||
|
||||
### 1.1 Sprint 1: The Fast PoC Baseline
|
||||
- Establish interactive end-to-end processing pipeline
|
||||
- Rapid UI prototype with high-fidelity mockups
|
||||
- Configure initial FastAPI server pipelines to process raw array matrices
|
||||
- Output early inference mask previews onto browser-based preview window canvas
|
||||
- Wire ultrasound image ingest pathways into classification pipeline
|
||||
|
||||
### 1.2 Sprint 2: Multi-Modal & NLP Integration
|
||||
- Embed localized NLP translation modules
|
||||
- Implement client-side privacy scrubbing masks for Decree 13 compliance
|
||||
- Configure on-device string parsers to sanitize personal identification tokens
|
||||
- Deploy multi-modal diagnostic chat system
|
||||
- Build zero-friction NLP conduits for clinical abbreviations
|
||||
|
||||
---
|
||||
|
||||
## 2. Architectural Constraints for Sprint 1-2
|
||||
|
||||
| Constraint | Implementation |
|
||||
| --- | --- |
|
||||
| **Air-Gapped Hospital LAN** | No external cloud processing during normal operation. All inference and storage on-premise via K3s. NFR-16a governs emergency cloud fallback only. |
|
||||
| **Code & Issue Hosting (NFR-16a exception)** | GitLab CE and Jira run self-hosted on a cloud VM (not SaaS). All code, tickets, and pipeline configs transit over public internet to reach the VM. Compensating controls: no PHI in commits or tickets (pre-push hook + team policy); SSH-only git access with certificate pinning; daily GitLab RDB backup to hospital MinIO (not cloud storage); cloud VM IAM with minimum-role access and 2FA; VM access restricted to hospital whitelisted IPs. This exception is reviewed at PoC sign-off. |
|
||||
| **Network Bandwidth (10 Mbps max)** | Optimize payload sizes, leverage local caching (IndexedDB, Redis). |
|
||||
| **DICOM Compliance** | Process DICOM images via pydicom stream over local HTTPS. |
|
||||
| **Client Memory ≤ 150 MB** | PWA uses React + Zustand + LiteRT (no dedicated GPU required). |
|
||||
| **Latency ≤ 1.5s for inference** | Heavy ML offloaded to Triton Inference Server with ONNX/TensorRT. |
|
||||
| **Zero PHI Leakage** | Client-side scrubbing before network transfer; no PHI in Git/Jenkins. |
|
||||
|
||||
---
|
||||
|
||||
## 3. System Context Diagram (C4) - Sprint 1-2 Scope (FR-25 Synovitis Grading)
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_Sprint1_2_Context_FR25"
|
||||
!include <C4/C4_Context>
|
||||
|
||||
title System Context - Sprint 1-2: FR-25 Synovitis Grading & NLP Integration
|
||||
' Sprint 1-2 scope: FR-25 only with NLP safety layer
|
||||
' Active Users: UP5 | Governance: UP1, UP4 | Future: UP6, UP7, UP8
|
||||
|
||||
' === ACTORS IN SPRINT SCOPE ===
|
||||
Person(radiologist, "Diagnostic Radiologist (UP5)", "FR-25: load scan, review AI grade (0-3), confirm/override, finalize & sign, view GradCAM, engage circuit breaker, review RAG evidence")
|
||||
|
||||
' === GOVERNANCE / NFR ALIGNMENT ===
|
||||
Person_Ext(senior_expert, "Healthcare Senior Expert (UP1)", "Governance: clinical protocol validation, MOH guideline approval, model threshold sign-off")
|
||||
Person_Ext(support_staff, "Support Staff (UP4)", "Registration, case queue management")
|
||||
|
||||
' === FUTURE SPRINTS ===
|
||||
Person_Ext(therapist, "Physical Therapist (UP6)", "Future sprint: prescription parser, 3D mapping, kinetic overlay, patient education")
|
||||
Person_Ext(ortho_surgeon, "Orthopedic Surgeon (UP7)", "Future sprint: treatment planning, aggregated dashboard, 3D anatomy education")
|
||||
Person_Ext(patient_caregiver, "Patient & Caregiver (UP8)", "Future sprint: self-monitoring portal, inflammation tracker, medication reminders")
|
||||
|
||||
Person(admin, "System Administrator", "K3s deployment, model updates, NGINX failover, Prometheus/Grafana monitoring")
|
||||
|
||||
' === SYSTEM ===
|
||||
System(mpps, "VKIST MSK Processing System\n(FR-25 Sprint 1-2)", "Knee ultrasound AI: angle→inflammation→segmentation\nGrading: synovitis 0-3 with GradCAM\nSafety: Circuit Breaker, BERT drift, RAG-Referee\nNLP: Decree 13 scrubbing, GemmaE2B/MedGemma explanations\nReporting: Circular 46 PDF")
|
||||
|
||||
' === EXTERNAL SYSTEMS IN SCOPE ===
|
||||
System_Ext(pacs, "Hospital PACS / Ultrasound", "DICOM source (C-MOVE + direct capture)")
|
||||
System_Ext(emr, "Hospital EMR/HIS", "Finalized reports, audit logs (HL7/FHIR)")
|
||||
System_Ext(triton, "Triton Inference Server", "GPU inference: angle, inflammation, segmentation + embeddings")
|
||||
System_Ext(knowledge, "Clinical Knowledge Stack", "ladybugDB (ontology), pgvector (MOH guideline vectors in Postgres HNSW), EmbeddingGemma (RAG embedding model), GemmaE2B (browser WebLLM), MedGemma (cloud Vertex AI)")
|
||||
|
||||
' === RELATIONSHIPS ===
|
||||
Rel(radiologist, mpps, "Load scan, review grade, confirm/override, finalize, view explanations, engage safety dialog", "HTTPS")
|
||||
Rel(admin, mpps, "Deploys, monitors, configures", "HTTPS/SSH")
|
||||
Rel(senior_expert, mpps, "Validates protocols, approves thresholds", "HTTPS")
|
||||
Rel(support_staff, mpps, "Registration, case queue", "HTTPS")
|
||||
|
||||
Rel(therapist, mpps, "Not in Sprint 1-2 scope", "—")
|
||||
Rel(ortho_surgeon, mpps, "Not in Sprint 1-2 scope", "—")
|
||||
Rel(patient_caregiver, mpps, "Not in Sprint 1-2 scope", "—")
|
||||
|
||||
Rel(mpps, pacs, "DICOM import", "C-MOVE + upload")
|
||||
Rel(mpps, emr, "Finalized reports, audit", "HL7/FHIR")
|
||||
Rel(mpps, triton, "ML inference", "gRPC :8001")
|
||||
Rel(mpps, knowledge, "RAG + ontology + LLM", "gRPC/HTTP + C++")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Container Diagram (C4) - Sprint 1-2 Implementation
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_Containers_Sprint1_2"
|
||||
!include <C4/C4_Container>
|
||||
|
||||
title Container Diagram - VKIST MSK Platform (Sprint 1-2)
|
||||
|
||||
Person(radiologist, "Radiologist (UP5)", "Performs primary diagnosis, view validation, and severity grading.")
|
||||
Person(therapist, "Physical Therapist (UP6)", "Observes scans, evaluates kinetic data, and structures physical rehabilitation plans.")
|
||||
|
||||
System_Boundary(hospital_lan, "Air-Gapped Hospital LAN (Max 10 Mbps)") {
|
||||
|
||||
Container(pwa, "React PWA Frontend", "React, TS, Zustand, LiteRT, MediaPipe, Dexie.js", "Interactive visualization, local WebAssembly inference for view-angle validation, encrypted IndexedDB cache, DICOM upload, edge guardrail (Transformers.js BERT + OpenRedaction + pii-filter) in dedicated WebWorkers")
|
||||
|
||||
Container(nginx, "NGINX Reverse Proxy + Keepalived", "NGINX 1.27, Keepalived", "SSL/TLS termination, single VIP, instant failover (<=2s), routes to active FastAPI node")
|
||||
|
||||
System_Boundary(backend_servers, "Application Server Cluster") {
|
||||
Container(fastapi, "FastAPI Application Server", "Python, Uvicorn, SQLAlchemy, ReportLab", "API orchestration, angle classification routing, inflammation detection, segmentation post-processing, measurement, severity analysis, PDF report generation, client-side scrubbing validation")
|
||||
|
||||
System_Boundary(ml_inference, "ML Inference Stack (Sprint 1-2)") {
|
||||
Container(triton, "Triton Inference Server", "NVIDIA Triton, ONNX, OpenVINO, TensorRT", "Executes 3-step ML pipeline: angle classification, inflammation detection, segmentation (UNet3+/DeepLabV3)")
|
||||
ContainerDb(model_store, "Local Model Store", "ONNX/Torch weights on disk", "Stores model checkpoints mounted read-only into Triton")
|
||||
}
|
||||
|
||||
System_Boundary(observability, "Observability (Sprint 1-2)") {
|
||||
Container(prometheus, "Prometheus", "Prometheus", "Scrapes FastAPI and Triton metrics")
|
||||
Container(grafana, "Grafana", "Grafana", "Operational dashboards for inference latency, GPU utilization, cache hit rate")
|
||||
}
|
||||
|
||||
System_Boundary(data_stack, "Data Stack (Sprint 1-2)") {
|
||||
ContainerDb(postgres, "PostgreSQL + PostGIS", "PostgreSQL", "Stores session metadata, EMR records, spatial markers, audit ledger")
|
||||
ContainerDb(redis, "Redis Cache", "Redis", "Session state, JWT tokens, rate limiting")
|
||||
ContainerDb(s3, "S3 Object Store (MinIO)", "MinIO", "DICOM-derived images, segmentation overlays, Grad-CAM heatmaps, report blobs")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Rel(radiologist, pwa, "Uploads DICOM, views overlays, accepts/rejects grades")
|
||||
Rel(therapist, pwa, "Reviews scan results, annotates feedback")
|
||||
|
||||
Rel(pwa, nginx, "HTTPS requests with encrypted DICOM payloads", "HTTPS (Port 443)")
|
||||
Rel(nginx, fastapi, "Routes API requests", "HTTP/1.1 (Port 8000)")
|
||||
|
||||
Rel(fastapi, redis, "Session verification, rate limiting", "TCP (Port 6379)")
|
||||
Rel(fastapi, postgres, "Metadata, audit ledger, spatial queries", "SQL/TCP (Port 5432)")
|
||||
Rel(fastapi, s3, "Image and report blob storage", "S3 API")
|
||||
|
||||
Rel(fastapi, triton, "Offloads ML inference: angle, inflammation, segmentation", "gRPC (Port 8001)")
|
||||
Rel(triton, model_store, "Reads model weights", "Local FS")
|
||||
|
||||
Rel(prometheus, fastapi, "Request latency, error rate, DB pool", "HTTP /metrics")
|
||||
Rel(prometheus, triton, "Model latency, VRAM, GPU utilization", "HTTP /metrics")
|
||||
Rel(grafana, prometheus, "Dashboard queries", "HTTP (Port 9090)")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Component Diagram (C4) - FastAPI Server Internals (Sprint 1-2)
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_Backend_Components_Sprint1_2"
|
||||
!include <C4/C4_Component>
|
||||
|
||||
title Component Diagram - FastAPI Application Server (Sprint 1-2)
|
||||
|
||||
Container_Boundary(backend, "FastAPI Application Server") {
|
||||
|
||||
System_Boundary(api_routers, "API Routers") {
|
||||
Component(analyze_router, "Analysis Router", "FastAPI Router", "/api/analyze - Main ML pipeline endpoint\nHandles angle model selection, inflammation detection, segmentation routing, measurement, severity analysis")
|
||||
Component(save_router, "Save Router", "FastAPI Router", "/api/save - Patient data persistence\nSanitizes metadata, stores images and PDF reports")
|
||||
Component(export_router, "PDF Export Router", "FastAPI Router", "/api/export-pdf - On-demand PDF generation")
|
||||
Component(health_router, "Health Router", "FastAPI Router", "/api/health - Liveness probe")
|
||||
}
|
||||
|
||||
System_Boundary(ml_pipeline, "ML Pipeline Components (Sprint 1-2)") {
|
||||
Component(angle_classifier, "Angle Classifier", "PyTorch/ONNX", "Classifies ultrasound plane (med-lat, post-trans, sup-trans-flex, sup-up-long)\nModels: ConvNeXt-Tiny, DenseNet-121, ResNet-50, EfficientNet-B2, Swin-V2-S")
|
||||
Component(inflammation_detector, "Inflammation Detector", "PyTorch/ONNX", "Binary classifier for inflammation presence\nModel: EfficientNet-B0 (2-class)")
|
||||
Component(segmentation_engine, "Segmentation Engine", "PyTorch/ONNX", "Pixel-wise anatomical segmentation\nModels: DeepLabV3-ResNet50/101, UNet-ResNet101, EfficientFeedback, UNet3Plus-Attention")
|
||||
Component(measurement_engine, "Measurement Engine", "Python/NumPy/OpenCV", "Calculates synovium thickness in mm, effusion metrics\nPIXEL_TO_MM = 45.0 / 655.0")
|
||||
Component(severity_analyzer, "Severity Analyzer", "Python", "Classifies synovitis grade (0-3) with combined effusion+synovium scoring\nGenerates Vietnamese clinical descriptions")
|
||||
Component(preprocessor, "Image Preprocessor", "Python/PIL/OpenCV", "CLAHE contrast enhancement, resize, normalize")
|
||||
Component(overlay_renderer, "Overlay Renderer", "Python/PIL", "Creates color-coded segmentation overlays with measurement annotations")
|
||||
}
|
||||
|
||||
System_Boundary(knowledge_stack, "Knowledge Stack (Sprint 2)") {
|
||||
Component(nlp_scrubber, "Patient Data Scrubber", "Microsoft Presidio", "Re-verify edge redaction; refine residual PII; error if unresolvable")
|
||||
Component(rag_coordinator, "RAG Coordinator", "Python", "Retrieves MOH guideline passages from pgvector (PostgreSQL HNSW index); mandatory pre-generation retrieval for all LLM tiers (browser WebLLM and cloud MedGemma)")
|
||||
Component(ontology_query, "Ontology Query Engine", "C++ bindings", "ladybugDB graph traversal for anatomical entity relationships")
|
||||
Component(guardrail_edge, "Edge Guardrail", "Transformers.js BERT", "Client-side hallucination/mal-intention/scope-breach scoring; triggers session termination + cloud mitigate via FastAPI guardrail-check endpoint")
|
||||
Component(referee, "RAG-Referee", "BERT classifier", "Server-side 3-axis citation contestant validation (attribution, cohesion, factual contestant status)")
|
||||
}
|
||||
|
||||
System_Boundary(reporting, "Reporting & Audit") {
|
||||
Component(pdf_generator, "PDF Report Generator", "ReportLab", "Bilingual (VI/EN) medical reports per Circular 46/2018/TT-BYT")
|
||||
Component(audit_logger, "Immutable Audit Logger", "SQLAlchemy/Postgres", "Append-only clinical decision and AI interaction logs\nPrevents UPDATE/DELETE via DB triggers")
|
||||
}
|
||||
|
||||
System_Boundary(infrastructure, "Infrastructure") {
|
||||
Component(cache_client, "Redis Client", "redis-py", "Session state, JWT tokens, rate limiting")
|
||||
Component(storage_client, "S3 Client", "boto3", "DICOM images, overlays, reports, Grad-CAM heatmaps")
|
||||
Component(db_client, "PostgreSQL Client", "SQLAlchemy", "Relational/spatial queries, audit ledger")
|
||||
Component(triton_client, "Triton gRPC Client", "gRPC", "Inference offloading for heavy ML models")
|
||||
}
|
||||
}
|
||||
|
||||
' Internal relationships
|
||||
Component(analyze_router, preprocessor, "Uploads image, applies CLAHE")
|
||||
Component(analyze_router, angle_classifier, "Classifies angle plane")
|
||||
Component(analyze_router, inflammation_detector, "Detects inflammation (if post-trans or sup-up-long)")
|
||||
Component(analyze_router, segmentation_engine, "Runs segmentation (if inflammation detected)")
|
||||
Component(analyze_router, measurement_engine, "Measures thickness (sup-up-long only)")
|
||||
Component(analyze_router, severity_analyzer, "Grades synovitis severity")
|
||||
Component(analyze_router, overlay_renderer, "Generates visualization overlay")
|
||||
|
||||
Component(save_router, nlp_scrubber, "Validates scrubbing before save")
|
||||
Component(save_router, pdf_generator, "Triggers PDF generation")
|
||||
Component(save_router, audit_logger, "Logs save action immutably")
|
||||
|
||||
Component(export_router, pdf_generator, "Generates on-demand PDF")
|
||||
|
||||
Component(rag_coordinator, triton_client, "RAG embedding extraction via Triton (EmbeddingGemma)")
|
||||
Component(rag_coordinator, ontology_query, "Graph traversal for evidence")
|
||||
Component(rag_coordinator, referee, "Mandatory RAG-Referee validation per axis")
|
||||
Component(guardrail_edge, "Edge Guardrail", "Transformers.js BERT", "Scores hallucination/mal-intention/scope-breach in WebWorker")
|
||||
Component(guardrail_edge, referee, "Violation signal → server-side referee gate")
|
||||
|
||||
' Infrastructure connections
|
||||
Component(analyze_router, cache_client, "Session validation")
|
||||
Component(analyze_router, storage_client, "Image/report I/O")
|
||||
Component(analyze_router, db_client, "Metadata, audit")
|
||||
Component(analyze_router, triton_client, "ML inference offload")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Deployment Diagram (C4) - Sprint 1-2 Hospital LAN Runtime
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_Deployment_Sprint1_2"
|
||||
!include <C4/C4_Deployment>
|
||||
|
||||
title Deployment Diagram - VKIST MSK Platform (Sprint 1-2)
|
||||
|
||||
Deployment_Node(hw, "Hospital On-Premise Hardware\n(Dell PowerEdge / Local Server)") {
|
||||
Deployment_Node(k8s_alt, "K3s Runtime (Sprint 1-2)") {
|
||||
Deployment_Node(nginx_node, "NGINX Container") {
|
||||
Container(nginx_c, "NGINX 1.27 + Keepalived", "Reverse proxy, SSL, VIP failover")
|
||||
}
|
||||
Deployment_Node(fastapi_node, "FastAPI Container") {
|
||||
Container(fastapi_c, "FastAPI Uvicorn", "Python app: analysis, save, export, NLP scrub, RAG")
|
||||
}
|
||||
Deployment_Node(prom_node, "Prometheus Container") {
|
||||
Container(prom_c, "Prometheus", "Metrics collection")
|
||||
}
|
||||
Deployment_Node(grafana_node, "Grafana Container") {
|
||||
Container(graf_c, "Grafana", "Dashboards")
|
||||
}
|
||||
}
|
||||
|
||||
Deployment_Node(vm_db, "Database / Storage VM") {
|
||||
Deployment_Node(pg_node, "PostgreSQL + pgvector Node") {
|
||||
ContainerDb(pg_c, "PostgreSQL + PostGIS + pgvector", "Primary DB, vector HNSW index, audit ledger")
|
||||
}
|
||||
Deployment_Node(redis_node, "Redis Node") {
|
||||
ContainerDb(redis_c, "Redis", "Session cache, rate limit")
|
||||
}
|
||||
Deployment_Node(s3_node, "MinIO Node") {
|
||||
ContainerDb(s3_c, "MinIO S3", "Object storage for images and reports")
|
||||
}
|
||||
}
|
||||
|
||||
Deployment_Node(triton_hw, "Inference Server (GPU Node)") {
|
||||
Deployment_Node(triton_node, "Triton Container") {
|
||||
Container(triton_c, "Triton Inference Server", "ONNX/TensorRT models, ensemble pipelines")
|
||||
}
|
||||
Deployment_Node(model_vol, "Model Volume") {
|
||||
ContainerDb(model_c, "Model Weights", "Read-only mount: .pth / .onnx files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Deployment_Node(client, "Clinician Workstation / Tablet") {
|
||||
ContainerDb(browser, "Browser PWA", "React, Service Worker, IndexedDB")
|
||||
}
|
||||
|
||||
Rel(browser, nginx_c, "HTTPS (port 443)", "TLS 1.3")
|
||||
Rel(nginx_c, fastapi_c, "HTTP (port 8000)", "Internal LAN")
|
||||
Rel(fastapi_c, pg_c, "SQL/TCP (port 5432)", "LAN")
|
||||
Rel(fastapi_c, s3_c, "S3 API", "LAN")
|
||||
Rel(fastapi_c, triton_c, "gRPC (port 8001)", "LAN (or same host)")
|
||||
|
||||
Rel(prom_c, fastapi_c, "/metrics scrape", "HTTP")
|
||||
Rel(prom_c, triton_c, "/metrics scrape", "HTTP")
|
||||
Rel(graf_c, prom_c, "Dashboard", "HTTP")
|
||||
|
||||
Rel(triton_c, model_vol, "Loads weights", "Read-only FS")
|
||||
|
||||
Rel(browser, browser, "Local-only WebAssembly inference (LiteRT/MediaPipe)", "No network")
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Sprint 1-2 ML Workflow: Ultrasound Processing Pipeline
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_ML_Pipeline_Sprint1_2"
|
||||
@startuml
|
||||
title ML Pipeline - Knee Ultrasound Analysis (Sprint 1-2)
|
||||
|
||||
start
|
||||
|
||||
:Clinician uploads DICOM via PWA;
|
||||
:PWA encrypts payload, sends over HTTPS;
|
||||
|
||||
:NGINX routes to active FastAPI node;
|
||||
|
||||
:FastAPI receives image;
|
||||
:Apply CLAHE preprocessing (contrast enhancement);
|
||||
|
||||
:RUN ANGLE CLASSIFICATION;
|
||||
:Model: ConvNeXt/DenseNet/ResNet/EfficientNet/Swin-V2;
|
||||
:Classes: med-lat, post-trans, sup-trans-flex, sup-up-long;
|
||||
|
||||
if (Angle class?) then (post-trans)
|
||||
: Inflammation Detection;
|
||||
:Model: EfficientNet-B0 (binary);
|
||||
if (Inflammation?) then (yes)
|
||||
:Post-Trans Segmentation;
|
||||
:Model: DeepLabV3-ResNet101;
|
||||
:Classes: fat, tendon, muscle, femur, artery, synovium, baker's cyst;
|
||||
:Generate segmentation masks;
|
||||
:Create overlay image;
|
||||
else (no)
|
||||
:Skip segmentation;
|
||||
:Set severity = 0;
|
||||
endif
|
||||
elseif (sup-up-long)
|
||||
: Inflammation Detection;
|
||||
if (Inflammation?) then (yes)
|
||||
:Suprapatellar Segmentation;
|
||||
:Model: UNet3Plus-Attention / DeepLabV3 / EfficientFeedback;
|
||||
:Classes: effusion, fat, fat-pat, femur, synovium, tendon;
|
||||
:Generate segmentation masks;
|
||||
|
||||
:Measure Synovium Thickness;
|
||||
:ROI: middle 1/3 of bounding box;
|
||||
:Unit: mm (PIXEL_TO_MM = 45/655);
|
||||
|
||||
:Create overlay with measurement annotations;
|
||||
|
||||
:Analyze Severity;
|
||||
:Combined score: effusion (60%) + synovium (40%);
|
||||
:Grade: 0 (Rất nhẹ) / 1 (Nhẹ) / 2 (Trung bình) / 3 (Nặng);
|
||||
else (no)
|
||||
:Skip segmentation and measurement;
|
||||
:Set severity = 0;
|
||||
endif
|
||||
else (other angles)
|
||||
:Only angle classification result returned;
|
||||
endif
|
||||
|
||||
:Generate enhanced image (CLAHE);
|
||||
:Assemble JSON response with overlays, masks, measurements, severity;
|
||||
|
||||
if (Save requested?) then (yes)
|
||||
:NLP Scrubber validates Decree 13 compliance;
|
||||
:Store images to MinIO S3;
|
||||
:Store metadata to PostgreSQL;
|
||||
:Append immutable audit log;
|
||||
:Generate bilingual PDF report;
|
||||
endif
|
||||
|
||||
:Return response to PWA;
|
||||
|
||||
stop
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Data Flow Diagram (Activity View)
|
||||
|
||||
```plantuml
|
||||
@startuml "VKIST_MSK_DataFlow_Sprint1_2"
|
||||
title Data Flow - VKIST MSK Platform (Sprint 1-2)
|
||||
|
||||
|PWA Client|
|
||||
start
|
||||
:Upload DICOM image;
|
||||
:Apply CLAHE preprocessing locally;
|
||||
:Encrypt payload;
|
||||
:Send HTTPS POST to /api/analyze;
|
||||
|
||||
|NGINX Gateway|
|
||||
:Receive request;
|
||||
:SSL termination;
|
||||
:Route to active FastAPI node;
|
||||
|
||||
|FastAPI Server|
|
||||
:Receive image;
|
||||
:Validate JWT + RBAC;
|
||||
:Load angle classifier model;
|
||||
:Run angle classification;
|
||||
if (Angle class?) then (post-trans)
|
||||
:Load inflammation detector;
|
||||
:Run inflammation detection;
|
||||
if (Detected?) then (yes)
|
||||
:Load post-trans segmentation model;
|
||||
:Run DeepLabV3-ResNet101 inference via Triton;
|
||||
:Receive masks from Triton;
|
||||
:Create segmentation overlay;
|
||||
:Return JSON with overlay + masks;
|
||||
else (no)
|
||||
:Return JSON with angle + inflammation only;
|
||||
endif
|
||||
elseif (sup-up-long)
|
||||
:Load inflammation detector;
|
||||
:Run inflammation detection;
|
||||
if (Detected?) then (yes)
|
||||
:Load suprapatellar segmentation model;
|
||||
:Run UNet3Plus-Attention inference via Triton;
|
||||
:Receive masks from Triton;
|
||||
:Measure synovium thickness (ROI middle 1/3);
|
||||
:Calculate effusion metrics;
|
||||
:Analyze severity (0-3 combined score);
|
||||
:Create overlay with measurement annotations;
|
||||
:Return JSON with overlay + measurement + severity;
|
||||
else (no)
|
||||
:Return JSON with angle + inflammation only;
|
||||
endif
|
||||
else (other)
|
||||
:Return angle classification only;
|
||||
endif
|
||||
|
||||
|PWA Client|
|
||||
:Receive JSON response;
|
||||
:Display enhanced image + overlays;
|
||||
:Render measurement annotations;
|
||||
:Show severity grade with color coding;
|
||||
|
||||
if (User action: Save?) then (yes)
|
||||
|PWA Client|
|
||||
:Run NLP scrubber (Decree 13);
|
||||
:Remove PII from metadata;
|
||||
:Send HTTPS POST to /api/save;
|
||||
|
||||
|FastAPI Server|
|
||||
:Validate scrubbing compliance;
|
||||
:Store images to MinIO S3;
|
||||
:Store metadata to PostgreSQL;
|
||||
:Append immutable audit log;
|
||||
:Generate bilingual PDF report (Circular 46);
|
||||
:Return success + folder path;
|
||||
|
||||
|PWA Client|
|
||||
:Confirm save to user;
|
||||
endif
|
||||
|
||||
stop
|
||||
|
||||
@enduml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Non-Functional Requirements Coverage (Sprint 1-2)
|
||||
|
||||
| NFR | Sprint 1-2 Implementation |
|
||||
| --- | --- |
|
||||
| **NFR-1** (DICOM Speed ≤ 3.0s) | Local Triton inference + Redis caching of recent sessions |
|
||||
| **NFR-4** (Client Memory ≤ 150 MB) | React PWA with Zustand; LiteRT WebAssembly (no GPU); Dexie.js for local cache |
|
||||
| **NFR-5** (Inference ≤ 1.5s) | Triton with TensorRT/OpenVINO quantization; ONNX runtime |
|
||||
| **NFR-6** (VRAM ≤ 2 GB) | Quantized ONNX models; batch size 1; DeepLabV3-ResNet101 optimized |
|
||||
| **NFR-7** (UI Refresh ≤ 200ms) | Token streaming via SSE; frontend state updates async |
|
||||
| **NFR-8** (Fault Tolerance) | IndexedDB local cache; service worker offline mode; session recovery |
|
||||
| **NFR-9** (Availability ≥ 99.9%) | NGINX + Keepalived active-passive VIP; Docker restart policies |
|
||||
| **NFR-10** (Generative Safety) | Edge guardrail: prompt rules + BERT detection (hallucination/mal-intention/scope-breach) → session termination → cloud mitigate (Vertex AI). Mandatory RAG pre-processing for all LLM tiers (not optional tool calling). RAG-Referee citation contestant validation (3-axis). Server-side Decree 13 redaction ground-check before Vertex AI egress. |
|
||||
| **NFR-11** (Onboarding ≤ 45 min) | High-fidelity Figma prototypes in Sprint 1; structured workflows |
|
||||
| **NFR-13** (Grad-CAM Zero-Click) | Overlay rendered directly on upload; zero extra UI steps |
|
||||
| **NFR-14** (No client GPU) | PWA falls back to CPU-bound rendering; LiteRT for lightweight inference |
|
||||
| **NFR-16** (Air-Gapped) | Entire inference and storage stack on-premise via K3s; no external cloud for clinical data. NFR-16a exception: GitLab/Jira on cloud VM with compensating controls. |
|
||||
| **NFR-17** (Immutable Audit) | PostgreSQL triggers prevent UPDATE/DELETE on audit tables |
|
||||
| **NFR-18** (RAG Citations) | pgvector retrieves MOH guideline passages (PostgreSQL HNSW index); LLM generates footnoted explanations |
|
||||
| **NFR-19** (HITL Gate) | Database state machine prevents FINALIZED status without clinician digital signature |
|
||||
|
||||
---
|
||||
|
||||
## 10. Key Decree 13 / Circular 46 Compliance (Sprint 2)
|
||||
|
||||
### 10.1 Decree 13/2023/ND-CP - Personal Data Protection
|
||||
- Client-side scrubbing via `nlp_scrubber` component before any network transfer
|
||||
- Regex-based PII removal (names, IDs, phone numbers) in browser
|
||||
- No PHI stored in Git, Jenkins artifacts, or external systems
|
||||
|
||||
### 10.2 Circular 46/2018/TT-BYT - EMR Compliance
|
||||
- PDF reports generated per official MOH format (bilingual VI/EN)
|
||||
- Audit trail immutable via database triggers
|
||||
- Reports stored with cryptographic checksums in MinIO
|
||||
|
||||
---
|
||||
|
||||
## 11. Infrastructure Decisions (Sprint 1-2)
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
| --- | --- | --- |
|
||||
| **Orchestration** | K3s (Kubernetes-certified, lightweight edge distribution) | Chosen over Docker Compose, Docker Swarm, Nomad, ECS Fargate, Cloud Run. NFR-16 requires on-premise, eliminating cloud-only options. Docker Swarm offers lowest PoC cost but is in maintenance mode with highest migration overhead. K3s is already production-grade; scaling path is multi-cluster federation, not platform replacement. |
|
||||
| **CI/CD** | Jenkins on hospital K3s | Runs inside trusted LAN; connects to cloud-hosted GitLab for source. No external build triggers. |
|
||||
| **Code Hosting** | Self-hosted GitLab CE on cloud VM (NFR-16a exception) | Reliability over hospital-hardware self-hosting; not SaaS. Compensating controls: no PHI in commits (pre-push hook), SSH-only access with cert pinning, daily RDB backup to hospital MinIO, cloud IAM with 2FA + minimum-role, IP whitelist to hospital networks. Exception reviewed at PoC sign-off. |
|
||||
| **Issue Tracking** | Self-hosted Jira on same cloud VM (NFR-16a exception) | Clinical feedback and tickets stay within controlled access boundary. No PHI in tickets per team policy. Same compensating controls as GitLab. |
|
||||
| **Model Serving** | Triton Inference Server | ONNX/TensorRT support; ensemble pipelines; HTTP/gRPC |
|
||||
| **Reverse Proxy** | NGINX + Keepalived | VIP for high availability; SSL termination; instant failover |
|
||||
| **Local LLM** | Browser WebLLM (GemmaE2B) + Cloud MedGemma (Vertex AI) | Browser: Vietnamese + clinical language support, local inference (air-gapped). Cloud: MedGemma for NFR-16a fallback + BERT-triggered arbiter. Triton hosts CV models + EmbeddingGemma only. |
|
||||
| **Vector Search** | pgvector (PostgreSQL HNSW index) | Already deployed with Postgres; zero additional infrastructure; ~15K MOH vectors at ~5-20ms query latency fits NFR-7. Complex SQL filtering for clinical queries. Qdrant deferred to Phase 2. |
|
||||
| **Ontology DB** | ladybugDB embedded | C++ library embedded in FastAPI process; SNOMED-CT/LOINC mappings |
|
||||
|
||||
---
|
||||
|
||||
## 12. Technology Stack Summary
|
||||
|
||||
| Layer | Technology | Purpose |
|
||||
| --- | --- | --- |
|
||||
| Frontend | React, TypeScript, Zustand, Dexie.js, LiteRT, MediaPipe, Transformers.js, OpenRedaction, pii-filter, js-data-anonymizer | PWA with offline support, local inference, and edge guardrail (BERT hallucination/mal-intention detection, Decree 13 PII scrubbing) |
|
||||
| Guardrail | Transformers.js BERT (WebWorker), OpenRedaction, pii-filter, js-data-anonymizer, FastAPI `phi_scrub` middleware, Vertex AI Model Garden safety filters | Edge behavior control without NeMo/GuardrailsAI; prompt-rule + BERT detection; session termination + cloud mitigate; mandatory RAG pre-processing (not optional tool-call); server-side redaction ground-check before NFR-16a egress |
|
||||
| Gateway | NGINX 1.27 + Keepalived | SSL termination, VIP, load balancing |
|
||||
| API | FastAPI, Uvicorn, SQLAlchemy, ReportLab | REST API, ORM, PDF generation |
|
||||
| ML Inference | Triton, ONNX, TensorRT, OpenVINO | Model serving with quantization |
|
||||
| Models | ConvNeXt, DenseNet, ResNet, EfficientNet, Swin-V2, UNet3Plus, DeepLabV3 | Angle, inflammation, segmentation |
|
||||
| Data | PostgreSQL + PostGIS, MinIO, Redis | Relational, object, cache |
|
||||
| Knowledge | pgvector, ladybugDB, GemmaE2B/MedGemma, EmbeddingGemma, BioClinicalBERT | RAG (pgvector HNSW), ontology (ladybugDB), Vietnamese/clinical LLM, 768-dim RAG embeddings, BERT drift/referee |
|
||||
| Observability | Prometheus, Grafana | Metrics and dashboards |
|
||||
| Code Hosting | Self-hosted GitLab CE (cloud VM, NFR-16a exception) | Source control, issue tracking, merge requests, container registry |
|
||||
| CI/CD | Jenkins on hospital K3s | Build, test, deploy pipeline; connects to cloud-hosted GitLab |
|
||||
|
||||
---
|
||||
|
||||
## 13. Cross-References
|
||||
|
||||
| Document | Relevance |
|
||||
| --- | --- |
|
||||
| [SOLUTION_ARCHITECTURE_SPEC.md](../SOLUTION_ARCHITECTURE_SPEC.md) | Main architecture spec, NFR definitions, pattern citations, trade-off analysis |
|
||||
| [Backend Specification](CODEBASE/backend/spec/backend-spec.md) | FastAPI server internal design, API contracts, RAG coordinator |
|
||||
| [Knowledge Stack Specification](CODEBASE/knowledge/spec/knowledge_spec.md) | Qdrant/ladybugDB schema, embedding models, LLM endpoints |
|
||||
| [CI/CD Deployment Pipeline](Design_Material/CI_CD_docs/CI_CD_DEPLOYMENT_PIPELINE.md) | Jenkins pipeline, Docker Compose runtime, offline bundles, rollback |
|
||||
| [DATA_SPEC.md](CODEBASE/data/spec/data_spec.md) | Data contracts, schema definitions |
|
||||
| [CONTEXT_VISION_SCOPE.md](../../PROJ_LEVEL_READING/PLAN/CONTEXT_VISION_SCOPE.md) | Project vision, sprint timelines, user personas |
|
||||
|
||||
---
|
||||
|
||||
## 14. Design Decisions
|
||||
|
||||
| # | Decision | Alternatives Considered | Rationale |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | Triton for model serving | TorchServe, FastAPI direct | Triton supports ONNX/TensorRT ensembles, dynamic batching, and concurrent model execution |
|
||||
| 2 | K3s over Docker Compose/Swarm/Nomad/ECS/Cloud Run | Docker Compose, Docker Swarm, Nomad, ECS Fargate, Cloud Run | NFR-16 requires on-premise, eliminating cloud-only platforms (ECS, Cloud Run). Docker Swarm is in maintenance mode with highest migration cost to production. K3s is already production-grade; scaling to N hospitals is multi-cluster federation, not platform replacement. |
|
||||
| 3 | MinIO over AWS S3 | Direct filesystem, NFS | S3-compatible API allows future cloud migration; object versioning built-in |
|
||||
| 4 | pgvector over Qdrant/Pinecone | Qdrant, Pinecone, Weaviate | Postgres already deployed; pgvector adds zero infrastructure overhead. At ~15K MOH guideline vectors, HNSW query latency (~5-20ms in shared_buffers) fits within NFR-7 budget. Qdrant advantage appears at millions of vectors or >500 QPS; not needed at PoC scale. Phase 2: introduce Qdrant if corpus exceeds ~100K vectors. |
|
||||
| 5 | ladybugDB embedded | Separate graph DB service | Embedded C++ library reduces latency; no separate process to manage |
|
||||
|
||||
---
|
||||
|
||||
## 15. Open Questions / Future Work (Post Sprint 2)
|
||||
|
||||
| ID | Question | Target Sprint |
|
||||
| --- | --- | --- |
|
||||
| Q1 | How to handle concurrent clinician sessions with shared Triton GPU? | Sprint 3 |
|
||||
| Q2 | Should model weights be versioned in PostgreSQL for A/B testing? | Sprint 4 |
|
||||
| Q3 | How to integrate DICOM C-STORE for automatic PACS ingestion (currently C-MOVE only)? | Sprint 5 |
|
||||
| Q4 | Will GemmaE2B be swapped for MedGemma or Willa when hospital provides GPU cluster? | Sprint 5 |
|
||||
| Q5 | Should Prometheus scrape endpoints be authenticated for NFR-17 audit compliance? | Sprint 3 |
|
||||
Reference in New Issue
Block a user